The Vector team is pleased to announce version 0.19.0!
In addition to the below features, enhancements, and fixes, we’ve been
hard at work improving Vector’s performance and were able to move the
needle 10-100% for most configurations in our soak test
framework
from the last release, v0.18
.
Be sure to check out the upgrade guide for breaking changes in this release.
v0.18.0
, if there was any data in existing disk buffers, it will not be sent. Vector’s starting with clean or empty disk buffers are unaffected. Fixed in v0.19.1. See #10430 for more details.framing.character_delimited.delimiter
in configs causes Vector to fail to start with invalid type: string
. Fixed in v0.19.1.decoding.codec
on sources, invalid data will cause the source to cease processing. Fixed in v0.19.2.encoding.only_fields
failed to deserialize correctly for sinks that used fixed encodings (i.e. those that don’t have encoding.codec
). Fixed in v0.19.2.when_full
of block
were incorrectly counting buffer_events_total
by including discarded events. Fixed in v0.19.2.component_id
). Fixed in v0.19.2.aws_s3
source.VRL now allows for writing multi-line string literals by ending the line with a backslash (\).
Example:
.thing = "foo \
bar"
assert(.thing == "foo bar")
A couple of enhancements have been made to the influxdb_logs
sink to modify how Vector encodes events.
A measurement
config field was added to allow overriding the
measurement name (previously hardcoded to vector
). Along with
this the namespace
option was made optional as measurement
can be used to set the full name directly.
For example:
[sinks.log_to_influxdb]
type = "influxdb_logs"
measurement = "vector-logs"
endpoint = "http://localhost:9999"
Now outputs events like:
vector-logs,metric_type=logs,host=example.com message="hello world",size=10 {timestamp}
A metric_type
config field was added to allow customizing the
metric type (previously hardcoded to log
).
For example:
[sinks.log_to_influxdb]
type = "influxdb_logs"
measurement_type = "foo"
endpoint = "http://localhost:9999"
Now outputs events like:
ns.vector,metric_type=foo,host=example.com message="hello world",size=10 {timestamp}
kubernetes_logs
source has been updated to read older pod
logs first. This should result in better behavior with Vector
releasing file handles for rotated pod files more quickly..
s in the variable names as
these commonly appear in environment variables set by Java
properties files.VRL has added new functions for interacting with event metadata.:
get_metadata_field("key")
set_metadata_field("key", "value")
remove_metadata_field("key")
Right now, the only event metadata that is accessible is Datadog
API keys (datadog_api_key
) or Splunk HEC channel tokens
(splunk_hec_token
) that are associated by the source, but we
expect metadata use-cases to grow.
This can be used with, for example, CSV enrichment tables to
lookup the datadog_api_key
to use with events based on other
metadata.
statsd
sink now compresses histograms to result in smaller
payloads without data loss.blackhole
sink when a rate limit
lower than 1024
was used.splunk_hec
source and sink have added support for the
acknowledgement part of Splunk’s HTTP Event Collector protocol.
This improves delivery guarantees for data from Splunk clients
and when sending events to Splunk. See the highlight
article for
more details.datadog_agent
source is now able to accept metrics from
the Datadog Agent; however, some changes are pending in the
Datadog
Agent to be
able to send metrics to Vector. We expect this to be released in
version 6.33 / 7.33 of the Datadog Agent.humio_logs
sink now transmits sub-millisecond timestamps
to Humio.splunk_hec
source and sink have added support for passing
the channel token events were sent with from the source to the
sink. This makes it easier to use Vector in-between a Splunk
sender and receiver to transform the data. See the highlight
article for
more details.elasticsearch
sink
to allow suppressing the type
field from being sent by Vector.
This field is deprecated in Elasticsearch v7 and will be removed
in v8.connection_limit
option has been added to TCP-based
sources like socket
and syslog
to limit the number of
allowed TCP connections. This can be useful to limit resource
utilization by Vector.headers_key
config option for the kafka
source was
restored. This was accidentally renamed to headers_field
in
v0.18.0. For compatibility with v0.18.0, headers_field
will
also be accepted.loki
sink now accepts any 200-level HTTP response from
servers as success. This was added for compatibility with other
Loki-compatible APIs like cLoki which didn’t respond with the
expected 204 response code.host
and pid
fields are correctly added to all internal
logs now. Previously they were only added to start-up logs, but
not logs while Vector was running.--config-dir
, no longer tries to load
unknown file extensions. This was a regression in v0.18
.max_length
option available on some decoding
framers for
sources previously caused Vector to stop decoding a given input
stream (like a TCP connection) when a frame that was too big was
encountered. It now correctly just discards that frame and
continues.framing.character_delimited.delimiter
option
available on some sources allowed for characters greater than
a byte, but the implementation assumed the delimiter was only
one byte. Vector now correctly errors if a delimiter that is
greater than a byte is used. Only byte delimiters are allowed
for efficiency in scanning.Sign up to receive emails on the latest Vector content and new releases
Thank you for joining our Updates Newsletter