Vector v0.28.0 release notes
The Vector team is pleased to announce version 0.28.0!
This is a smaller maintenance release primarily including bug fixes and small enhancements, while we do some background work to enable upcoming new features.
With this release we also completed an initiative to generate Vector’s reference documentation from the configuration structures in the code which will result in less inaccuracies in published documentation.
Be sure to check out the upgrade guide for breaking changes in this release.
Upgrading Vector
When upgrading, we recommend stepping through minor versions as
these can each contain breaking changes while Vector is pre-1.0.
These breaking changes are noted in their respective upgrade
guides.
- AWS components, except the
aws_s3sink, are not functional due to issues with request signing. This is fixed in v0.28.1. - The
framing.*.max_lengthconfiguration options cannot be used on thesocketsource as Vector returns an error about them conflicting with the deprecated top-levelmax_lengthconfiguration option. This is fixed in v0.28.1. - The
http_serversource incorrectly defaults toGETrather thanPOSTformethod. Fixed in 0.28.2. - The
elasticsearchsink panics when starting ifbulk.indexis unspecified and the defaultmodeofbulkis used. Fixed in 0.28.2. - The
syslogsource incorrectly inserted the source IP of incoming messages assource_idrather thansource_ip. Fixed in 0.28.2.
12 enhancements
- The
azure_blobsink now allows setting a customendpointfor use with alternative Azure clouds like USGov and China. Thanks to archoversight for contributing this change! - The
clickhousesink now supports adate_time_best_effortconfig option to have ClickHouse parse a greater variety of timestamps (like RFC3339). Thanks to DarkWanderer for contributing this change! - The
httpsink now supportspayload_prefixandpayload_suffixoptions to prepend and append text in the HTTP bodies it is sending. This happens after batches are encoded and so can be used, for example, to wrap the batches in a JSON envelope. Thanks to jdiebold for contributing this change! - The
aws_kinesis_firehosenow has astore_access_key, similar to thesplunk_hecanddatadog_agentsources, to store the token that the incoming request was sent with in the event secrets. This can be read later in VRL to drive behavior. Thanks to dizlv for contributing this change! - The
reducetransform now has amax_eventsoption that can be used to limit the total number of events in a reduced batch. Thanks to jches for contributing this change! - The
kafkasource now tries to commit offsets during shutdown to avoid duplicate processing on start-up. Thanks to aholmberg for contributing this change! - Vector no longer panics when attempting to create more than 254 allocation groups for memory allocation tracking. This would happen when there were more than 254 components in a config or during unit tests where a lot of components are spun up independently.
- The
reducetransform performance improved by only flushing when events were ready to be flushed and avoiding repeated checks for stale events. Thanks to dbcfd for contributing this change! - A
seahashfunction was added to VRL for a non-cryptographic fast hash. Thanks to psemeniuk for contributing this change! - The
pulsarsink now supports batching via the addedbatch.max_eventsconfiguration option. Thanks to zamazan4ik for contributing this change! - The
kafkasource is now capable of making consumer lag metrics available via theinternal_metricssource. This can be enabled by settingmetrics.topic_lag_metrictotrue. Note that this can result in high cardinality metrics given they are tagged with the topic and partition id. Thanks to zamazan4ik for contributing this change! - The
redissource now retries failed requests with an exponential back-off rather than immediately (which can cause high resource usage). Thanks to hargut for contributing this change!
3 new features
encode_gzipanddecode_gzipfunctions were added to VRL to interact with gzip’d data. Thanks to zamazan4ik for contributing this change!encode_zlibanddecode_zlibfunctions were added to VRL to interact with zlib compressed data. Thanks to zamazan4ik for contributing this change!encode_zstdanddecode_zstdfunctions were added to VRL to interact with zstd compressed data. Thanks to zamazan4ik for contributing this change!
7 bug fixes
- Vector reload better detects which components have changed, no longer reloading components that have configuration options that are maps where none of the keys or values changed. Thanks to aholmberg for contributing this change!
- Loading of secrets in configuration now allows the same secret to be used multiple times. Previously this would result in Vector raising an error that the secret was undefined on the second or later uses.
- Changing
max_line_byteson thefilesource no longer typically invalidates all previous checksums. It still will invalidate the checksum if the value is set to lower than the length of the line used that checksum but this should be much less common. Thanks to Ilmarii for contributing this change! - Vector GCP sinks now correctly refresh authentication tokens when healthchecks are disabled. Thanks to punkerpunker for contributing this change!
- VRL now correctly calculates the type definitions of boolean expressions that short-circuit. This should avoid panics and the inability to write what should be valid VRL programs in rare situations.
- The
gcp_stackdriver_metricsnow correctly encodes metrics to send to GCP. Thanks to jasonahills for contributing this change! - The
axiomsink now always sets thetimestamp-fieldheader, which tells Axiom where to find the timestamp, to@timestamprather than the configuredlog_schema.timestamp_keysince Vector was always sending it as@timestamp.the upgrade guide for details.
Sign up to receive emails on the latest Vector content and new releases
Thank you for joining our Updates Newsletter