Vector v0.17.0 release notes

The Vector team is pleased to announce version 0.17.0!

Be sure to check out the upgrade guide for breaking changes in this release.

This release includes:

It also contains a number of additional enhancements and bug fixes. Check out the highlights and changelog for more details.

Known issues

  • events_out_total and processed_events_total ceased being published by sources and transforms. This will be fixed in a subsequent release, but these metrics have also been replaced by component_sent_events_total which is being implemented for all components.
Upgrading Vector
When upgrading, we recommend stepping through minor versions as these can each contain breaking changes while Vector is pre-1.0. These breaking changes are noted in their respective upgrade guides.

Highlights

Changelog

46 enhancements

vrl add undefined variable compiler error 8910 sinks Use variance calculation to better adapt to RTT times in ARC 8325 host_metrics source Add initial cgroups metrics 8806 sinks Make adaptive concurrency the default 8598 blackhole sink print on interval instead of after X events 8968 reduce transform Add additional merge strategies 8559 loki sink Define events 8958 pipelines rename ComponentId to ComponentKey 8955 loki sink Add logfmt encoding 8983 vrl Add function for retrieving deduplicating an array 9012 journald source Add include_matches/exclude_matches options 8974 humio_logs sink humio_metrics sink Add index and indexed_fields options to Humio sink 8981 enriching add between date search to enrichment tables 9021 aws_kinesis_firehose sink Check batch configuration limits 9088 enriching add find_enrichment_table_records VRL function 9069 vrl Added case sensitive option to enrichment VRL functions 9085 enriching Add select parameter to enrichment VRL functions 9102 releasing Add section for debian package 9164 codecs kafka source Integrate Decoder/DecodingConfig with kafka source 9071 cli Write logs to stderr 9166 enriching error if enrichment functions reference a non existent column 9174 cli Fix API client subscription drop 9250 vrl allow statements in if predicates over multiple lines. 9177 external docs Serve metadata as JSON 9276 observability file source Improve file source observability 9205 observability sinks Emit events for sent events in all HttpSink sinks 9275 kafka sink add header support to kafka sink 9171 vrl Move rate limiting span to just around log function 9294 enriching enrichment functions need to specify date conditions 9296 vrl Add tally function 9015 unit tests Add testing for component specification features 9312 vrl Allow to_float to take timestamps 9373 vrl added reverse_dns remap function 8717 external docs Update unit testing docs to cover VRL conditions 9377 codecs exec source Integrate Decoder/DecodingConfig with exec source 9337 external docs Link to Helm chart repo 9396 dev Added check-build-tools to Makefile (#8146) 8147 graphql api Refactor outputEvents subscription -> outputEventsByComponentIdPatterns 9339 prometheus_scrape source Allowing tagging metrics with instance and endpoint 9330 datadog_logs sink Set DD origin headers 9367 sources observability Add events to HttpSource framework 9379 codecs exec source Integrate Decoder/DecodingConfig with nats source 9400 codecs datadog_agent source Integrate Decoder/DecodingConfig with datadog source 9430 vrl Add default_known_ports parameter to parse_url function 9376 codecs aws_kinesis_firehose source Integrate Decoder/DecodingConfig with aws_kinesis_firehose source 9332 aws_ec2_metadata transform dont error when receiving a 404 when fetching metadata 9492

13 new features

pipelines create WIP pipeline parser 8572 pipelines compiler verification 8810 pipelines add –pipeline-dir argument to cli 9002 pipelines add component_scope field to internal metrics 9051 new sink Add ‘splunk_hec_metrics’ sink 8947 vrl add find function 9090 new sink Add ‘datadog_archives’ sink with S3 support 8929 config handle BufferConfig parsing without a type tag 9151 kubernetes_logs source add pod annotations 9180 datadog_logs sink use v2 endpoint for logs 9277 datadog_agent source support v1 and v2 log apis 9176 vrl dynamic path functions 9284 aws service add support to use AWS credentials file for AWS authentication 9060

48 bug fixes

releasing Add enrichment tables to all builds 8939 css website github icon should have hover effect in dark mode. 8964 splunk_hec source Do not blow up on trailing whitespace 8997 vrl handle complex ruby objects 8956 external docs Add template note to config params 9032 config error on duplicate inputs 9011 ci Regenerate intermediate test CA 9053 external docs Fix component docs rendering nits 9074 config ensure components don’t have dots in their names 9083 aws_cloudwatch_logs sink Allow adaptive concurrency 9007 ci Put collects back 9097 tests work around internal_logs lack of isolation 9099 tests Fix smoke test first line function for e2e kubernetes tests 9096 enriching Usable in Test 9095 aws_s3 sink Fix batching parameters 9106 ci Overwrite wix install if exists 9116 aws service more precise aws retry logic 9104 architecture use a fixed vec allocation for batches 9128 prometheus_scrape source allow : in metric names 9123 vrl Re-add VRL posiition to log() calls 8995 ci Use use_v2_api: false for k8s datadog agent tests 9155 http source adjust deflate compression handling 9121 networking reorder service layers to prevent unbounded retries 9026 vrl Add ipv6 support to parse_aws_alb_log.rs 9163 aws_s3 sink ship final batches when shutting down 9184 host_metrics source Detect cgroup memory controller before scanning 9168 cli Fix top quitting when the host has gone away 9139 datadog_archives sink Correct encoding tags 9252 external docs Update GitHub org name throughout the website 9249 external docs Add source/edit buttons to docs and guides 9248 vrl fix byte-swapped integers in ip_aton() and ip_ntoa() functions 9208 cli Fix tap signal handling 9254 releasing Use bash rather than sh for script installer 9266 external docs Improve VRL function templating 9263 ci Re-add markdown and cue checks 9285 external docs Change sidebar ordering of VRL and config docs 9271 transforms add sanity check in route & compound transforms 9301 docs add missing word 9359 vrl reverse if else alternatives before nesting 9363 external docs Provide the correct optionality for the VRL assert message 9384 generator source Give the generator source a default interval 9390 buffers Use tokio runtime in buffer benchmarks 9391 external docs Fix configuration reference 9397 codecs Propagate empty byte frames in message based transports 9399 datadog_logs sink always send unix timestamp in milliseconds 9436 datadog_logs sink expect ACCEPTED(202) instead of OK(200) response from Logs v2 API 9437 vrl Correct parse_url’s value type 9473 sink loki return empty json when buffer is empty 9464

What’s next

Component metric standardization
We are in the process of ensuring that all Vector components report a consistent set of metrics to make it easier to monitor the performance of Vector. These metrics are outlined in this new instrumentation specification).
VRL iteration support
A common request from users when the incoming log event shape is unknown, to be able to iterate over the keys and values in those log events. We recently published an RFC for this and expect to implement this support this quarter.
New unified Vector helm chart
Taking all of our learnings from the existing agent and aggregator Helm charts, we’ve started work on a new Helm chart that will unify both the agent and aggregator deployments into one chart, which can be configured by deployment role, to simplify deployment and chart maintenance. You can follow along in vectordotdev/helm-charts#38.

Download Version 0.17.0

Linux
tar.gz
ARM64 ARMv7 x86_64
Linux (deb)
deb
ARM64 ARMv7 x86_64
Linux (rpm)
rpm
ARM64 ARMv7 x86_64
macOS
tar.gz
x86_64
Windows
zip
x86_64
Windows (MSI)
msi
x86_64
browse all files...