Vector v0.22.2 release notes

The Vector team is pleased to announce version 0.22.2!

Note this release does include one potentially impactful behavior change to fix a bug in the internal_logs source. The internal_logs source now, as was intended, applies the same rate limiting logic applied to logs that Vector would write to stderr. This means, rather than seeing each log individually, duplicate logs in small time windows are rolled up and logged like:

{"host":"consigliere","message":"Internal log [function call error for "parse_json" at (14:34): unable to parse json: trailing characters at line 1 column 8] has been rate limited 282948 times.","metadata":{"kind":"event","level":"ERROR","module_path":"vrl_stdlib::log","target":"vrl_stdlib::log"},"pid":2064943,"source_type":"internal_logs","timestamp":"2022-06-14T22:52:50.329188616Z","vector":{"component_id":"transform0","component_kind":"transform","component_name":"transform0","component_type":"remap"}}
{"host":"consigliere","internal_log_rate_secs":1,"message":"function call error for "parse_json" at (14:34): unable to parse json: trailing characters at line 1 column 8","metadata":{"kind":"event","level":"ERROR","module_path":"vrl_stdlib::log","target":"vrl_stdlib::log"},"pid":2064943,"source_type":"internal_logs","timestamp":"2022-06-14T22:52:50.329201656Z","vector":{"component_id":"transform0","component_kind":"transform","component_name":"transform0","component_type":"remap"},"vrl_position":54}
{"host":"consigliere","message":"Internal log [function call error for "parse_json" at (14:34): unable to parse json: trailing characters at line 1 column 8] is being rate limited.","metadata":{"kind":"event","level":"ERROR","module_path":"vrl_stdlib::log","target":"vrl_stdlib::log"},"pid":2064943,"source_type":"internal_logs","timestamp":"2022-06-14T22:52:50.329212676Z","vector":{"component_id":"transform0","component_kind":"transform","component_name":"transform0","component_type":"remap"}}

This change was made to bring internal_logs in line with Vector’s logging and avoid unexpected high bills from ingestion in downstream systems when a catastrophic failure occurs.

Note: Please see the release notes for v0.22.0 for additional changes if upgrading from v0.21.X. In particular, the upgrade guide for breaking changes.

Upgrading Vector

When upgrading, we recommend stepping through minor versions as these can each contain breaking changes while Vector is pre-1.0. These breaking changes are noted in their respective upgrade guides.

Changelog

2 bug fixes

The disk buffer max record size was increased to match the max data file size of 128 MB. This makes it much less likely that a panic will be hit due to large record batches coming in. We have work planned to make this more robust by splitting up batches larger than the data file size.
The internal_logs source now correctly applies the same log rate limiting that is applied to logs Vector logs to stderr.

Vector v0.22.2 release notes

Changelog

2 bug fixes

Download Version 0.22.2