Remap support for the reduce
transform (multi-line logs)
Use VRL to specify conditions for reducing multi-log lines into a single log event
The reduce
transform for Vector enables you to reduce multiple log
events into a single event, which is useful for logs that are split across
multiple lines, such as JVM stack traces. With reduce
you can specify both a
merge strategy for the events and a condition for specifying when a group of
events either starts or ends (that group is then collapsed, i.e. reduced into
a single event).
Previously, the reduce
transform required you to specify conditions using
check_fields
. But now you can use Vector Remap Language expressions in the
remap
transform to specify those conditions. This should make using
reduce
more natural and expressive.
Example
The example configuration below shows the same reduce
transform using the old
system (check_fields
) and the new system (remap
):
[transforms.merge_stack_trace]
type = "reduce"
inputs = ["jvm-logs"]
merge_strategies.message = "concat_newline"
# Using check_fields
-starts_when.type = "check_fields"
-starts_when.message.regex = "/^\\w.*/"
-starts_when.severity.eq = "info"
# Using remap
+starts_when.type = "remap"
+starts_when.source = 'match(string!(.message), r'^\\w.*') && .severity == "info"'