fluentd is an amazing piece of software but can sometimes give one a hard time. Case in point, how can one add a field to an output only if a certain string exists in another record.
Turns out with a little regex, it’s decently simple.
The below code will add a field called “_newfield” with value “OURSTRING” when the “log” record contains the exact same string at the beginning of the line. The regex can be further adjusted as needed.
<filter **> @type parser key_name log reserve_data true emit_invalid_record_to_error false <parse> @type regexp expression /^(?<_newfield>^OURSTRING)/ </parse> </filter>
“emit_invalid_record_to_error false” will avoid sending error logs if the regex does not match while “reserve_data true” will preserve the initial message and just add a field to it before passing it out further.