The Fluent Bit Modify filter transforms event records after ingestion. It conditionally adds, sets, renames, copies, removes, and reorders record elements, making it useful when a payload needs enrichment, cleanup, or a different shape before it is routed onward. section 7.1 section 7.3
Use `condition` to make transformations apply only to records that pass a Boolean test. Conditions can check whether a key exists or is absent, match key names with regular expressions, compare values, and test values against regular expressions. Without a positive condition result, the filter does not execute its configured operations. section 7.3
The filter supports these record-level operations:
- `add` adds an element, while `set` adds or replaces it; choose `set` when the target value must be present even if the key already exists. section 7.3 section 7.3.3
- `rename` and `copy` change or duplicate elements. `hard_` variants force the operation even when the target element already exists. section 7.3
- `remove` deletes unwanted elements, helping reduce payload content before it is sent across data centers or clouds. section 7.3
- `move_to_start` and `move_to_end` reorder attributes within the event record. section 7.3
Trim records from known IP origins
A CheckList filter can first mark matching events with `known_origin: yes`. A record_modifier filter can add `hostname` and a UUID to every event. The Modify filter then runs only when `known_origin` exists: it removes `agent`, `request`, and `referrer`; renames `remote_user` to `user`; and sets `identifiedValue` to `matched`. Events without the marker retain their original request details, while matched events become shorter and carry the renamed and added fields. section 7.3.3 section 7.3.4
Listing 7.2 CheckList and modify: ip-checklisted-modifying-filter.conf
[SERVICE]
flush 1
parsers_file ${FLUENTBIT_HOME}/conf/parsers.conf
[INPUT]
name tail
path ./nginx.json.log
read_from_head true
tag nginx-feed
parser json
[FILTER]
name checklist
match nginx-feed
file ./fluentbit/ip-checklist.data
lookup_key remote_ip
ignore_case false
mode partial
record known_origin yes
print_query_time false #1
[FILTER]
name record_modifier #2
match nginx-feed
record hostname ${HOSTNAME} #3
uuid_key myUUID #4
[FILTER]
name modify
match nginx-feed
condition Key_Exists known_origin #5
remove agent request #6
remove referrer #7
rename remote_user user #8
set identifiedValue matched #9
[OUTPUT]
name stdout
match *
This configuration shows the ordering of CheckList, record_modifier, and Modify filters, plus a runnable `tail` input and stdout output. Inspect the `condition Key_Exists known_origin` line before the remove, rename, and set directives. section 7.3.4
Modify is distinct from record_modifier: record_modifier supports allowlists, removal, added records, and UUID generation, whereas Modify supplies conditional processing and additional rename, copy, and reorder behavior. For potentially incomplete parsed events, a `key_exists` condition can also check for populated attributes before further handling. section 7.3 section 6.8.2
