liveBook · Manning

Custom parser

As explained in Logs and Telemetry

A custom parser is a parser defined with user-specified expressions to extract structured fields from input content.

A custom parser uses user-defined expressions to extract structured fields from content that a standard parser does not fully capture. In Fluent Bit, define it with format regex and apply it through a parser filter. section 6.7

A practical multi-parser pipeline can:

  • Apply another parser to the extracted IPv4 field and retrieve octet1. section 6.7

Extracting nginx fields

Define myNginxSubset with a regex parser, then run the simulator and Fluent Bit configuration. Output can contain octet1, IPv4, agentType, remote, method, path, code, and size.

How Logs and Telemetry covers it

How deep it goes
Demonstrates regex definitions, ordered filters, preserved data, runnable commands, and resulting structured output.
Examples
Uses nginx access logs to extract IPv4, agentType, and the first IPv4 octet.
What it assumes
Assumes log parsing and regular expressions while demonstrating the pipeline from standard nginx parsing to dependent custom parsers.
What you can do after
Build it from scratch by defining regex parsers and applying them in an ordered Fluent Bit configuration.
How it connects
Connects regex parsing with parser filters, nginx parsing, field extraction, and IPv4 octet processing.
Cover of Logs and Telemetry
liveBookmanning.com
Read Logs and Telemetry on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Logs and Telemetry. The section links point to the parts of the book it draws on.