liveBook · Manning

Ltsv

As explained in Logs and Telemetry

LTSV is a log format that represents records as label-value fields separated by tabs.

LTSV (Label Tab-Separated Value) represents each record as labeled fields separated by delimiters. In Fluent Bit, it can format output files and parse incoming log lines into an internal JSON representation. section 5.5 section 6.9.2

Configure LTSV according to the direction of the data flow:

  • For file output, set format to ltsv; use delimiter for field separation and label_delimiter for separating labels from values. section 5.5
  • For parsing, set format to ltsv, identify the event timestamp with time_key and time_format, and declare field types with types. section 6.9.2
  • Declare types such as integer, float, bool, hex, or string so parsed values receive appropriate internal representations. section 6.9.2

Parsing a timestamped LTSV record

The line `line:1 widget:gadget aKey:aValue time:[16/Jun/2024:14:55:01 +0000]` can be parsed with `time_key time`, `time_format [%d/%b/%Y:%H:%M:%S %z]`, and `types line:integer widget:string aKey:string`. The resulting record contains typed fields such as `line`, `widget`, and `aKey`.

Listing 5.5 Different file formats: simple-file.conf
[SERVICE]
  flush 1
  log_level error

[INPUT]
  name http    
  listen 0.0.0.0
  port 9881
  tag http-in

[OUTPUT]        #1
  match *
  name stdout
  json_date_key received-date
  json_date_format iso8601
  format json_lines




[OUTPUT]
  match *
  name file       #2
  file output1.csv          #3
  format csv        #4
  delimiter \t      #5

[OUTPUT]
  match *
  name file
  file output1.ltsv
  format ltsv          #6
  label_delimiter space       #7
  delimiter comma           #8

[OUTPUT]
  match *
  name file
  file output1.txt
  format plain

[OUTPUT]
  match *
  name file
  file output1.msgpack
  format msgpack           #9

[OUTPUT]
  match *
  name file
  format template           #10
  file output1.templated.txt
  template addressing={hello} value={val} --- {time}     #11

Compare LTSV output with CSV, plain text, JSON Lines, MessagePack, and templated output in one Fluent Bit configuration. section 5.5

How Logs and Telemetry covers it

How deep it goes
Explains delimiters, timestamp parsing, and value typing through Fluent Bit configurations, but does not detail the parser implementation.
Examples
Uses a timestamped LTSV line, a Fluent Bit parser configuration, and a multi-format file-output configuration.
What it assumes
Assumes familiarity with Fluent Bit configuration and structured log records while demonstrating LTSV syntax from concrete examples.
What you can do after
Operate/configure Fluent Bit to emit LTSV files or parse LTSV lines with timestamps and typed fields.
How it connects
Connects LTSV with CSV, JSON, regular-expression parsing, MessagePack, and templated output.
Cover of Logs and Telemetry
liveBookmanning.com
Read Logs and Telemetry on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of Logs and Telemetry. The section links point to the parts of the book it draws on.