liveBook · Manning

Flink SQL

As explained in Apache Kafka in Action

Flink SQL is Apache Flink's SQL interface for defining continuously evaluated queries over streaming and table-oriented data.

Flink SQL defines continuously evaluated queries over streaming data. Instead of returning one final answer, a query produces an ongoing stream of changes. Tables provide a convenient relational view of state, while connected Kafka topics supply the underlying events. section 12.3

A typical Flink SQL workflow connects event topics to tables and then composes relational operations:

  • Create a table whose schema describes the event fields and whose connector configuration identifies the Kafka topic, bootstrap server, and JSON format. section 12.3
  • Filter records, such as selecting products whose price exceeds 100. section 12.3
  • Aggregate changes by key, such as counting price changes for each product. section 12.3
  • Split larger flows into views, then join notification events with each user's currently active devices. section 12.4.4

Routing notifications to active devices

An active-device view groups device records by user and collects active device details. A second query joins notifications with that view, producing notification records enriched with the recipient's current device list. The resulting stream can be written to another Kafka topic. section 12.4.4

CREATE TABLE productprices (
  product_id INT,
  product_name STRING,
  price FLOAT,
  oldPrice FLOAT
) WITH (
  'connector' = 'kafka',
  'topic' = 'products.prices.changelog',
  'properties.bootstrap.servers' = 'localhost:9092',
  'format' = 'json',
  'key.format' = 'json',
  'key.fields' = 'product_id',
  'value.fields-include' = 'EXCEPT_KEY'
);

Use the table definition to inspect how a Kafka topic becomes a typed Flink SQL table, including keys, connector settings, and JSON serialization. section 12.3

How Apache Kafka in Action covers it

How deep it goes
Explains table connections, filters, aggregations, views, and joins through examples, but defers detailed SQL syntax.
Examples
Uses product price filtering, price-change counting, and notification enrichment with active user devices.
What it assumes
Assumes familiarity with SQL-like queries and Kafka topics while introducing streaming tables and continuously changing results.
What you can do after
Build it from scratch by defining Kafka-backed tables and composing continuously evaluated filters, aggregations, views, and joins.
How it connects
Connects SQL with Kafka topics, streaming tables, aggregations, joins, views, and stream processing.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Apache Kafka in Action. The section links point to the parts of the book it draws on.