Chapter 11: Building the qualitative engine with news analysis and LLMs

Introduction

The introduction explains why financial news can complement market data and frames the proposed qualitative engine as a grounded research aid rather than an automated trader.

• Limits of numerical signals: Explain why the chapter adds qualitative analysis to the quantitative engine: price and volume features capture numerical patterns, but important shifts in narratives, fears, and expectations may not appear in the numbers until later.
  Why it matters: It explains why news analysis adds something the existing market-data model may miss.
• Qualitative engine research role: Describe the intended role of the qualitative engine as an LLM-powered research associate that turns sourced financial news into structured signals, including emerging narratives and how quickly attention changes. Distinguish that role from an oracle or automated trader, and explain why its signal complements the quantitative one.
  Why it matters: It sets the intended use and limits of the news-derived signal before the chapter builds it.

Key terms: quantitative engine, qualitative events, narratives, qualitative engine, AI-powered research associate, Retrieval-Augmented Generation (RAG), velocity of attention, hybrid strategy

11.1 The new playbook: From market numbers to market narratives

This section frames LLMs as aids to investment judgment rather than replacements for it.

• LLMs and investment judgment: Explain why using LLMs in investment work calls for skepticism and an understanding of their strengths and limitations, rather than treating them as substitutes for human judgment.
  Why it matters: This stance keeps the promise of LLMs from becoming a reason to surrender investment judgment.

Key terms: LLMs, investment, judgment, limitations, strengths

11.1.1 The hard truth: A reality check on LLMs in investment research

This section explains why LLMs can help synthesize investment information but cannot be trusted as precise fact-retrieval tools or as proof of a predictive edge.

• Probabilistic analysis paradox: Explain why an LLM may synthesize lengthy filings quickly yet struggle to extract a few exact financial figures, and connect that contrast to how it generates answers.
  Why it matters: Fluent synthesis is not evidence that an exact figure has been retrieved correctly.
• Financial fact-checking limits: Describe the source, numerical, and hallucination risks that make unverified LLM answers unsuitable for precise investment analysis.
  Why it matters: A convincing answer can still be unsafe to use as an investment fact or calculation.
• Temporal leakage in signals: Explain how information absorbed during pretraining can make an investment signal appear predictive, and state why RAG does not fully resolve that risk.
  Why it matters: An apparently successful signal may reflect knowledge of the outcome rather than genuine foresight.

Key terms: Moravec’s paradox, probabilistic, deterministic, fact-checkers, hallucinate plausibly, human oversight, temporal leakage, RAG grounding, hypothesis, edge

11.1.2 Our philosophy: The LLM as a probabilistic analyst, not a deterministic oracle

This section positions the LLM as a first-pass analyst for news narratives and defines the qualitative engine that turns those narratives into probabilistic signals for human review.

• LLM and human judgment: Explain why the LLM is treated as a probabilistic analyst rather than a fact-checker or final decision-maker, and how its first-pass synthesis leaves judgment to the human analyst.
  Why it matters: It clarifies which work the LLM speeds up and which decisions people must still make.
• Qualitative engine signals: Describe what the qualitative engine extracts from news and how that differs from the quantitative engine’s use of prices. Give an example of a narrative change it could assess.
  Why it matters: It shows how news can provide a different kind of investment signal from market-price data.

Key terms: probabilistic analyst, first-pass analysis, human analyst, final arbiter of judgment, qualitative engine, quantitative engine, retrieval-augmented workflow, narrative signals, probabilistic signals

11.1.3 The strategist’s hunt for alpha: Where qualitative signals shine

The section explains how to choose narrative-driven assets for qualitative signals, using uranium as an example while emphasizing probabilistic interpretation and auditability.

• Narrative-Driven Asset Selection: Explain why the qualitative engine targets assets whose prices respond to news and uncertain future events rather than chiefly to stable, quantifiable fundamentals.
  Why it matters: Choosing an asset whose outlook changes with the news gives qualitative analysis a useful role.
• Auditable Signal Rationale: Explain why each generated signal should retain its source text and reasoning.
  Why it matters: A traceable signal can be examined rather than accepted as a mysterious LLM judgment.
• Uranium Case Study Choice: Justify the choice of the URA uranium ETF using its news-driven catalysts, the uncertain implications of policy language, and accessible public news.
  Why it matters: URA makes the chapter’s qualitative approach concrete without treating announcements as guaranteed outcomes.
• Framework Transfer Across Themes: Describe how the same narrative-driven approach could apply to semiconductor or critical-mineral ETFs by identifying news that affects their outlook.
  Why it matters: The useful pattern is matching an investment thesis to consequential news, not memorizing one ETF.

Key terms: narratives, probabilistic events, qualitative signals, auditability, source text, reasoning process, URA, narrative-driven catalysts, probabilistic signal, language inflection

11.1.4 Designing the qualitative engine architecture

This section maps how financial news becomes a structured qualitative signal and explains why the LLM must be grounded in current, verifiable sources.

• News-to-signal pipeline: Trace the qualitative engine from financial news sources through retrieval and LLM analysis to a machine-readable signal. Distinguish its unstructured news input from the quantitative engine’s structured numbers.
  Why it matters: This flow shows how narrative information becomes a signal that can sit alongside quantitative analysis.
• Grounding in current sources: Explain why the RAG architecture supplies the LLM with retrieved financial news rather than relying on the foundation model alone.
  Why it matters: Grounding ties the LLM’s signal to current financial information that can be checked.

Key terms: qualitative signal pipeline, RAG feature engineering pipeline, vector embeddings, metadata, vector database, structured prompt template, RAG architecture, grounding, current, domain-specific data, verifiable sources

11.2 Implementing the news ingestion and processing pipeline

The section explains why a reliable news ingestion pipeline is the foundation of the qualitative engine.

• Reliable news ingestion foundation: Explain why building a robust, reliable data ingestion pipeline is the critical first step toward turning raw news into an actionable qualitative signal.
  Why it matters: Better analysis depends first on a dependable flow of information.

Key terms: qualitative engine, data ingestion pipeline, structured, actionable signal

11.2.1 The data sourcing reality: Building our own information infrastructure

The section explains why market analysis needs curated information sources rather than relying on an LLM’s training alone, and why the book uses accessible public sources despite their limitations.

• Limits of model knowledge: Explain why a foundation model cannot be relied on by itself for market answers, even when its answers sound convincing.
  Why it matters: A fluent market answer is not a substitute for giving the model reliable financial information.
• Public-source data tradeoff: Justify the choice to build a small information infrastructure from public APIs and news aggregators, while describing the quality constraint that choice creates.
  Why it matters: Source choice determines both what readers can reproduce and what information the eventual analysis can use.

Key terms: foundation model, real signal, primary-source filings, institutional context, mini information infrastructure, premium, low-latency data feeds, publicly available APIs, news aggregators, information-constrained environment

11.2.2 Our pragmatic toolkit: A four-source blend

The section explains why the qualitative engine blends four imperfect free news sources, assigning each a role according to its content and historical reach.

• Multisource news feed rationale: Explain why the pipeline blends news sources instead of relying on one, and how it combines richer recent coverage with longer-term headline history.
  Why it matters: The blend determines what evidence the qualitative engine can draw on for recent signals and historical comparisons.
• Recent summary source roles: Distinguish the near-term roles and limitations of NewsAPI and World Nuclear News RSS.
  Why it matters: The two recent feeds provide complementary descriptive content without offering unrestricted or extensive history.
• Historical headline source roles: Distinguish how Google News RSS and GDELT extend historical coverage and what their headline-only content cannot provide.
  Why it matters: These feeds extend the backtest window, but their headlines should not be mistaken for detailed article evidence.

Key terms: composite feed, multisource, multigranularity architecture, NewsAPI, World Nuclear News (WNN) RSS, daily signal generation, Google News RSS, GDELT, historical headlines backfill, headline trends

11.2.3 Implementing the fetcher functions

This section implements four complementary news fetchers that save standardized Parquet data and explains how their different content strengths and production update needs shape the qualitative engine.

• Standardized fetcher outputs: Explain how the modular fetchers hide source-specific details, return standardized pandas DataFrames, and persist their results as Parquet files. Justify Parquet using its compression, type preservation, and read/write performance.
  Why it matters: Consistent, efficient outputs let the next pipeline stage work with news collected from different services.
• Complementary source strengths: Distinguish the four sources by content richness, specialization, and historical reach, and explain why both summary-rich and headline-only sources belong in the corpus.
  Why it matters: The sources complement one another rather than offering interchangeable copies of the same news.
• Incremental production collection: Describe how the one-time educational collection would change for production: fetch only new articles on daily runs, write timestamped outputs that are safe to re-execute, and handle cross-source overlaps through downstream URL-based deduplication.
  Why it matters: Incremental, safely repeatable collection keeps a production news corpus current without losing its history.

Key terms: modular fetcher functions, standardized pandas DataFrame, Parquet, metadata, NewsAPI, WNN RSS, Google News RSS, GDELT, semantic fuel, news frequency

11.3 Building the RAG feature pipeline: From raw news to retrieval-ready vectors

This section explains why the qualitative engine turns news from disparate sources into searchable vectors that complement structured market data.

• Purpose of the RAG pipeline: Explain how the RAG pipeline makes news usable alongside quantitative financial data: it turns disparate, unstructured sources into a unified, searchable form so models can consider narratives behind market numbers.
  Why it matters: It connects market narratives to the numerical signals a model already uses.

Key terms: RAG pipeline, qualitative signals, mathematical vectors, searchable knowledge base

11.3.1 Data unification and cleaning

This section turns four differently structured news files into one clean dataset ready for embedding and retrieval.

• Source-specific news cleaning: Explain why the four Parquet sources need separate cleaning functions before they can become a single dataset.
  Why it matters: Cleaning each source appropriately helps keep downstream retrieval reliable.
• Content-first merging and deduplication: Trace how the cleaned sources are combined and how the preferred version of a duplicate story is chosen.
  Why it matters: Choosing the richer copy preserves more useful information without retaining duplicate stories.
• Retrieval-ready timestamps and fields: Explain why publication datetimes become Unix timestamps and describe what the final DataFrame provides.
  Why it matters: Numeric publication times make the news records usable for time-aware retrieval.

Key terms: Parquet files, golden source, specialized functions, concatenate, sort, deduplication, published_timestamp, Unix timestamps, metadata filtering, text_for_ embedding

11.3.2 Chunking strategy: Choosing the right granularity

This section explains why short news headlines and summaries should be embedded intact rather than split into smaller RAG chunks.

• Data-dependent chunking choice: Explain why chunking is useful for long documents but counterproductive for the short news items in this pipeline, and describe how those items are handled instead.
  Why it matters: Keeping each news item intact preserves the context that retrieval needs.

Key terms: chunking, granularity, natural chunks

11.3.3 Text embedding with OpenAI

This section explains how news snippets become searchable vectors and why consistent embeddings and batch API calls matter to the pipeline.

• Semantic meaning in vectors: Explain how an embedding model converts text snippets into vectors that support search by meaning.
  Why it matters: This is why the news pipeline turns text into vectors before searching it.
• Embedding model consistency: Justify the model choice for news ingestion and explain why production embeddings should use a pinned model version.
  Why it matters: Comparable embeddings help keep news search relevant as the dataset grows.
• Batch embedding workflow: Trace how the cleaned news dataset is embedded in batches and explain why batching is used.
  Why it matters: Batching makes embedding a large news dataset practical without changing what the pipeline saves.

Key terms: embeddings, high-dimensional vector, semantic space, text-embedding-3-small, embedding stability, semantic drift, model version, batch processing, OpenAI API, vectors

11.3.4 Vector database storage with ChromaDB

This section explains why the news embeddings are stored in a searchable vector database and why their source text and metadata must be stored alongside them.

• Focused news retrieval: Explain why the pipeline stores news embeddings in ChromaDB rather than putting all collected articles into an LLM prompt.
  Why it matters: Focused retrieval makes a growing news collection useful as context for the LLM.
• Auditable RAG storage: Describe why an enterprise-grade RAG system stores source text and metadata alongside vectors.
  Why it matters: A retrieved news item must provide both material the LLM can use and a traceable record of where it came from.

Key terms: ChromaDB, vector search, embeddings, semantically relevant articles, vectors, source texts, metadata, auditability

11.3.5 Verification: Trust, but verify the pipeline

A test query verifies that the RAG pipeline retrieves relevant articles while exposing a headline-only bias in its top results.

• Semantic Retrieval Verification: Explain how the test query checks the RAG pipeline and what its results establish about semantic search.
  Why it matters: A retrieval test checks the knowledge base before an LLM Analyst relies on it.
• Headline Similarity Bias: Explain the quality imbalance revealed by the top results, why it occurs, and how the section proposes handling it.
  Why it matters: Relevant search results can still favor thin sources, which affects how the retrieved material should be used.

Key terms: RAG pipeline, test query, semantic search, verifiable context, headline-only articles, semantic similarity, content-rich sources, weighting factor, prompt level

11.4 Building the LLM Analyst: From retrieval to quantified signals

This section defines the news-derived signals the LLM Analyst should produce and frames the LLM as an evidence-grounded information extractor rather than a predictor.

• Three qualitative news signals: Describe what policy tone, supply shock risk, and novelty each assess in retrieved news.
  Why it matters: These distinctions give each news-derived score a specific meaning.
• Evidence-grounded analyst role: Distinguish the LLM Analyst’s structured assessment of retrieved evidence from a standalone prediction of the future.
  Why it matters: Grounding each assessment in supplied evidence makes the signal interpretable rather than an unsupported forecast.

Key terms: policy tone, supply shock risk, novelty, LLM Analyst, disciplined information extractor, structured assessment, RAG

11.4.1 Retrieval strategy: A two-step process for context and precision

The section designs a two-step news retrieval strategy that gives the LLM broad market context before supplying recent, substantive articles for signal extraction.

• Broad scan and deep dive: Explain why retrieval starts with a broad trend scan and then narrows to a deep dive, including what each step retrieves and contributes to analysis.
  Why it matters: The LLM gets a sense of the wider narrative before extracting signals from detailed recent news.
• Metadata-based retrieval filtering: Describe how the retrieval strategy uses metadata filters to keep results timely and focused on substantive content.
  Why it matters: Filtering helps the two retrieval steps return news suited to their different purposes.

Key terms: Trend Scan, Deep Dive, dominant narratives, core signal extraction, where clause, server-side filtering, published_ts, has_content

11.4.2 Structured prompt engineering: The art of instructing an analyst

This section explains how a structured, evidence-bound prompt makes an LLM news analyst’s signals more repeatable, machine-readable, and auditable.

• Evidence-bound analyst instructions: Explain why the prompt specifies an analyst role, limits the model to provided context, and directs it through a two-step analysis.
  Why it matters: These instructions make qualitative signals less dependent on vague, unconstrained model responses.
• Structured auditable signal output: Describe how the JSON template, reasoning field, and response validation make the analyst’s output usable and traceable.
  Why it matters: Downstream systems need signals they can parse and decisions reviewers can trace.
• Modular foundation model choice: Explain why the pipeline can swap foundation models and how a finance-focused model’s potential benefits must be weighed against its constraints.
  Why it matters: Model choice can be tested against the task rather than assumed from a model’s financial specialization.

Key terms: prompt as a contract, quantitative analyst, Based ONLY on the context provided, two-step analysis, JSON schema, sentiment_score, supply_risk, policy_tone, conviction, key_events

11.4.3 The final output: A structured, auditable signal

This section shows how the LLM Analyst turns news into a structured, auditable qualitative signal and distinguishes that educational pipeline from a production system.

• Structured qualitative signal: Describe the LLM Analyst’s JSON output as a quantifiable qualitative signal, including what its scores, key_events, and reasoning contribute.
  Why it matters: The structure makes a qualitative news reading usable as a signal without losing the explanation behind it.
• Dataset entry and audit trail: Explain why the complete JSON object is treated as a dataset entry rather than just a prediction.
  Why it matters: Storing both scores and reasoning lets later analysis use the signal and inspect why it was produced.
• Multiple analyst model outputs: Describe how the common output interface permits multiple LLMs to assess the same news and how their signals could be combined.
  Why it matters: A common signal format allows differing model perspectives to be compared or combined.
• Production readiness requirements: State the operational concerns that separate this educational signal pipeline from a production deployment.
  Why it matters: A useful prototype signal is not, by itself, reliable enough for live financial operations.

Key terms: JSON, sentiment_score, supply_risk, policy_tone, key_events, reasoning, dataset entry, numerical feature, audit trail, qualitative feature class

11.5 Summary