LLM application

As explained in Machine Learning Platform Engineering

An LLM application is a software system whose functionality uses a large language model to process inputs and produce outputs.

An LLM application is a production software system that uses a large language model to process inputs and generate outputs. It combines established ML infrastructure with LLM-specific components and operational controls. Nondeterministic responses, multistep reasoning, prompt sensitivity, safety risks, observability, and token costs require adapted testing and monitoring rather than a complete replacement of traditional MLOps. section 12.0 section 13.0

A deployable LLM application typically manages:

• Application infrastructure: containers, load balancers, autoscaling, Kubernetes, deployment automation, and CI/CD. section 12.1 section 12.1.2
• Prompt artifacts: system prompts, user templates, few-shot examples, reasoning templates, versioning, and tests. section 12.1.2 section 13.0
• Model configuration: context and output token limits, sampling parameters, model selection, retries, and timeouts. section 12.1.2
• Retrieval assets: document preprocessing, chunking, embeddings, vector indexes, metadata schemas, retrieval parameters, and semantic search. section 12.1.2 section 1.4.2
• Quality and risk controls: semantic evaluation, observability, cost tracking, safety guardrails, governance, and adversarial testing. section 13.0 section 13.3

Documentation assistant

DakkaBot applies Retrieval-Augmented Generation to company documentation. A build-time pipeline ingests and embeds documents into a vector database; query time embeds a user question, retrieves relevant chunks, augments a prompt, and generates an answer. Production hardening adds observability, semantic evaluation, cost optimization, and safety guardrails. section 1.4.2 section 12.2.1

Figure 12.5 Build-time setup (top) ingests documents into a vector database and configures the LLM. Query-time flow (middle) handles user questions through embedding, retrieval, augmentation, and generation. Implementation requires three lines of LangChain code.

Trace the application’s build-time ingestion and query-time embedding, retrieval, augmentation, and generation flow. section 12.2.1

How Machine Learning Platform Engineering covers it

How deep it goes

Explains architecture, metadata, RAG flow, testing, monitoring, and governance, stopping short of implementation code in the supplied evidence.

Examples

Uses DakkaBot, company documentation, document ingestion, vector embeddings, semantic retrieval, prompt augmentation, and generated answers.

What it assumes

Assumes existing ML platform practices while introducing RAG, prompt engineering, semantic evaluation, and LLM safety requirements.

What you can do after

Build it from scratch as an observable RAG service with managed prompts, semantic tests, cost tracking, and multilayered safety controls.

How it connects

Connects MLOps, LLMOps, Retrieval-Augmented Generation, vector databases, embedding models, prompt engineering, Kubernetes, CI/CD, and semantic evaluation.

liveBookmanning.com

Read Machine Learning Platform Engineering on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  AI Agents and ApplicationsAn LLM application is software that provides a use-case-specific interface or workflow around a large language model.

Compare how 2 books explain LLM application →

Related concepts

• promptuses · An input, typically containing instructions or context, supplied to an artificial-intelligence model to guide or elicit its output.
• language modeluses · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
• retrieval-augmented generationused by · Retrieval-augmented generation is a technique that retrieves relevant information from external sources and supplies it as context to a generative model when producing an output.
• monitoringuses · Monitoring is the ongoing observation and measurement of a system’s behavior, state, or operational signals to assess its condition and detect changes or problems.
• vector databaseuses · A vector database is a data-management system that stores and indexes numerical vector representations and supports retrieval through similarity or distance searches.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning Platform Engineering. The section links point to the parts of the book it draws on.