liveBook · Manning

Chapter 1: Why your AI projects need a platform

What you will learn in chapter 1 of Designing AI Systems

This chapter introduces why moving AI prototypes into production requires shared platform infrastructure rather than one-off application code.

After this chapter you will be able to

Section by section

Introduction

This chapter introduces why moving AI prototypes into production requires shared platform infrastructure rather than one-off application code.

Key terms: prototype, production, performance monitoring, cost controls, safety guardrails, operational infrastructure, platform thinking, AI sprawl, technical debt, platform services

1.1 The AI Wild West: why winging it doesn't scale

This section introduces Sam’s fast, successful-looking AI chatbot prototype built with a script, an API, prompts, and manually curated FAQ content.

Key terms: Python script, OpenAI's API, prompts, working prototype, FAQ answers, hard codes, two weeks, demo, Leadership, promoted

1.1.1 The production wake-up call

This section shows how a successful AI prototype fails in production without the operational, safety, knowledge, cost, and improvement capabilities of a real platform.

Key terms: monitoring, logging, concurrent request handling, rate limiting, scalability, usage tracking, cost attribution, spending controls, guardrails, content safety

1.1.2 The universal pattern: prototype vs. production

This subsection establishes that AI prototypes commonly fail to meet production demands when they move from controlled demos to real-world use.

Key terms: AI prototypes, production systems, controlled demo environments, real-world usage patterns, actual user loads, business requirements

1.1.3 What if the infrastructure already existed?

This subsection shows how ready-to-use platform services would let an AI application inherit production capabilities instead of rebuilding infrastructure for each problem.

Key terms: platform, ready-to-use services, Infrastructure Layer, Observability Service, response latencies, token usage, request queuing, rate limiting, cost management system, real-time spending tracking

1.1.4 AI sprawl

This section defines AI sprawl as uncoordinated growth of disconnected AI services that duplicates infrastructure, fragments security and learning, and erodes organizational agility.

Key terms: AI sprawl, platform foundation, disconnected AI services, organizational agility, session management, Redis, PostgreSQL, file storage, token tracking, APIs

1.2 The AI infrastructure reality: the model is just 2% of the story

This section establishes that AI system complexity lies overwhelmingly in surrounding infrastructure rather than in the model call itself.

Key terms: Hidden Technical Debt in Machine Learning Systems, model API call, surrounding infrastructure, system complexity, Generative AI

1.2.1 Traditional ML Customer Support (2015)

This subsection shows that even a traditional 2015 ML customer-support classifier required far more production infrastructure than model inference code.

Key terms: model inference code, supporting infrastructure, pre-trained classifier, category prediction, data processing systems, structured features, batch workflows, training data, periodic model updates, single model version

1.2.2 Modern GenAI Customer Support (2026)

Modern GenAI customer-support applications hide minimal model code behind a large, GenAI-specific infrastructure stack that should be provided as shared platform services.

Key terms: model code, supporting infrastructure, chat completion request, GPT-4, knowledge management, document ingestion pipelines, vector embedding generation, semantic search, context assembly, safety and compliance

1.3 Thinking from first principles: what AI applications actually need

This section derives the production requirements that distinguish a serious AI platform from a one-off GenAI prototype.

Key terms: context-aware intelligence, state, organizational knowledge, persistent contextual intelligence, multi-step orchestration, dynamic tool integration, authentication flows, API quotas, graceful degradation, recovery strategies

1.4 Your AI platform blueprint: what we're building

This section maps AI application requirements to a reusable, layered service-oriented platform blueprint that lets teams share infrastructure instead of rebuilding it per project.

Key terms: layered, service-oriented architecture, core requirements, reusable infrastructure, service boundaries, Data Service, Session Service, Workflow Service, document ingestion, vector embedding generation, semantic search

1.5 Platform in action

This section traces how a retail assistant uses shared platform services to turn a customer question into a contextual, grounded, safe, observable response.

Key terms: Workflow Service, Session Service, Guardrails Service, Data Service, Model Service, Observability Service, conversation history, semantic search, knowledge base, grounded responses

1.6 Why platform thinking changes everything

This section explains how a shared AI platform replaces duplicated, inconsistent team-by-team infrastructure with reusable services that make AI development secure, observable, adaptable, and sustainable.

Key terms: shared service components, reusable services, Infrastructure Layer, Observability Service, Guardrails Service, input sanitization, token usage, response quality, knowledge fragmentation, shared services

1.7 Summary

This summary recaps why production AI needs a shared service-oriented platform rather than disconnected prototype infrastructure.

Where this is used later

Concepts explained in this chapter

Chapter 2: Building with the platform: SDK and API design →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.