10 Application architecture for generative AI apps

This chapter covers

An overview of GenAI application architecture and the emerging GenAI app stack
The different layers that make up the GenAI app stack
GenAI architecture principles
The benefits of orchestration frameworks and some of the popular ones
Model ensemble architectures
How to create a strategic framework for a cross-functional AI Center of Excellence

The enterprise architecture landscape continues to change, moving inexorably toward more self-directed systems—intelligent, self-managing applications that are capable of learning from interactions and adapting in real time. Furthermore, increasing digitization fuels the AI digital transformation. This ongoing progression underscores a transformative era in enterprise technology, poised to redefine the very nature of software development and deployment.

10.1 Generative AI: Application architecture

10.1.1 Software 2.0

10.1.2 The era of copilots

10.2 Generative AI: Application stack

10.2.1 Integrating the GenAI stack

10.2.2 GenAI architecture principles

10.2.3 GenAI application architecture: A detailed view

10.3 Orchestration layer

10.3.1 Benefits of an orchestration framework

10.3.2 Orchestration frameworks

10.3.3 Managing operations

10.3.4 Prompt management

10.4 Grounding layer

10.4.1 Data integration and preprocessing

10.4.2 Embeddings and vector management

10.5 Model layer

10.5.1 Model ensemble architecture

10.5.2 Model serving

10.6 Response filtering

Summary