Chapter 5: Building a ChatGPT clone with Aspire, Ollama, and Semantic Kernel

Introduction

The introduction motivates a product-specific chat assistant and identifies Semantic Kernel and Ollama as tools for adding a self-hosted LLM to an Aspire application.

• Product-specific chat assistant: Explain how an LLM could help the online shop by answering questions about its products and acting as a virtual store assistant.
  Why it matters: This gives the chat feature a purpose within the online shop.
• Semantic Kernel interface role: Describe Semantic Kernel as .NET libraries that provide a simple interface between LLMs and custom application code.
  Why it matters: It identifies the role Semantic Kernel plays when adding LLM functionality to a .NET application.
• Ollama self-hosting option: Explain that some LLMs can be hosted on your own infrastructure using Ollama instead of relying on another provider’s server, and that Semantic Kernel and Aspire can work with Ollama.
  Why it matters: Self-hosting is the alternative to using someone else’s LLM server that makes the chapter’s local chat application possible.

Key terms: LLM, domain-specific ChatGPT clone, virtual store assistant, Semantic Kernel, .NET libraries, LLMs, custom code, Ollama, self-host, Aspire integration library

5.1 Hosting an LLM anywhere with Ollama

This section explains why to run an LLM with Ollama and how to host a model in Aspire for a ChatGPT-like application.

• Local Ollama hosting benefits: Explain what Ollama provides and why running an LLM locally can be useful instead of relying on a cloud-hosted AI API.
  Why it matters: These benefits explain why the chapter hosts its own model rather than calling a remote AI API.
• Aspire Ollama container setup: Describe how the Aspire host sets up Ollama and why the example gives its container a persistent lifetime, a data volume, and Open WebUI.
  Why it matters: This setup makes the model available within the Aspire-hosted application without repeatedly downloading it.
• Model reference and startup dependency: Trace how the example adds a model, supplies it to the web app, and delays the app’s startup until that model is ready.
  Why it matters: The chat app depends on the model, so its startup must wait for the model to become usable.
• Open WebUI and custom chat: Distinguish the chat interaction already available through Open WebUI from the custom application functionality the chapter still aims to build.
  Why it matters: Being able to chat with a model is a useful milestone, but it is not the same as integrating chat into a tailored enterprise application.

Key terms: Ollama, local execution, LLM, CommunityToolkit.Aspire.Hosting.Ollama, persistent lifetime, WithDataVolume(), WithOpenWebUI(), AddModel(), phi35, phi3.5

5.2 Adding Semantic Kernel

This section identifies where Semantic Kernel belongs in the Aspire chat application and which Ollama connector package to install.

• Semantic Kernel client placement: Explain that Semantic Kernel provides adapters for different LLM-hosting technologies and acts as the LLM client, so its packages belong in the application using chat completion—OnlineShop.Web in this example.
  Why it matters: This tells you which project needs the LLM client dependency.
• Ollama connector package installation: State which NuGet package enables Semantic Kernel's Ollama adapter in OnlineShop.Web and why prerelease packages must be included when installing it.
  Why it matters: Selecting the Ollama connector and allowing prerelease versions makes the specified dependency available to the web application.

Key terms: Semantic Kernel, adapter libraries, LLM client, chat-completion functionality, OnlineShop.Web, Microsoft .SemanticKernel.Connectors.Ollama, NuGet package, preview, prerelease versions

5.2.1 Building the chatbot backend

Shows how to connect a Semantic Kernel backend to an Aspire-hosted Ollama model by obtaining its endpoint through a service-discovered connection string.

• Ollama connection method mismatch: Explain why the usual service-discovered HttpClient approach is not used for this Semantic Kernel connection to Ollama.
  Why it matters: This explains why the backend needs a bridge between Aspire’s connection string and Semantic Kernel’s endpoint URI.
• Connection string endpoint bridge: Trace how the backend obtains the Ollama model endpoint and supplies it to Semantic Kernel registration.
  Why it matters: It lets Semantic Kernel reach the Aspire-hosted model without hard-coding the Ollama container’s address.

Key terms: service discovery, Ollama community toolkit, OllamaSharp, connection string, endpoint URI, Aspire host, phi35, ConnectionStrings, Semantic Kernel registration

5.2.2 Adding Semantic Kernel backend logic

This section adds session-specific chat history and a streaming SignalR hub to the Semantic Kernel chatbot backend.

• Session-specific chat history: Explain why chat completion needs history and how the history service manages it for each user session.
  Why it matters: Keeping a session’s earlier messages lets the chatbot respond in the context of the conversation.
• Shared history service lifetime: Justify registering the chat-history service as a singleton in Program.cs.
  Why it matters: The service lifetime supports tracking multiple conversations through one backend service.
• Streaming SignalR chat hub: Describe the SignalR hub’s role in receiving prompts and streaming responses, and where the hub is registered.
  Why it matters: The hub connects UI prompts to responses that appear progressively to the user.
• Async stream cancellation: Explain what EnumeratorCancellation does for the StreamAnswer() cancellation token and why it matters for an async stream.
  Why it matters: A caller can stop waiting for further streamed answer items.
• Prompt randomness setting: Describe what PromptExecutionSettings configures and what temperature represents.
  Why it matters: Temperature is the setting this section identifies for controlling response randomness.

Key terms: chat completion, chat history, connection ID, AddUserMessage(), GetChatHistory(), RemoveHistory(), singleton, Program.cs, user sessions, SignalR hub

5.3 Finalizing the user interface

The section introduces the browser-based SignalR client and tells the reader where to add its CDN-hosted library.

• SignalR client library placement: Describe how the section directs you to add the JavaScript SignalR library to the browser app: use a public CDN and place its HTML element in App.razor immediately before the closing head tag.
  Why it matters: This identifies where the browser app loads the library needed for its JavaScript SignalR client.

Key terms: SignalR, content delivery network (CDN), App.razor

5.3.1 Adding a SignalR JavaScript client

This section builds a browser-side SignalR client that connects the chat page to its hub and handles streamed responses and chat controls.

• Chat client setup and helpers: Describe how the chat.js export function organizes the SignalR client, checks that the client is available, and provides helpers for connection status, page activity, and the returned-character count.
  Why it matters: These checks and helpers let the client report its state as a conversation proceeds.
• Hub connection and lifecycle: Trace how the client constructs the chat hub URL, creates and starts a hub connection, and updates the UI when that connection disconnects, reconnects, or closes.
  Why it matters: The chat page needs a connection to the correct hub and a way to show changes in that connection.
• Prompt streaming and controls: Describe how form submission triggers the prompt function, how that function listens for streamed characters from the hub, and which chat buttons receive event listeners.
  Why it matters: This links a user's prompt to streamed chat output and makes the page's controls responsive.

Key terms: chat.js, SignalR client, connection status, SignalR hub, hub-connection object, chat path, form-submission event, stream of characters, Stop, Clear

5.3.2 Building the chat page

This section creates the chat page’s Razor component and HTML while noting that the page cannot function until it is connected to the server.

• Chat Razor component setup: Describe where Chat.razor is placed and what its directives enable: a page path, JavaScript interop, and interactive mode so JavaScript can work.
  Why it matters: These are the page-level preparations for a chat interface that will use JavaScript.
• Chat markup and connection limit: Describe the chat page’s HTML elements and explain why the page does nothing if launched at this stage.
  Why it matters: It distinguishes building the visible interface from making the chat work.

Key terms: Chat.razor, Razor component, JavaScript interop, interactive mode, input field, output field, buttons, SignalR client

5.3.3 Adding the SignalR client to the page

This section explains why the chat page loads its SignalR JavaScript client through Blazor interop, then adds navigation so users can reach the page.

• Chat client loading choice: Explain why the SignalR client’s chat.js file should be loaded dynamically on the chat page rather than referenced globally or placed in the page’s HTML body.
  Why it matters: Loading the client on the page that has its required elements avoids an unnecessary global script and keeps the page markup manageable.
• Chat page navigation link: Describe the final step that lets users reach the chat page and how the completed interface is checked.
  Why it matters: The navigation link makes the finished chat interface accessible from the application.

Key terms: SignalR JavaScript client, chat.js, App.razor, JavaScript interop, NavMenu .razor, chat page, chat interface

5.4 Summary