Chapter 10: Building a voice-enabled AI chat application

Introduction

The introduction explains why a local voice chat app is more complex than a text chatbot: it connects audio capture, speech recognition, and language model inference in a pipeline.

• Three-stage voice AI pipeline: Explain that the voice chat application combines audio capture, speech recognition, and language model inference, with each stage’s output feeding the next.
  Why it matters: This sequence gives you a way to understand how the voice features fit together.

Key terms: audio capture, speech recognition, language model inference, pipeline

10.1 The voice AI pipeline

The section traces a spoken request through three local stages and explains why the voice chat pipeline keeps its data on the Mac.

• Three-stage voice AI pipeline: Trace a spoken request from browser audio capture through MLX Whisper transcription to Ollama LLM inference, identifying what each stage receives or produces.
  Why it matters: This data flow shows how speech becomes input for the chat model.
• Local pipeline privacy: Explain why the voice AI pipeline does not need the internet during use after the initial model downloads, and what data stays on the machine.
  Why it matters: It explains the privacy benefit of running the entire voice chat pipeline locally.

Key terms: audio capture, st.audio_input(), WAV bytes, MLX Whisper, LLM inference, Ollama, locally, initial one-time model downloads

10.2 Setting up the voice chat application project

Sets up the Chapter 10 project files and dependencies and clarifies how to use successive versions of the application code.

• Voice chat project files: Describe where to create the ch10 folder and the purpose of the files needed for the voice chat application.
  Why it matters: Putting the right files together prepares the project for the application built in the following sections.
• Chapter dependency setup: Explain the role of requirements.txt and where to run the dependency-installation command when using the downloaded source package.
  Why it matters: Installing from the correct folder lets the project use its listed dependencies.
• Successive application code versions: Explain how to handle later code blocks that update voice_chat.py, especially the final version in section 10.4.
  Why it matters: Replacing rather than stacking code versions keeps the final application file as intended.

Key terms: ch10, my-ai-chatbot, voice_chat.py, model_config.json, setup_ollama.sh, voice_input.py, requirements.txt, dependencies, downloaded source package, earlier-stage code

10.3 Building the application step by step

The section introduces incremental development as building the voice chat application in stages and testing each stage before adding the next.

• Incremental development and staged testing: Explain that the application is built in three stages, adding capability and testing each stage before moving to the next.
  Why it matters: Testing each stage before adding more capability lets you check the application as you build it.

Key terms: incremental development, stages

10.3.1 Stage 1: Transcription in Streamlit

This section introduces browser microphone input in Streamlit and shows how to check that the first transcription stage works.

• Streamlit microphone audio path: Explain how st.audio_input() gets a user's recording from the browser into the Streamlit app's Python code.
  Why it matters: It identifies where voice input enters the Python application.
• Stage one transcription check: Describe how to try the audio input in the browser and recognize that stage 1 is working.
  Why it matters: A visible transcription provides a simple check before connecting voice input to the LLM.

Key terms: st.audio_input(), WAV bytes, Web Audio API, microphone access, transcribed text, You said:

10.3.2 Why use a temporary WAV file?

Explains why the voice chat app writes recorded WAV bytes to a temporary file before passing them to MLX Whisper, and why the file must remain until transcription reads it.

• Temporary WAV file bridge: Explain how a temporary WAV file bridges Streamlit’s audio input and the file path passed to MLX Whisper in this chapter, and why the file is not deleted when the with block exits.
  Why it matters: The temporary file lets the recorded audio reach Whisper without disappearing before Whisper reads it.

Key terms: st.audio_input(), UploadedFile, WAV bytes, transcribe(), tempfile.NamedTemporaryFile(), delete=False

10.3.3 Stage 2: Connecting transcription to the LLM

This stage connects spoken-question transcription to Ollama so the app displays the transcript and a streaming AI response while retaining conversation context.

• Voice question and contextual response: Describe what happens when a user speaks a question: the app displays the transcribed words and streams the AI’s response, while conversation stored in session state gives the AI context from earlier turns.
  Why it matters: This connects the visible voice-chat experience with the reason earlier exchanges can inform later answers.

Key terms: Ollama, transcribed words, streaming response, session state, context

10.3.4 Stage 3: The complete application

10.4 The complete voice chat application

10.4.1 Running the application

This section explains how to prepare Ollama and launch the completed voice chat application in a local browser.

• Automated Ollama preparation: Explain how ch10/setup_ollama.sh prepares Ollama before Streamlit starts, including how it chooses and downloads a model.
  Why it matters: The script makes Ollama setup repeatable without downloading an existing model again.
• Manual Ollama setup: Describe the alternative manual sequence for getting Ollama and a model ready.
  Why it matters: The manual route shows what must be in place when the helper script is not used.
• Local Streamlit launch: Describe when and where to start Streamlit and how to open the running application.
  Why it matters: This connects the prepared local model to the browser interface the reader will use.

Key terms: ch10/setup_ollama.sh, ollama.default_model, model_config.json, ollama list, gemma3:1b, Ollama server, Ollama API, Streamlit, ch10 folder, local URL

10.4.2 Updating the Ollama model menu

This section explains how to add an Ollama model to the voice chat app’s sidebar menu by updating its configuration file.

• Ollama model menu updates: Describe how to make a newer or different Ollama model available in the app: choose and pull its tag, add it to the configuration file, and restart Streamlit. Explain what the configuration fields control.
  Why it matters: You can offer another local model in the app without editing the Streamlit layout code.

Key terms: Ollama model tag, ch10/model_config.json, name, label, notes, default_model

10.5 Understanding the complete voice chat application code

10.5.1 Browser page configuration

This subsection explains how the application configures its browser page before other Streamlit calls.

• Browser page configuration: Explain what st.set_page_config() sets, why it must be the first Streamlit call, and what layout="wide" does for this application.
  Why it matters: This explains both the required placement of the page configuration call and the space it gives the voice chat interface.

Key terms: st.set_page_config(), layout="wide"

10.5.2 The sidebar: Controlling two models

Explains how the voice chat sidebar selects the response and transcription models and adjusts the variability of Ollama’s replies.

• Two sidebar model controls: Distinguish the Ollama model selected to write responses from the Whisper model selected to transcribe microphone recordings.
  Why it matters: The two selectors affect different parts of a voice conversation.
• Friendly labels and model identifiers: Explain why the selectboxes display readable model labels but pass technical identifiers to the model libraries, including an Ollama model tag and a Whisper repository path.
  Why it matters: Readable choices can coexist with the precise names the code needs to run a model.
• Ollama temperature setting: Describe how the sidebar’s temperature slider affects Ollama replies and why 0.7 is suggested as a starting point for a general voice assistant.
  Why it matters: The slider lets the reader adjust the character of replies without changing either selected model.

Key terms: Ollama selectbox, Whisper selectbox, label-to-identifier mapping, model_config.json, model tag, WHISPER_MODELS, Hugging Face repository path, temperature slider, Ollama options dictionary

10.5.3 transcribe_audio(): The bridge between Streamlit and MLX Whisper

Explains how transcribe_audio() passes Streamlit audio to MLX Whisper and handles the input, temporary file, and transcript.

• Streamlit to Whisper bridge: Trace how transcribe_audio() takes uploaded audio, writes it to a temporary file, and passes that file to MLX Whisper for transcription.
  Why it matters: This explains how recorded audio reaches the transcription model.
• Temporary file cleanup: Explain why os.unlink(tmp_path) is placed in a finally block.
  Why it matters: A transcription error should not leave temporary audio files behind.
• Audio read position: Describe what audio_file.read() does and why seek(0) is needed before reading the audio again.
  Why it matters: Resetting the position lets the application read the same audio again.
• Transcript whitespace cleanup: Explain why the function applies strip() to result["text"].
  Why it matters: The displayed transcript starts cleanly rather than with an unwanted space.

Key terms: transcribe_audio(), temporary file, MLX Whisper, try / finally, os.unlink(tmp_path), orphaned temp files, st.audio_input(), audio_file.read(), .seek(0), read position

10.5.4 stream_response(): A reusable streaming function

This section explains why the app’s repeated streaming logic is refactored into stream_response() and what the function does.

• Streaming logic refactoring: Explain why streaming is moved into a named function: several parts of the app need the same Ollama call, chunk handling, and screen updates, so they can reuse that logic without changing the app’s behavior.
  Why it matters: It explains why the streaming code lives in one place instead of being repeated throughout the app.
• Reusable streamed response function: Describe how stream_response(messages) gives the rest of the app a streamed assistant reply: it creates a chat bubble, renders incoming text as it arrives, removes the cursor when finished, and returns the full response text.
  Why it matters: It shows what a caller gets from the function without having to repeat the streaming and display code.

Key terms: refactoring, streaming, stream_response(), messages, assistant message bubble, streaming loop

10.5.5 handle_user_message(): The unified entry point

This section explains how one function handles user text from both voice and keyboard input before requesting an LLM response.

• Unified user message handling: Explain what handle_user_message() does and why both voice and text input call it.
  Why it matters: A shared entry point lets different input methods use the same conversation and response flow.

Key terms: handle_user_message(), conversation history, streaming response, voice input path, text input path

10.5.6 Dual input: Voice and text

The section explains why the Streamlit chat app offers both voice and text input and routes them through one handler.

• Dual input design: Explain that the microphone recorder and text field both feed into handle_user_message(), and why the text option matters when speaking is impractical or a transcription needs correction.
  Why it matters: You can still chat when you cannot speak or need to correct what the app heard.

Key terms: microphone recorder, text field, text fallback, handle_user_message()

10.6 Running and testing the application

10.6.1 Startup checklist

This section gives the checks to make before launching the voice chat app.

• Voice app startup prerequisites: State how to check that Ollama is running, the model selected in model_config.json is downloaded, and the virtual environment is active before running the app.
  Why it matters: These checks help you catch missing startup requirements before trying to use the app.

Key terms: Ollama, setup_ollama.sh, ollama serve, ollama list, model_config.json, virtual environment

10.6.2 Testing checklist

This section gives a checklist for testing voice and text input, shared conversation context, model switching, and conversation clearing in the completed app.

• Voice and text input checks: Describe how to test a basic voice question, transcription of a technical term, and a typed-message fallback, including what should happen in each case.
  Why it matters: These checks distinguish a working voice-and-text chat from an app that only accepts one input or produces inaccurate transcripts.
• Cross-input conversation context: Explain the test that checks whether conversation context carries between voice and text messages.
  Why it matters: The test checks that switching input methods does not break the conversation.
• Model switching comparison: Describe how to test a sidebar model change and what differences to look for.
  Why it matters: Repeating the same input makes differences after a model change easier to notice.
• Conversation clearing check: State how to test the Clear Conversation control and its expected result.
  Why it matters: This confirms that the user can start again without the previous chat history remaining.

Key terms: Voice input, transcript, Whisper, Text fallback, Conversation context, voice message, text input, Model switching, Ollama model, Whisper model

10.6.3 Troubleshooting

10.7 Understanding the latency budget

Explains why voice chat has a noticeable wait before the first response and how model choice, hardware, and progress feedback affect that wait.

• Voice response latency: Explain what voice-chat latency measures and why voice input can feel slower than text chat.
  Why it matters: The wait before the first response is a key part of how responsive a voice assistant feels.
• Transcription speed and progress feedback: Describe how a larger Whisper model and Mac hardware affect the wait, and why the app shows a transcription spinner.
  Why it matters: Model selection and visible progress both shape the experience of waiting for a voice response.

Key terms: latency, transcribe, streaming text, whisper-large-v3-mlx, transcription latency, st.spinner("Transcribing..."), Apple silicon

10.8 Exercises

10.9 Summary