liveBookmanning.com

What you will learn in Build Applications with Local AI Models on a Mac

Create private chatbots, voice apps, RAG systems, and more

by Keiji Kamigusa

Read it on liveBook Get the book at manning.com

Chapter by chapter

Concepts explained in this book

• large language modelA machine-learning model trained on large amounts of text to process and generate language.
• virtual environmentA Python virtual environment is an isolated environment with its own installed packages.
• OllamaOllama is a tool for downloading, running, and serving language models locally.
• pipelineA sequence of processing stages in which each stage feeds its output to the next.
• MLX WhisperMLX Whisper is a speech-recognition implementation based on Whisper and optimized for Apple Silicon through MLX.
• system promptA system message supplies instructions that guide an AI model's behavior in a conversation.
• local aiLocal AI is artificial intelligence whose models or computations run on a user's own device rather than on remote infrastructure.
• TerminalTerminal is the macOS application for accessing a command-line interface.
• StreamlitStreamlit is a Python framework for creating interactive web applications from Python code.
• HomebrewHomebrew is a package manager for installing and managing software on macOS and other Unix-like systems.
• lsA command-line utility that lists the contents of a directory.
• ModelfileA Modelfile is Ollama's declarative file format for specifying the base model, prompts, parameters, and other settings of a custom model.
• speech recognitionThe conversion of spoken audio into written text.
• fine-tuningFine-tuning is the process of further training a pretrained model on selected examples to adapt its behavior or capabilities.
• retrieval-augmented generationRetrieval-augmented generation is a technique that retrieves relevant information from an external knowledge source and supplies it to a language model alongside a question or instruction.
• GGUFA file format for storing machine-learning model tensors and metadata for inference, including quantized models.
• PythonA high-level general-purpose programming language known for readable syntax and a broad software ecosystem.
• command-line interfaceA command-line interface is an interaction interface in which users operate software by entering textual commands.
• LangChainLangChain is a software framework for building applications with language models by connecting them to data sources and composable components.
• parameter countModel size is a measure of a language model's scale, commonly expressed by its number of learned parameters.
• local AI chat applicationThe book's locally run, voice-enabled chat application transcribes speech and returns AI responses.
• mixture of expertsA neural network architecture that routes inputs through selected expert subnetworks rather than activating every parameter for every input.
• context windowContext length is the maximum number of tokens a language model can consider at once.
• session contextSession state is application data associated with a user's session and retained across interactions or reruns.
• Open WebUIOpen WebUI is an open-source web interface for interacting with locally hosted language models and related services.
• privacyPrivacy is the protection of a person's information from unauthorized collection, retention, use, or disclosure.
• temperatureA language-model generation setting that controls the randomness of token selection.
• inferenceModel inference is the process of applying a trained model to input data to produce an output.
• AI agentAn AI agent is a software system that perceives information, reasons about tasks, and takes actions to accomplish goals, including software-development tasks.
• Visual Studio CodeA source-code editor developed by Microsoft with extensibility for programming workflows.
• cloud-based AI serviceA service that provides AI capabilities through remotely operated servers.
• mlx_whisper.transcribeAn MLX Whisper function that transcribes audio from a file path or audio array.
• airplane mode testAn airplane mode test is a procedure that verifies whether an application continues operating when a device has no network connectivity.
• isolationThe separation of a project's packages and runtime from those of other projects or the host system.
• costThe monetary expense of operating an AI system, including service charges and local operating expenses.
• cd commandA command that changes the current working directory.
• UnslothA software library that accelerates and streamlines fine-tuning of language models while aiming to reduce memory use.
• MistralA family of large language models developed by Mistral AI for natural-language processing and generation.
• embeddingAn embedding is a numerical vector representation of an item, such as text, positioned to capture aspects of its meaning or features.
• command search pathAn environment variable containing directories searched to locate executable commands.
• WhisperWhisper is an automatic speech-recognition model developed by OpenAI for transcribing multilingual audio.
• LlamaA family of large language models developed by Meta.
• pwdA command that prints the path of the current working directory.
• GemmaGemma is a family of open language models developed by Google.
• integrated terminalA command-line interface embedded within a code editor.
• requirements.txtA requirements.txt file is a dependency specification file that lists packages and the versions required by a Python project.
• ollama.chat()A function in the Ollama Python library that sends conversational messages to a specified model and returns its response.
• package managerA software tool that installs, updates, removes, and manages software packages and their dependencies.
• web applicationAn application whose user interface and functionality are delivered through a web browser.
• vector databaseA vector database stores vector representations and supports searching for vectors that are similar or close to a query vector.
• ollama serveA command that starts the Ollama service and keeps it running while listening for requests.
• macOSApple's desktop operating system for Mac computers.
• stack traceA Python diagnostic report showing the sequence of calls leading to an exception and where it occurred.
• QwenQwen is a family of language models developed by Alibaba Cloud.
• model familyA group of related language models that share a common architecture, training lineage, or design.
• promptAn input instruction or request submitted to an AI model for processing.
• home directoryA filesystem directory assigned to a user for storing personal files and configuration data.
• dialogueThe example chatbot's ordered exchange of messages between a user and an AI assistant.
• Chroma DBChromaDB is an open-source vector database for storing, querying, and managing embeddings and associated data.
• ollama listAn Ollama command that lists the models available locally.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.