Chapter 19: Using Python libraries
The introduction explains Python’s “batteries included” philosophy and introduces the chapter’s survey of libraries.
After this chapter you will be able to
- Batteries Included Philosophy. Explain that a stock Python installation includes a rich standard library, so many tasks do not require installing additional libraries.
- Standard library components. Distinguish the built-in types, constants, functions, and exceptions available without an import from the extensive collection of modules that makes up the largest part of Python’s library.
- Standard library data type categories. Describe the three standard-library categories discussed here and the kinds of data or operations each covers.
- File and storage module range. Describe the kinds of work covered by the standard library’s file and storage category: file access, data persistence, compression, and special file formats.
- Operating system module scope. Describe the kinds of tasks covered by the standard library’s operating-system modules, without needing to explain how the modules work.
- Internet formats and services modules. Describe the two kinds of support in this category: encoding and decoding standard internet data formats, and writing clients or servers for common or custom services.
- Development and runtime modules. Describe the kinds of help Python modules provide for working with code at runtime, including testing, profiling, tracebacks, garbage collection, and imports.
- Standard library limits. Explain that Python’s standard library provides a great deal out of the box but does not include every functionality a project might need.
- Library source trustworthiness. Explain why finding a Python package through a search engine does not by itself make it safe to use.
- OS-packaged library tradeoffs. Describe when an OS-packaged library can simplify installation and what limitations it may have.
- PyPI as a package source. Explain why PyPI is a useful next stop after searching the standard library: finding a suitable source package can be difficult, and PyPI provides an official, searchable repository, though it is not the only one.
- Pip package installation role. Explain why installing a third-party package manually can be troublesome and how pip helps.
- User-only package installation. Explain when the --user flag is useful and where it installs a library: in the user’s home directory rather than the main system Python installation.
- Virtual environment isolation. Explain what a virtual environment contains and why it helps when projects need different Python or package versions.
- Environment creation and activation. Describe the two steps for starting to use a virtual environment and what Python and pip do after activation.
- Standard tools and alternatives. Explain why pip and venv are a starting point, and identify the kinds of alternatives the author mentions.
What it builds on
- Python is complete: “Batteries included” (chapter 1) · The earlier introduction to Python as “batteries included”.
- Library and third-party modules (chapter 10) · The distinction between library modules and third-party modules.
- Built-in data types (chapter 3) · Familiarity with Python’s built-in data types.
- Colaboratory: Jupyter notebooks in the cloud (chapter 2) · Introduces Colaboratory as a Python environment.
Section by section
Introduction
The introduction explains Python’s “batteries included” philosophy and introduces the chapter’s survey of libraries.
- Batteries Included Philosophy: Explain that a stock Python installation includes a rich standard library, so many tasks do not require installing additional libraries.
Why it matters: This explains why the standard library is a useful first place to look for a solution.
Key terms: batteries included, stock install, standard library
19.1 “Batteries included”: The standard library
This section defines what Python’s standard library includes and recommends checking it before looking for an external library.
- Standard library components: Distinguish the built-in types, constants, functions, and exceptions available without an import from the extensive collection of modules that makes up the largest part of Python’s library.
Why it matters: This distinction helps you recognize what Python provides directly and what its modules provide. - Checking before external libraries: Explain why the section recommends consulting Python’s library reference before searching for an external library.
Why it matters: Checking what Python already offers may save you from searching elsewhere for functionality it includes.
Key terms: built-in data types, built-in functions, exceptions, modules, library reference, Python documentation, external library
19.1.1 Managing various data types
This subsection groups standard-library modules by the kinds of data they help handle.
- Standard library data type categories: Describe the three standard-library categories discussed here and the kinds of data or operations each covers.
Why it matters: These categories help you identify where to look in the standard library for a data-handling task.
Key terms: standard library, string services, data types, numeric and mathematical modules
19.1.2 Manipulating files and storage
This subsection identifies files and storage as a broad category of standard-library modules.
- File and storage module range: Describe the kinds of work covered by the standard library’s file and storage category: file access, data persistence, compression, and special file formats.
Why it matters: It helps you recognize the range of file and storage tasks supported by the standard library.
Key terms: standard library, file access, data persistence, compression, special file formats
19.1.3 Accessing operating system services
This subsection identifies the kinds of operating-system services covered by Python’s standard library modules.
- Operating system module scope: Describe the kinds of tasks covered by the standard library’s operating-system modules, without needing to explain how the modules work.
Why it matters: It helps you recognize when a standard library module may support an operating-system-related task.
Key terms: operating system modules, command-line parameters, log files, threads or processes, non-Python libraries
19.1.4 Using internet protocols and formats
This subsection describes the standard library category for internet data formats and network services.
- Internet formats and services modules: Describe the two kinds of support in this category: encoding and decoding standard internet data formats, and writing clients or servers for common or custom services.
Why it matters: It helps you recognize whether you need a module for a data format or for a network service.
Key terms: internet protocols and formats, MIME, JSON, XML, HTTP, generic socket server
19.1.5 Development and debugging tools and runtime services
This subsection identifies the kinds of standard-library modules that support development, debugging, and interaction with Python code at runtime.
- Development and runtime modules: Describe the kinds of help Python modules provide for working with code at runtime, including testing, profiling, tracebacks, garbage collection, and imports.
Why it matters: These categories show where the standard library can help you inspect and work with code while it runs.
Key terms: testing tools, profilers, error tracebacks, garbage collection, importing
19.2 Moving beyond the standard library
Explains why Python users may need functionality beyond its standard library.
- Standard library limits: Explain that Python’s standard library provides a great deal out of the box but does not include every functionality a project might need.
Why it matters: It explains why you may need to look beyond what Python includes.
Key terms: batteries included, standard library
19.3 Adding more Python libraries
This section explains how to find additional Python libraries and weigh the convenience of OS-packaged installations against trust, version, and flexibility concerns.
- Library source trustworthiness: Explain why finding a Python package through a search engine does not by itself make it safe to use.
Why it matters: Choosing a library includes deciding whether its code is worth trusting. - OS-packaged library tradeoffs: Describe when an OS-packaged library can simplify installation and what limitations it may have.
Why it matters: An easy installation route may not provide the version or installation flexibility you need.
Key terms: Python package, module, source, quality of the code, OS, installer, package manager, dependencies, prebuilt packages
19.4 The Python Package Index
This section introduces PyPI as an official, searchable place to find Python packages when the standard library does not provide the needed functionality.
- PyPI as a package source: Explain why PyPI is a useful next stop after searching the standard library: finding a suitable source package can be difficult, and PyPI provides an official, searchable repository, though it is not the only one.
Why it matters: It gives you a practical place to look when Python’s standard library does not meet your needs.
Key terms: Python Package Index, PyPI, standard library, source packages
19.5 Installing Python libraries using pip and venv
This section explains why pip is useful for installing third-party libraries, outlines basic installation options, and distinguishes managing a local Python installation from using Colaboratory.
- Pip package installation role: Explain why installing a third-party package manually can be troublesome and how pip helps.
Why it matters: This explains what pip handles for you when a library is not already available. - Basic pip installation options: Describe the command-line options this section gives for installing a library, upgrading it, or requesting a particular version.
Why it matters: These options let you choose whether to install a library, update it, or request a specific version. - Colaboratory installation differences: Distinguish why Colaboratory users usually need less installation and environment management than people managing their own Python installation.
Why it matters: It helps you tell which installation advice applies to your Python setup.
Key terms: pip, Python Package Index, dependencies, requests, –-upgrade switch, Colaboratory, temporary environment, virtual environments
19.5.1 Installing with the --user flag
This subsection explains when and why to install a Python library for one user with the --user flag.
- User-only package installation: Explain when the --user flag is useful and where it installs a library: in the user’s home directory rather than the main system Python installation.
Why it matters: It lets you install a library for yourself without changing the system installation or affecting other users.
Key terms: --user flag, user’s home directory, main system instance of Python
19.5.2 Virtual environments
This subsection explains how virtual environments isolate a project's Python and packages, and outlines how to create, activate, and use one.
- Virtual environment isolation: Explain what a virtual environment contains and why it helps when projects need different Python or package versions.
Why it matters: Isolation lets multiple projects use the Python and package versions they need without interfering with one another. - Environment creation and activation: Describe the two steps for starting to use a virtual environment and what Python and pip do after activation.
Why it matters: Creating and activating the environment puts package management and Python use in the intended project setup.
Key terms: virtual environment, virtualenv, system Python, activate, pip, default Python
19.5.3 Other options
This section compares the standard pip and venv tools with other options for managing Python versions, environments, and packages.
- Standard tools and alternatives: Explain why pip and venv are a starting point, and identify the kinds of alternatives the author mentions.
Why it matters: You can start with the standard tools and explore alternatives once your requirements are clearer. - Anaconda ecosystem tradeoff: Describe Anaconda's focus and how its conda package manager and package selection compare with PyPI.
Why it matters: Anaconda offers a focused alternative when scientific or data-science dependencies matter.
Key terms: pip, venv, Pyenv, poetry, rye, uv, Anaconda, conda, PyPI
19.6 Summary
Where this is used later
- Structured data formats (chapter 22) · The discussion of structured data formats builds on the distinction between internet formats and network services.
- Retrieving files over HTTP/HTTPS (chapter 22) · Retrieving files over HTTP/HTTPS uses a common internet service named here.
- Installing pandas (chapter 24) · Prior introduction to installing libraries with pip and using virtual environments.
Concepts explained in this chapter
- virtual environmentA virtual environment is an isolated Python installation with its own interpreter and packages, stored in a self-contained directory structure.
- standard libraryA standard library is the collection of libraries and modules distributed with a programming language implementation to provide commonly needed functionality.
- dependencyA software dependency is an external software component or prerequisite that an application or package needs to build or operate.
- moduleA module is a distinct, bounded software unit that groups related code or functionality and provides an organizational or interaction boundary within a larger system.
- primitive data typeA primitive data type is a fundamental, language-provided type for representing a basic value, rather than a composite of multiple values.
- packageA package is a structured collection of software modules organized under a common namespace.
- PythonPython is a high-level general-purpose programming language with an interpreted execution model and dynamic features.
- encodingencode is a string method that converts text into a bytes object using a character encoding.
- jsonJSON is a Python library for encoding and decoding JSON data.
- xmlXML is a text-based markup and data-interchange format that represents structured information using nested elements and attributes.
- built-in functionappend is a Python list method that adds one object to the end of a list.
- exceptionAn exception is an abnormal runtime event represented in Python by an object that can interrupt normal execution and be handled by exception-handling constructs.
- command-line argumentA command-line argument is a value supplied to a program when it is invoked through a command-line interface.
- databaseA database is an organized collection of persistent data managed for storage, retrieval, and modification.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.