Chapter 10: Modules and scoping rules
This introduction explains why modules are useful for organizing larger or reusable Python code.
After this chapter you will be able to
- Reasons to use modules. Explain why a programmer might organize code into modules, even though doing so is optional.
- Module files and contents. Describe a module as a file that defines a group of Python functions or other objects, with its name derived from the filename. State that modules can contain Python source code or compiled C or C++ code and are used the same way.
- Separate module namespaces. Explain how separate module namespaces let a program use identically named functions from different modules, as with mymodule.reverse and othermodule.reverse.
- Module files and imports. Describe how mymath.py defines pi and area, and how importing mymath makes those definitions available under the module’s name. Explain that the .py suffix marks a Python source file but is omitted from the import statement.
- Qualified module names. Explain why import mymath does not make pi or area directly accessible and how to access them instead. Describe how qualification keeps names from different modules distinct, while definitions inside a module can refer to one another without qualification.
- Reloading changed modules. Explain why repeating import does not pick up changes to a module already loaded in an interactive session, and describe the two ways the section gives to load the changed version.
- Qualified module imports. Explain that a basic import makes a module available, but references to names inside it must be prefixed with the module name.
- Selective name imports. Describe how importing specific names from a module lets subsequent code use those names without the module-name prefix.
- Wildcard imports and name clashes. Explain which names a wildcard import makes directly available, how __all__ changes that selection, and why this form can cause name clashes or obscure where names came from.
- Module search order and outcome. Explain that sys.path lists the directories Python searches in order for an import, using the first matching module or raising ImportError if none is found.
- Module path starting locations. Describe how PYTHONPATH or an installation-dependent default initializes sys.path, and distinguish its first entry when running a script from its first entry in an interactive session.
- Module placement choices. Distinguish where to put a module used by one program from where to put site-specific modules used by several programs, and explain the caution about Python’s normal search directories.
- Search path extension tradeoffs. Compare changing sys.path in code, setting PYTHONPATH, and using a .pth file, including how PYTHONPATH and .pth files affect search order.
- Underscore names and wildcard imports. Explain that `from module import *` imports names without a leading underscore but skips names that begin with one, allowing module authors to mark internal names while leaving them accessible by other means.
- Standard library module availability. Explain that installing Python makes its standard-library modules available, but a program must explicitly import the modules or names it uses. Describe the standard-library documentation as a place to discover modules beyond those covered in the book.
- Third-party module installation. Explain what must happen before a program can import a third-party module: find it through the Python Package Index, then download and install it in a directory on the module search path.
- Namespace bindings and name lookup. Explain what a namespace contains and trace how Python resolves an identifier through local, global, and built-in namespaces.
- Top-level and function namespaces. Distinguish top-level namespace bindings from a function’s local bindings, and describe which global namespace a function uses.
- Built-in name shadowing. Explain what happens when a local binding uses the name of a built-in function, and describe how to regain access after doing this interactively.
- Reusable standalone module. Describe how to package the functions from the end of chapter 9 as a standalone module and test that they work from a new Jupyter notebook.
- AI-assisted module repackaging. Explain how to direct an AI tool to move previously written functions into a separate module and modify the remaining code to import and call them. State that the prompt must specify the separation and that the tool needs access to the earlier code.
What it builds on
- Colaboratory: Jupyter notebooks in the cloud (chapter 2) · The use of Colaboratory notebooks as the working environment.
- Basic function definitions (chapter 9) · The example uses a function definition inside the module.
- Local, nonlocal, and global variables (chapter 9) · Earlier treatment of local and global variables provides context for these namespace rules.
- Useful functions (chapter 9) · The functions created at the end of chapter 9 are the material to package.
Section by section
Introduction
This introduction explains why modules are useful for organizing larger or reusable Python code.
- Reasons to use modules: Explain why a programmer might organize code into modules, even though doing so is optional.
Why it matters: Modules give you a practical way to keep growing or reusable code manageable.
Key terms: module, Python standard library
10.1 What is a module?
This section defines Python modules, explains how their namespaces prevent name clashes, and shows how to make a module available in Colaboratory.
- Module files and contents: Describe a module as a file that defines a group of Python functions or other objects, with its name derived from the filename. State that modules can contain Python source code or compiled C or C++ code and are used the same way.
Why it matters: This gives you the basic unit Python uses to organize reusable code. - Separate module namespaces: Explain how separate module namespaces let a program use identically named functions from different modules, as with mymodule.reverse and othermodule.reverse.
Why it matters: Separate namespaces let you combine code without requiring every function to have a unique name across the program. - Modular standard functionality: Explain why providing standard Python functions through modules helps keep Python manageable: many are outside the language core and can be loaded as needed.
Why it matters: This explains why useful Python functionality often comes from a module rather than being available directly in the language core. - Modules in notebook sessions: Describe how to make a module available in Colaboratory by uploading it to the current notebook session or running the supplied notebook cell that creates or fetches it; contrast this with copying a module to the current directory for a local interpreter.
Why it matters: Making the file available first lets the notebook's later cells access the module.
Key terms: module, Python source code, compiled modules, namespace, identifiers, name-clash problems, mymodule.reverse, othermodule.reverse, standard Python functions, modules
10.2 A first module
The section builds a small Python module and shows how to import its names, keep them distinct through qualification, and reload changes during interactive development.
- Module files and imports: Describe how mymath.py defines pi and area, and how importing mymath makes those definitions available under the module’s name. Explain that the .py suffix marks a Python source file but is omitted from the import statement.
Why it matters: This connects a Python source file to the name used to access its definitions. - Qualified module names: Explain why import mymath does not make pi or area directly accessible and how to access them instead. Describe how qualification keeps names from different modules distinct, while definitions inside a module can refer to one another without qualification.
Why it matters: Qualification lets a program use definitions from multiple modules without confusing names that happen to match. - Selective name imports: Distinguish from mymath import pi from importing the module as a whole: the former makes pi directly accessible, but does not do the same for area.
Why it matters: A selective import is useful when you want to use a particular module name often without repeating the module prefix. - Reloading changed modules: Explain why repeating import does not pick up changes to a module already loaded in an interactive session, and describe the two ways the section gives to load the changed version.
Why it matters: Without a restart or reload, an interactive test may still be using the old version of your module.
Key terms: module, .py, import statement, qualification, attribute, mymath.pi, mymath.area, from modulename import attribute, from mymath import pi, reload
10.3 The import statement
This section explains three forms of Python’s import statement, how each makes module names available, and why wildcard imports require care.
- Qualified module imports: Explain that a basic import makes a module available, but references to names inside it must be prefixed with the module name.
Why it matters: It explains why importing a module does not let you use its contents without the module prefix. - Selective name imports: Describe how importing specific names from a module lets subsequent code use those names without the module-name prefix.
Why it matters: It lets you distinguish importing a module from bringing selected names directly into your code. - Wildcard imports and name clashes: Explain which names a wildcard import makes directly available, how __all__ changes that selection, and why this form can cause name clashes or obscure where names came from.
Why it matters: It helps you predict what a wildcard import exposes and why the convenience can make code harder to follow.
Key terms: import statement, module name, from ... import ..., from modulename import *, public names, __all__, name clash
10.4 The module search path
This section explains how Python uses sys.path to locate imported modules and how that path is initialized.
- Module search order and outcome: Explain that sys.path lists the directories Python searches in order for an import, using the first matching module or raising ImportError if none is found.
Why it matters: Search order explains both which module an import selects and why an import can fail. - Module path starting locations: Describe how PYTHONPATH or an installation-dependent default initializes sys.path, and distinguish its first entry when running a script from its first entry in an interactive session.
Why it matters: The starting path helps explain why an import may behave differently in a script and an interactive session.
Key terms: sys.path, module search path, import statement, ImportError, PYTHONPATH, script, interactive session, current directory
10.4.1 Where to place your own modules
This section explains where to put your own modules so programs can import them, and how different ways of extending Python’s search path affect which modules are found.
- Interactive import assumption: Explain why the chapter’s mymath module was importable in Colaboratory but might not be importable in a production program.
Why it matters: An import that works in an interactive session may fail when the program runs elsewhere. - Module placement choices: Distinguish where to put a module used by one program from where to put site-specific modules used by several programs, and explain the caution about Python’s normal search directories.
Why it matters: Choosing a location that matches how a module is used makes it available without putting it at unnecessary risk during a Python upgrade. - Search path extension tradeoffs: Compare changing sys.path in code, setting PYTHONPATH, and using a .pth file, including how PYTHONPATH and .pth files affect search order.
Why it matters: Search order determines whether Python imports your module or an existing library module with the same name.
Key terms: sys.path, current directory, mymath module, site-specific modules, module search path, local code directories, PYTHONPATH, .pth file, sys.prefix
10.5 Private names in modules
This section explains how leading underscores keep module names out of wildcard imports without making them inaccessible.
- Underscore names and wildcard imports: Explain that `from module import *` imports names without a leading underscore but skips names that begin with one, allowing module authors to mark internal names while leaving them accessible by other means.
Why it matters: It helps you recognize which module names are intended for users without mistaking an underscore for an access restriction.
Key terms: from module import *, leading underscores, private names, internal names
10.6 Library and third-party modules
This section distinguishes standard-library modules, which come with Python, from third-party modules, which must be installed before they can be imported.
- Standard library module availability: Explain that installing Python makes its standard-library modules available, but a program must explicitly import the modules or names it uses. Describe the standard-library documentation as a place to discover modules beyond those covered in the book.
Why it matters: You can use functionality that ships with Python without installing another package, once you import it. - Third-party module installation: Explain what must happen before a program can import a third-party module: find it through the Python Package Index, then download and install it in a directory on the module search path.
Why it matters: An import cannot use a third-party module until that module has been installed where Python can find it.
Key terms: standard Python distribution, library modules, Python standard library, import, third-party modules, Python Package Index, module search path
10.7 Python scoping rules and namespaces
This section explains how Python namespaces bind names to objects, how name lookup works, and how bindings differ between functions and top-level code.
- Namespace bindings and name lookup: Explain what a namespace contains and trace how Python resolves an identifier through local, global, and built-in namespaces.
Why it matters: The lookup order explains both which object a name refers to and why an unresolved name raises an error. - Top-level and function namespaces: Distinguish top-level namespace bindings from a function’s local bindings, and describe which global namespace a function uses.
Why it matters: This distinction helps explain why a function can see names from its defining module but not unrelated names from its caller. - Built-in names and shadowing: Explain why defining a name such as list can prevent access to the built-in object of the same name.
Why it matters: It explains a common surprise when a variable name hides a built-in function. - Inspecting and removing bindings: Describe what locals and globals reveal, what del removes, and why deleting and reimporting a module does not load disk changes.
Why it matters: These tools let you inspect session names without mistaking removal of a name for reloading a module.
Key terms: namespace, identifier, binding, local namespace, global namespace, built-in namespace, parameter, module, __builtins__, list
10.7.1 The built-in namespace
This subsection shows how built-in names can be overridden and how namespace-inspection tools can help identify names in use.
- Built-in name shadowing: Explain what happens when a local binding uses the name of a built-in function, and describe how to regain access after doing this interactively.
Why it matters: It explains why a familiar built-in may suddenly stop working in an interactive session. - Namespace inspection with dir: Distinguish what dir returns when given a module from what it returns without arguments, and describe its debugging use.
Why it matters: Checking which names exist can help you spot a spelling mistake without assuming dir shows their values.
Key terms: built-in namespace, binding, list, del, dir, module, local namespace, identifiers
10.8 Creating a module
This section asks the reader to turn earlier functions into a reusable module, test it from a new notebook, and upload it to Colaboratory if needed.
- Reusable standalone module: Describe how to package the functions from the end of chapter 9 as a standalone module and test that they work from a new Jupyter notebook.
Why it matters: Testing from a separate notebook checks that the functions are usable outside the file where they were defined. - Colaboratory module upload: Describe how to upload a module created in a code editor or IDE to Colaboratory.
Why it matters: Uploading makes a locally created module available for the Colaboratory test.
Key terms: standalone module, main program, Jupyter notebook, Colaboratory, file folder icon, upload document icon
10.8.1 Solving the problem with AI-generated code
This section explains how to guide an AI tool to repackage existing functions into a module while keeping the code that uses them separate.
- AI-assisted module repackaging: Explain how to direct an AI tool to move previously written functions into a separate module and modify the remaining code to import and call them. State that the prompt must specify the separation and that the tool needs access to the earlier code.
Why it matters: Clear instructions and access to the existing code help the AI tool preserve the functions while changing how the code is organized.
Key terms: module, import, client code, prompt
10.8.2 Solutions and discussion
10.9 Summary
Where this is used later
- Programs and modules (chapter 11) · A module makes its functions available to other code.
- (chapter intro) (chapter 18) · The idea of a Python module as a unit of reusable code.
- What is a package? (chapter 18) · The earlier introduction to modules as units of Python code.
- import statements within packages (chapter 18) · Import statements within packages build on the forms of import introduced here.
- The __all__ attribute (chapter 18) · The discussion of __all__ builds on its effect on wildcard imports introduced here.
- Proper use of packages (chapter 18) · Draws on the earlier discussion of underscore-prefixed names in modules.
- “Batteries included”: The standard library (chapter 19) · The discussion of the standard library builds on its availability and breadth.
- Using Python to fetch files from an FTP server (chapter 22) · Using modules from Python’s standard library
Concepts explained in this chapter
- moduleA module is a distinct, bounded software unit that groups related code or functionality and provides an organizational or interaction boundary within a larger system.
- import statementAn import statement is a source-code declaration that makes names or definitions from another module, package, or namespace available in the current scope.
- reloadreload is a Python operation that re-executes a previously imported module's code in an existing interpreter session.
- namespaceA global namespace is the namespace associated with a module, script, or interactive session and visible as the defining environment for its functions.
- scopeA scope is a region of program execution in which name bindings can be resolved and accessed according to Python's lookup rules.
- built-in namespaceThe built-in namespace is the namespace containing Python's predefined names, including built-in functions and exceptions.
- globalsglobals is a Python built-in function that returns a dictionary representing the current global namespace bindings.
- localslocals is a Python built-in function that returns a dictionary representing the current local namespace bindings.
- sys.pathA module search path is the ordered set of directories that a language runtime searches to locate importable modules and packages.
- qualificationQualification is the practice of accessing a name through a containing module or namespace qualifier.
- bindingA binding is an association between an identifier and an object in a namespace.
- local namespaceA local namespace is the namespace containing bindings created within the currently executing function or code block.
- newlineA newline is a character or character sequence that marks the boundary between consecutive lines of text.
- raw string literalA raw string literal is a Python string literal in which backslashes are generally treated as literal characters rather than introducing escape sequences.
- wildcard importA wildcard import is an import statement form that brings the names exported by a module directly into the importing namespace.
- built-in functionappend is a Python list method that adds one object to the end of a list.
- dir()dir() is Python's built-in function for listing names and attributes available in a namespace or associated with an object, module, or type.
- dependencyA software dependency is an external software component or prerequisite that an application or package needs to build or operate.
- functionA function is a callable program unit that performs an operation, may accept inputs, and can produce a result or side effect.
- string modulestring is a Python standard-library module providing constants and utilities for common string-processing tasks.
- __all____all__ is a module-level sequence that specifies the names exported by a wildcard import.
- __init__.pyAn __init__.py file is a Python package file that can contain initialization code executed when the package is imported.
- global variableA global variable is a name bound at module or program level that can be accessed and modified by functions that explicitly declare it global.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.