Chapter 11: Python programs
This introduction explains why command-line Python scripts are worth learning after interactive work in Colaboratory, especially for processing many files.
After this chapter you will be able to
- Interactive and command-line Python. Distinguish using Python interactively in Colaboratory from creating programs or scripts for production use. Describe how a command-line script can receive arguments or options and have its input or output redirected.
- Basic script structure. Explain that a file of sequential Python statements can be a script, but a more useful basic structure is to define a controlling function and then call it. Describe why the section recommends making this a habit.
- Colaboratory command-line script execution. Explain how to run a saved script through Colaboratory’s host environment and why the script file must be present in the session.
- Command-line argument list. Explain how to access command-line arguments with sys.argv and interpret its contents. State why a number supplied on the command line may need conversion before use.
- Main script execution guard. Explain how checking __name__ lets a file call main when run as a script without running that guarded code when imported, and why this makes the file useful in both roles.
- Standard stream redirection. Explain how a script reads redirected input through sys.stdin and writes output through print or sys.stdout, and describe what < and > do at the command line.
- Appending and piping output. Distinguish > from >> and explain how | chains runs of the replacement script.
- ArgumentParser setup and argument types. Describe how to create an ArgumentParser, add arguments, and distinguish required positional arguments from optional arguments identified by a prefix such as '-'.
- Parsed values and errors. Describe how argparse makes parsed values available and what happens when an argument is invalid or required input is missing.
- Fileinput line iteration. Explain how fileinput takes command-line arguments as input filenames and lets a script iterate sequentially over their lines. Describe how standard input is used when there are no arguments or when an argument is a hyphen.
- Direct script execution on UNIX. Describe the steps for making a script directly executable on a UNIX-like system and how placing it on the path affects where it can be run from.
- macOS script launch options. Explain that macOS can run Python scripts from a terminal as on Linux/UNIX, or from Finder using Python Launcher.
- Command window script execution. Describe how to run a script from a Windows command window or PowerShell and why this option is especially flexible.
- Controlling function separation. Explain why a growing script can benefit from separating its controlling function from its other functions, and describe how this helps when the script is also used as a module.
- Main script execution guard. Explain what problem `if __name__ == "__main__":` prevents when a file serves as both a script and a module, and distinguish the file’s `__name__` in those two cases.
- Basic distribution tradeoffs. Explain the limitations of sharing Python source files or bytecode as an application. Describe why bundled source may be awkward for users unfamiliar with Python and why shipping only .pyc files is discouraged.
- Wheels for Python distribution. Explain that wheels are the current standard way to package and distribute Python modules and applications, and that they are designed to make installation more reliable and help manage dependencies.
- Executable zip archive mechanics. Explain how Python runs a zip archive containing multiple modules and what makes the archive directly executable.
- Standalone executable tools and trade-offs. Explain what py2exe and py2app produce, which platforms they target, and why a standalone executable may or may not be a good choice.
- Freeze executable creation. Explain that freeze generates C files from a Python program, then requires a C compiler to compile and link them into an executable that can run without Python installed.
- wc output option behavior. Describe how the refactored wc program should use -l, -w, and -c to choose which file statistics to display.
- AI refactoring prompt setup. Describe how to prepare the chatbot to refactor the earlier program: give it the existing code, specify that the result should mimic Unix wc with -l, -w, -c, and -L, and use Copilot’s full chat interface when asking it to create a new file.
What it builds on
- Writing and running code in Colaboratory (chapter 2) · Assumes experience writing and running code in Colaboratory.
- Basic function definitions (chapter 9) · Basic function definitions make it possible to understand defining and calling the script’s controlling function.
- The import statement (chapter 10) · The idea that another script can import a Python file as a module.
- What is a module? (chapter 10) · A module makes its functions available to other code.
- Writing a simple program to analyze a text file (chapter 8) · The earlier text-file analysis program provides the starting point for counting file statistics.
- Refactoring word_count (chapter 8) · This task reuses code from the chapter 8 lab.
Section by section
Introduction
This introduction explains why command-line Python scripts are worth learning after interactive work in Colaboratory, especially for processing many files.
- Interactive and command-line Python: Distinguish using Python interactively in Colaboratory from creating programs or scripts for production use. Describe how a command-line script can receive arguments or options and have its input or output redirected.
Why it matters: This distinction gives you a reason to move beyond running code interactively when you need a reusable program. - Value of command-line techniques: Explain why command-line techniques are worth understanding even if they feel less convenient in a GUI environment.
Why it matters: Even if you prefer a GUI, these techniques can help with repetitive file-processing work and with reading others’ code.
Key terms: interactive mode, Python programs or scripts, command line, arguments and options, redirect their input and output, command-line techniques, GUI environment, large numbers of files
11.1 Creating a very basic program
This section introduces a basic Python script structure: define a controlling function and then call it.
- Basic script structure: Explain that a file of sequential Python statements can be a script, but a more useful basic structure is to define a controlling function and then call it. Describe why the section recommends making this a habit.
Why it matters: Starting with a controlling function establishes a useful structure before a script becomes more complex.
Key terms: program, script, main, controlling function
11.1.1 Starting a script from a command line
This subsection explains how to start a saved Python script from a command line in Colaboratory, Linux/UNIX, macOS, or Windows.
- Colaboratory command-line script execution: Explain how to run a saved script through Colaboratory’s host environment and why the script file must be present in the session.
Why it matters: Running code in a notebook cell is different from asking the host environment to run a saved script. - Local terminal execution prerequisites: Describe the terminal setup the section gives for starting a script on Linux/UNIX, macOS, and Windows.
Why it matters: Knowing which terminal to use and where the script is helps you start it outside a notebook.
Key terms: Colaboratory, ! prefix, host environment, script file, path, terminal, Command Prompt, PowerShell, cd
11.1.2 Command-line arguments
This section explains how a script receives command-line arguments through sys.argv and how to interpret the values it contains.
- Command-line argument list: Explain how to access command-line arguments with sys.argv and interpret its contents. State why a number supplied on the command line may need conversion before use.
Why it matters: This lets you distinguish the command from its arguments and handle numeric inputs correctly.
Key terms: sys.argv, sys module, command-line arguments, list of strings
11.1.3 Executing code only as main script
This section explains how to run a script’s main code only when the file is executed directly, while keeping the file safe to import as a module.
- Main script execution guard: Explain how checking __name__ lets a file call main when run as a script without running that guarded code when imported, and why this makes the file useful in both roles.
Why it matters: The guard lets you run a file directly without triggering its main code when you import it to use or test its functions.
Key terms: __name__, __main__, main, module
11.1.4 Redirecting the input and output of a script
This section shows how command-line redirection and pipes let a Python script read, transform, and pass along streams of text.
- Standard stream redirection: Explain how a script reads redirected input through sys.stdin and writes output through print or sys.stdout, and describe what < and > do at the command line.
Why it matters: It lets you use the same script with files without changing its input and output code. - End of redirected input: Describe how reading sys.stdin as a file differs from calling input when no file is redirected.
Why it matters: It explains why a stream-reading script may appear to wait when run without redirected input. - Appending and piping output: Distinguish > from >> and explain how | chains runs of the replacement script.
Why it matters: These operators let small, specialized programs work together while controlling whether results replace or extend a file.
Key terms: standard input, standard output, sys.stdin, sys.stdout, print, <, input function, end-of-file indicator, Ctrl-D, >
11.1.5 The argparse module
This section shows how to use argparse to define command-line arguments and options, then access the parsed values.
- ArgumentParser setup and argument types: Describe how to create an ArgumentParser, add arguments, and distinguish required positional arguments from optional arguments identified by a prefix such as '-'.
Why it matters: This distinction lets you describe what a command-line user must supply and what they can choose to supply. - Typed arguments and option behavior: Explain the example's integer positional argument and how its optional filename and quiet options behave, including the effect of action="store_false".
Why it matters: The example shows that argparse can do more than collect strings: it can require a type and set a Boolean value when an option appears. - Parsed values and errors: Describe how argparse makes parsed values available and what happens when an argument is invalid or required input is missing.
Why it matters: Once the command line is parsed, the program needs to retrieve its inputs and recognize rejected ones.
Key terms: argparse, ArgumentParser, positional arguments, optional arguments, indent, int, -f, --file, quiet, action="store_false"
11.1.6 Using the fileinput module
This section shows how fileinput lets a script process lines from multiple files or standard input as one stream, and introduces functions for tracking and controlling that stream.
- Fileinput line iteration: Explain how fileinput takes command-line arguments as input filenames and lets a script iterate sequentially over their lines. Describe how standard input is used when there are no arguments or when an argument is a hyphen.
Why it matters: You can write one line-processing loop that works across several files or standard input. - Input position and stream controls: Describe what fileinput's status functions can report about the current line and file, and how a script can skip to the next file or close the stream.
Why it matters: These functions let a script respond to file boundaries and keep track of where its input came from. - Explicit filenames and in-place output: Explain how passing filenames to fileinput.input changes its input source, and state what its inplace option does at the level described here.
Why it matters: A script can choose its input files explicitly and, when appropriate, write output back to an input file.
Key terms: fileinput module, sys.argv, standard input, lineno, filelineno, filename, isfirstline, isstdin, nextfile, fileinput.input
11.2 Running scripts in different operating systems
11.2.1 Making a script directly executable on UNIX
Explains how to make a Python script directly executable on UNIX-like systems and notes how command-line redirection works there.
- Direct script execution on UNIX: Describe the steps for making a script directly executable on a UNIX-like system and how placing it on the path affects where it can be run from.
Why it matters: These steps let you launch a script by name rather than invoking Python explicitly each time. - Command-line input and output redirection: Explain what < and > do when used while running a script and identify the platform limitation noted here.
Why it matters: Redirection lets a script use files for input and output without changing the script itself.
Key terms: chmod, path, directly executable, <, >, redirected
11.2.2 Scripts on macOS
This section explains how to run Python scripts on macOS from a terminal or Finder and points to resources for Mac-specific scripting.
- macOS script launch options: Explain that macOS can run Python scripts from a terminal as on Linux/UNIX, or from Finder using Python Launcher.
Why it matters: You can choose a terminal or Finder workflow for running a script on a Mac. - Administrative scripting package guidance: State what kind of packages the section recommends investigating for macOS administrative scripts.
Why it matters: It points you toward relevant packages when ordinary Python scripting needs to interact with macOS automation.
Key terms: terminal window, Finder, Python Launcher, administrative scripts, Apple’s Open Scripting Architecture, Python
11.2.3 Script execution options in Windows
This section explains ways to run Python scripts on Windows, why a command window is especially flexible, and what to check if Python does not start.
- Command window script execution: Describe how to run a script from a Windows command window or PowerShell and why this option is especially flexible.
Why it matters: The command window gives you a way to run scripts while controlling their input and output. - Missing Python command diagnosis: Explain what it likely means if entering python3 at a Windows command prompt does not run Python, and state the remedies given.
Why it matters: Checking PATH can resolve a common obstacle before you try to run a script. - Alternative Windows execution options: Describe the other Windows script-running options mentioned: the Linux subsystem and making .py scripts automatically executable.
Why it matters: These alternatives show that starting a Windows script need not always mean invoking Python from a command window.
Key terms: command prompt, PowerShell, input and output redirection, python3, command path, Python executable, PATH environment variable, Linux subsystem for Windows, .py, magic extension
11.3 Programs and modules
This section shows how a controlling function can make a growing script usable both from the command line and as an importable module, with an optional regression-test mode.
- Controlling function separation: Explain why a growing script can benefit from separating its controlling function from its other functions, and describe how this helps when the script is also used as a module.
Why it matters: Separating the command-line interface makes functions easier to reuse without carrying the script’s user-facing behavior with them. - Main script execution guard: Explain what problem `if __name__ == "__main__":` prevents when a file serves as both a script and a module, and distinguish the file’s `__name__` in those two cases.
Why it matters: The guard lets someone import the file’s functions without accidentally launching its command-line interface. - Command-line controlling function duties: Describe how the example’s `main` function acts as a simple user interface by checking input, adapting it for `num2words`, and presenting a result or a useful error.
Why it matters: The working function can focus on its task while `main` handles how a person supplies input and sees the result. - Built-in regression test mode: Describe how the example’s `--test` mode uses a file of numbers and why its output is useful after changes to `num2words` or its supporting functions.
Why it matters: A repeatable set of inputs helps catch changes that break previously checked behavior.
Key terms: controlling function, scripts, modules, scripting plumbing, if __name__ == "__main__":, __name__, __main__, main, command-line arguments, num2words
11.4 Distributing Python applications
This section explains why simply sharing source files or bytecode is an imperfect way to distribute Python applications.
- Basic distribution tradeoffs: Explain the limitations of sharing Python source files or bytecode as an application. Describe why bundled source may be awkward for users unfamiliar with Python and why shipping only .pyc files is discouraged.
Why it matters: These limitations explain why distributing an application involves more than sending its code.
Key terms: source files, zip or tar file, bytecode, .pyc files
11.4.1 Wheels packages
Introduces wheels as the standard package format for distributing Python code and explains their intended benefits.
- Wheels for Python distribution: Explain that wheels are the current standard way to package and distribute Python modules and applications, and that they are designed to make installation more reliable and help manage dependencies.
Why it matters: This explains why wheels are a common format for distributing Python code.
Key terms: wheels, modules, applications, dependencies
11.4.2 zipapp and pex
This section explains how executable zip archives package multifile Python applications and compares zipapp with pex.
- Executable zip archive mechanics: Explain how Python runs a zip archive containing multiple modules and what makes the archive directly executable.
Why it matters: These details explain how a multifile application can be distributed as one ready-to-run file. - Zipapp and pex tools: Distinguish the standard-library zipapp module from pex as tools for creating executable zip applications.
Why it matters: The distinction helps you choose between a built-in packaging tool and a separately installed one with more options.
Key terms: __main__.py, entry point, sys.path, shebang line, executable zip file, zipapp, pex, standard library, pip
11.4.3 py2exe and py2app
This section identifies tools for creating standalone Python executables and explains their main trade-off.
- Standalone executable tools and trade-offs: Explain what py2exe and py2app produce, which platforms they target, and why a standalone executable may or may not be a good choice.
Why it matters: This helps you recognize when distributing a program without requiring Python may justify the drawbacks of a standalone executable.
Key terms: py2exe, py2app, standalone executables, native Python applications
11.4.4 Creating executable programs with freeze
This section explains how freeze creates a standalone executable from a Python program and why that distribution route may not be worth the effort.
- Freeze executable creation: Explain that freeze generates C files from a Python program, then requires a C compiler to compile and link them into an executable that can run without Python installed.
Why it matters: It clarifies both what freeze produces and the limits on where the result will run. - Standalone packaging tradeoff: Describe the practical requirements and caution the section gives for using freeze or similar standalone-packaging tools.
Why it matters: It helps you decide whether a standalone executable is worth the packaging effort.
Key terms: freeze, C files, C compiler, frozen application, Python source distribution, standalone application
11.5 Creating a program
This section sets out an exercise to refactor a Python version of wc so command-line options control which counts it displays.
- wc output option behavior: Describe how the refactored wc program should use -l, -w, and -c to choose which file statistics to display.
Why it matters: This behavior defines what the refactored program needs to do.
Key terms: wc, -l, -w, -c
11.5.1 Solving the problem with AI-generated code
This subsection explains how to give an AI chatbot the existing code and requirements for refactoring a word-count program.
- AI refactoring prompt setup: Describe how to prepare the chatbot to refactor the earlier program: give it the existing code, specify that the result should mimic Unix wc with -l, -w, -c, and -L, and use Copilot’s full chat interface when asking it to create a new file.
Why it matters: Giving the bot the starting code and the requested behavior makes the refactoring request specific.
Key terms: Colaboratory, Copilot, Unix wc utility, full chat interface
11.5.2 Solutions and discussion
11.6 Summary
Where this is used later
- Terminal input/output and redirection (chapter 13) · Terminal input/output and redirection builds on these standard-stream concepts.
- Solving the problem with AI-generated code (chapter 13) · Earlier treatment of redirecting a script’s input and output.
- Final fixes to wc (chapter 13) · The argparse module provides context for an optional filename argument.
- Development and debugging tools and runtime services (chapter 19) · The reader encounters more about testing Python code.
Concepts explained in this chapter
- command-line argumentA command-line argument is a value supplied to a program when it is invoked through a command-line interface.
- shell scriptA shell script is a text program interpreted by a command shell to execute commands and control operating-system operations.
- argparseargparse is Python's standard-library module for parsing command-line arguments and generating related help and error messages.
- argument parserArgumentParser is the argparse class that represents a command-line argument parser and defines how arguments are recognized and converted.
- maximum line lengthMaximum line length is the greatest number of characters or bytes in any single line of a text input.
- wc utilityThe wc utility is a Unix command-line tool that counts lines, words, and bytes in files or input streams and can report the maximum line length.
- input/output redirectionInput redirection is a shell mechanism that supplies a command's standard input from a file or another source instead of an interactive terminal.
- character countA character count is the number of characters contained in a file or text.
- comprehensionA list comprehension is a Python expression that constructs a list by iterating over an iterable and optionally filtering or transforming its elements.
- generator expressionA generator expression is a Python expression that lazily produces values from an iteration specification without constructing the complete result collection in memory.
- lines of codeA line count is the number of lines contained in a text file or text representation.
- word countA word count is the number of words identified in a text according to a specified word-segmentation rule.
- __name__ attributeThe __name__ attribute of a class contains the class's defining name.
- moduleA module is a distinct, bounded software unit that groups related code or functionality and provides an organizational or interaction boundary within a larger system.
- entry pointA main function, or controlling function, is a conventionally designated function that coordinates a program's execution and serves as its controlling entry point.
- functionA function is a callable program unit that performs an operation, may accept inputs, and can produce a result or side effect.
- evaluation suiteA regression test is a repeatable test that checks whether previously working behavior remains correct after code changes.
- standard inputsys.stdin is the Python standard-stream object that provides text input from the terminal or a redirected source.
- standard streamsStandard streams are the conventional input, output, and diagnostic communication channels provided to a running program.
- refactoringRefactoring is the systematic restructuring of existing source code to improve its internal qualities without changing its externally observable behavior.
- positional argumentA positional parameter is a function argument identified by its ordinal position in the argument list.
- conditional logicA conditional statement is a programming-language construct that selects code for execution based on a condition.
- conversational aiA chatbot is software that conducts conversational interactions with users through natural-language input and output.
- promptA prompt is an instruction or input supplied to an AI system to elicit a particular response or generated output.
- statementA statement is a syntactic unit representing an executable instruction in a programming language.
- else clauseAn else clause is the optional part of a try statement that executes only when its try block completes without raising an exception.
- import statementAn import statement is a source-code declaration that makes names or definitions from another module, package, or namespace available in the current scope.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.