Chapter 3: Working with Node.js and npm
The introduction explains why JavaScript suited Node.js’s original server goal and why Node.js and npm matter beyond server applications.
After this chapter you will be able to
- Non-blocking IO motivation. Explain why Ryan Dahl chose JavaScript for Node.js and what non-blocking IO was intended to improve.
- Tools across JavaScript projects. Explain why Node.js and npm can matter even for JavaScript projects aimed at the browser.
- Blocking IO and thread costs. Explain why a blocking file read wastes server resources and how threads address that problem at a cost.
- Single-threaded non-blocking reads. Trace how a Node.js request handler uses a callback with fs.readFile, and explain why JavaScript code must not block the thread.
- Node.js REPL interaction. Describe how to start the Node.js REPL, run JavaScript directly, and interpret the value it displays after execution.
- Node.js global object. Explain what inspecting the global object in the REPL reveals about the difference between Node.js and browser APIs.
- Readline in a command-line calculator. Explain what the readline API contributes to the calculator and why the example uses its promise-based flavor.
- Meaning of the mjs extension. Explain why the example is named index.mjs rather than index.js, and distinguish the module standards and keywords associated with each extension.
- Modules and node_modules convention. Explain that Node.js popularized modules as JavaScript files with their own namespace and introduced node_modules as a convention for loading modules without specifying complete file paths.
- npm and public registry. Describe npm as a Node.js package manager introduced alongside a public registry, and explain their effect on sharing JavaScript code.
- npm installation results. Trace what happens when npm installs Express: it downloads the named package from the public registry, recursively obtains its dependencies, and records the results in the project.
- Import forms and package search. Distinguish core API imports, relative-path imports, and package-name imports, then explain how Node.js searches for a package named by a bare specifier.
- Package entry point selection. Trace how Node.js selects a package entry point after finding its directory, distinguishing exports, a path within the import, and main.
- Nested dependencies and hoisting. Explain why npm may create nested node_modules directories for indirect dependencies and when it can hoist them to the top level.
- Lockfile and dependency roles. Explain why projects normally commit package-lock.json rather than node_modules, and distinguish the lockfile’s exact dependency tree from the version ranges in package.json.
- Semantic versions and caret ranges. Describe the parts of a semver version and explain which releases a caret range such as ^5.1.0 accepts under the compatibility convention.
- Within-range and latest updates. Distinguish updating a dependency within its package.json range using npm update --save <package> from installing its latest release regardless of that range using npm install <package>@latest.
- Forking a dependency. Describe how to modify a dependency through a fork and have npm install that fork.
- Persistent dependency patches. Explain why direct edits to node_modules are unstable and describe how patch-package makes a small dependency change persist across installs.
- JavaScript beyond the browser. Explain how Node.js expanded the kinds of tasks developers could use JavaScript for, including tasks they had previously handled with shell scripts or Python or Ruby.
- Defining and running scripts. Describe how to define a short script in the scripts section of package.json and run it with npm run <script>.
- Automatic script hooks. Explain when pre- and post-prefixed scripts run, including the build example, and describe the install-related life cycle scripts mentioned here.
- Package executables and script reuse. Explain how scripts can use executables from installed packages and invoke other npm scripts.
- Executable Node.js scripts. Describe how a shebang and executable permission let a JavaScript file run directly on Mac/Linux instead of being passed to the node command.
- Running installed package commands. Describe how npx runs an executable installed in the local project without requiring a package.json scripts entry.
- Registry executable fallback. Trace what npx does when a command is not a local executable, including the conditions for running a registry package and why this is useful for infrequent commands.
Section by section
Introduction
The introduction explains why JavaScript suited Node.js’s original server goal and why Node.js and npm matter beyond server applications.
- Non-blocking IO motivation: Explain why Ryan Dahl chose JavaScript for Node.js and what non-blocking IO was intended to improve.
Why it matters: This connects Node.js’s original purpose to the choice of JavaScript. - Node.js beyond servers: Describe how Node.js’s use expanded beyond its original server purpose.
Why it matters: It explains why Node.js is relevant to more than server development. - Tools across JavaScript projects: Explain why Node.js and npm can matter even for JavaScript projects aimed at the browser.
Why it matters: It distinguishes a project’s development tools from where its finished JavaScript runs.
Key terms: non-blocking IO, asynchronous server logic, Node.js, JavaScript, npm, target runtime, dependencies
3.1 The Node.js runtime
The section contrasts blocking file reads with Node.js’s single-threaded, non-blocking approach to handling requests.
- Blocking IO and thread costs: Explain why a blocking file read wastes server resources and how threads address that problem at a cost.
Why it matters: This is the problem Node.js’s non-blocking approach is meant to avoid. - Single-threaded non-blocking reads: Trace how a Node.js request handler uses a callback with fs.readFile, and explain why JavaScript code must not block the thread.
Why it matters: Non-blocking reads help a single JavaScript thread serve other requests while IO is pending. - Promise-based IO expression: Describe what promises and async/await change about expressing the file-read logic, and what behavior stays the same.
Why it matters: The syntax can change without giving up non-blocking IO.
Key terms: blocking IO, threads, overhead, single thread, callback, fs.readFile, non-blocking IO, Don’t block the thread!, promises, async/await
3.1.1 Running the Node.js REPL
This subsection shows how to explore Node.js in its REPL and why its global object looks different from a browser’s.
- Node.js REPL interaction: Describe how to start the Node.js REPL, run JavaScript directly, and interpret the value it displays after execution.
Why it matters: The REPL lets you try JavaScript in Node.js and understand the value it prints after each command. - Node.js global object: Explain what inspecting the global object in the REPL reveals about the difference between Node.js and browser APIs.
Why it matters: This distinction explains why you should not expect browser-style globals when exploring Node.js.
Key terms: node command, REPL, return value, console.log, global object, global variables, window, modules
3.1.2 Using Node.js APIs
This section uses a command-line calculator to introduce Node.js’s readline API and explains why the example file has an .mjs extension.
- Readline in a command-line calculator: Explain what the readline API contributes to the calculator and why the example uses its promise-based flavor.
Why it matters: It connects a Node.js API to a concrete command-line use. - Meaning of the mjs extension: Explain why the example is named index.mjs rather than index.js, and distinguish the module standards and keywords associated with each extension.
Why it matters: The extension tells Node.js which module standard to use for the example file. - Calculator input restriction: Describe the risk noted for unrestricted calculator input and the suggested kind of restriction.
Why it matters: It flags a limitation of the simple calculator without requiring a security implementation.
Key terms: readline API, callback-based, promise-based, REPL, .mjs, CommonJS, ECMAScript Modules (ESM), .js, require, import
3.2 Managing Node.js dependencies
Introduces how Node.js modules, node_modules, and npm made JavaScript code easier to load and share.
- Modules and node_modules convention: Explain that Node.js popularized modules as JavaScript files with their own namespace and introduced node_modules as a convention for loading modules without specifying complete file paths.
Why it matters: This gives you the basic vocabulary for understanding how Node.js projects use other code. - npm and public registry: Describe npm as a Node.js package manager introduced alongside a public registry, and explain their effect on sharing JavaScript code.
Why it matters: It explains why npm became useful for managing code shared across projects.
Key terms: modules, namespace, node_modules, npm, public registry, package manager
3.2.1 Installing dependencies with npm
The section creates a Node.js project, installs Express with npm, and runs a small server to use the new dependency.
- Project dependency manifest: Explain that initializing the project creates a skeletal package.json and that this example uses it to save dependencies rather than to prepare a package for distribution.
Why it matters: It explains why the project needs a package.json before adding Express. - npm installation results: Trace what happens when npm installs Express: it downloads the named package from the public registry, recursively obtains its dependencies, and records the results in the project.
Why it matters: Installing one package can bring in many others and change several project files. - Express server check: Describe why the example installs Express and how it checks that the resulting server runs.
Why it matters: It connects installing a dependency to using it in a running application.
Key terms: package.json, dependencies, npm registry, dependency tree, node_modules, package-lock.json, express, HTTP APIs, index.mjs, ESM module
3.2.2 Understanding node_modules
This section explains how Node.js resolves package imports through node_modules, how npm arranges dependencies there, and how local packages can be linked for import.
- Import forms and package search: Distinguish core API imports, relative-path imports, and package-name imports, then explain how Node.js searches for a package named by a bare specifier.
Why it matters: The form of an import tells you where Node.js will look for its code. - Package entry point selection: Trace how Node.js selects a package entry point after finding its directory, distinguishing exports, a path within the import, and main.
Why it matters: Finding a package directory is not enough; Node.js must also decide which code the import names. - Nested dependencies and hoisting: Explain why npm may create nested node_modules directories for indirect dependencies and when it can hoist them to the top level.
Why it matters: The directory layout explains how indirect dependencies can coexist without every importer receiving the same version. - Local package symlinks: Describe how npm link and workspaces make other local packages importable by package name.
Why it matters: Links let locally developed packages be imported as dependencies without copying their code into the project.
Key terms: core Node.js API, relative file path, bare specifier, node_modules, entry point, package.json, exports, main, indirect dependencies, nested node_modules
3.2.3 Syncing and updating dependencies
Explains how npm keeps dependency installations reproducible while allowing compatible updates, and how to update packages within or beyond their declared version ranges.
- Lockfile and dependency roles: Explain why projects normally commit package-lock.json rather than node_modules, and distinguish the lockfile’s exact dependency tree from the version ranges in package.json.
Why it matters: This distinction explains how a team can install consistent dependencies without committing a large node_modules tree. - Semantic versions and caret ranges: Describe the parts of a semver version and explain which releases a caret range such as ^5.1.0 accepts under the compatibility convention.
Why it matters: Caret ranges let a project receive compatible fixes and improvements without automatically accepting a new major release. - Within-range and latest updates: Distinguish updating a dependency within its package.json range using npm update --save <package> from installing its latest release regardless of that range using npm install <package>@latest.
Why it matters: Choosing the right command keeps the installed version, lockfile, and readable version history aligned with the intended update. - Lockfiles in consumed packages: Explain why using a lockfile in a package consumed by other projects is debated.
Why it matters: A package maintainer’s reproducible local installation does not imply consumers will install the same dependency tree.
Key terms: package-lock.json, package.json, node_modules, lockfile, version ranges, semver, major, minor, patch, pre-release
3.2.4 Forking and patching dependencies
This section explains when to fork a dependency and how to preserve a smaller change with patch-package.
- Forking a dependency: Describe how to modify a dependency through a fork and have npm install that fork.
Why it matters: A fork lets you use and maintain your own version of an open-source dependency. - Persistent dependency patches: Explain why direct edits to node_modules are unstable and describe how patch-package makes a small dependency change persist across installs.
Why it matters: A recorded patch lets collaborators receive a small dependency fix without maintaining a full fork. - Development dependency distinction: Distinguish a development dependency from a runtime dependency and state why patch-package is installed as the former.
Why it matters: This distinction keeps a development tool separate from packages required at runtime.
Key terms: fork, package.json, git link, npm install, node_modules, patch-package, .patch file, postinstall, --save-dev, devDependencies
3.3 Scripting with Node.js
This introduction explains why Node.js made JavaScript a practical language for scripting beyond the browser.
- JavaScript beyond the browser: Explain how Node.js expanded the kinds of tasks developers could use JavaScript for, including tasks they had previously handled with shell scripts or Python or Ruby.
Why it matters: This explains why developers might choose Node.js for scripts rather than switch languages.
Key terms: Node.js, JavaScript, shell scripting
3.3.1 Defining scripts in package.json
This section shows how to define and run npm scripts in package.json, including arguments, special script names, package executables, and scripts that call other scripts.
- Defining and running scripts: Describe how to define a short script in the scripts section of package.json and run it with npm run <script>.
Why it matters: Named scripts give a project a simple way to run its commands. - Script arguments and shorthand: Explain how to pass arguments to a script without having npm interpret them, and identify the script names that allow the npm <script> shorthand.
Why it matters: These conventions help you pass arguments to the intended command and use npm’s shortcuts correctly. - Automatic script hooks: Explain when pre- and post-prefixed scripts run, including the build example, and describe the install-related life cycle scripts mentioned here.
Why it matters: Script names can cause related commands to run automatically, not just when invoked individually. - Package executables and script reuse: Explain how scripts can use executables from installed packages and invoke other npm scripts.
Why it matters: Scripts can call project-installed tools and combine existing scripts.
Key terms: scripts, package.json, npm run, -- separator, npm <script>, test, start, restart, stop, prebuild
3.3.2 Writing executable scripts with Node.js
This section explains how to run Node.js files as executables, use dependencies and shell commands in scripts, and package a script for environments where dependencies cannot be installed.
- Executable Node.js scripts: Describe how a shebang and executable permission let a JavaScript file run directly on Mac/Linux instead of being passed to the node command.
Why it matters: It lets you launch a Node.js script like a system executable. - Script import resolution: Explain where Node.js resolves a script’s imports and why this helps scripts distributed in npm packages.
Why it matters: A packaged script can use its dependencies without changing how imports work when the script is launched directly. - Shell commands from scripts: Distinguish Node.js’s built-in way to call system commands from the more succinct approach offered by packages such as zx.
Why it matters: It explains why a script author might choose a shell-scripting package despite Node.js having built-in command APIs. - Bundled script deployment: Explain when bundling a Node.js script is useful and what the resulting file contains.
Why it matters: It offers a way to run a script where fetching its npm dependencies is impractical.
Key terms: shebang, chmod +x, executable permissions, import statements, node_modules, module resolution, child_process, zx, $, bundler
3.3.3 Running package executables with npx
This section explains how npm packages expose command-line executables and how npx runs them locally or temporarily from the registry.
- Package executable declarations: Explain how a package exposes a JavaScript file as a command-line executable through a bin entry in package.json, and what npm creates when the package is installed.
Why it matters: It explains where the commands run by scripts and npx come from. - Running installed package commands: Describe how npx runs an executable installed in the local project without requiring a package.json scripts entry.
Why it matters: It gives you a direct way to run an installed package’s command. - Registry executable fallback: Trace what npx does when a command is not a local executable, including the conditions for running a registry package and why this is useful for infrequent commands.
Why it matters: It explains how to use an occasional command without making its package a project dependency.
Key terms: bin, package.json, executable, npx, package executables, scripts entry, npx <command>, registry, node_modules, lockfile
3.4 Summary
Where this is used later
- Importing and exporting in ESM (chapter 13) · Assumes familiarity with Node.js as a runtime.
- How await works (chapter 8) · Develops the async/await mechanics only mentioned here.
- Bridging ESM with CommonJS (chapter 13) · Provides the initial distinction between ESM and CommonJS.
- Resolving imports with entry points and maps (chapter 13) · Provides the package-search behavior needed when considering how imports are resolved.
- Building JS bundles (chapter 13) · develops the broader topic of building JavaScript bundles
Concepts explained in this chapter
- node_modulesThe conventional directory in a Node.js project that contains installed JavaScript packages.
- npmThe package manager and registry ecosystem commonly used to install, publish, and manage JavaScript packages.
- Node.jsA standalone JavaScript runtime built around the V8 engine and designed for server-side and general-purpose applications.
- package.jsonpackage.json is a project manifest file that records package metadata, scripts, and dependencies for JavaScript package tools.
- dependency treeA hierarchical representation of a package's direct and transitive dependencies.
- package-lock.jsonA generated npm lockfile that records the resolved dependency tree and package versions for a project.
- bare specifierA bare specifier is a module import specifier expressed as a package name rather than a relative or absolute file path.
- exports fieldThe exports field is a package.json field that defines which package subpaths and entry points can be imported.
- dependencyA software component or package that another component requires in order to build or run.
- development dependencyA project dependency needed for development activities such as building and testing, but not normally at application runtime.
- ECMAScript moduleA JavaScript module whose dependencies and shared bindings are declared using standardized import and export syntax.
- CommonJSCommonJS is a JavaScript module-system specification traditionally based on synchronous require calls and module.exports assignments.
- JavaScriptJavaScript is a high-level programming language standardized as ECMAScript and widely used for web and general-purpose software development.
- async/awaitJavaScript syntax for expressing asynchronous control flow using async functions and await expressions.
- callbackA function supplied to another computation to be invoked when a specified event occurs or operation completes.
- promiseThe ECMAScript constructor and associated object abstraction for representing the eventual completion or failure of an asynchronous operation.
- import statementThe import keyword statically declares dependencies on exported bindings from other ECMAScript modules.
- bundlerA build tool that combines source modules and their dependencies into distributable output files.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
