Memory usage
Memory usage is the amount of computer memory occupied or allocated by a program, process, data, or computation during operation.
Fast Python: High performance techniques for large datasets
- How it is used
- Guides choices that keep data processing in memory, permit more concurrent processes, and control data-frame storage.
- How deep it goes
- Works through container-aware Python estimates and measured pandas type changes, while qualifying estimates for externally implemented objects.
- Examples
- Uses a station-file object graph, taxi-trip column measurements, and a CuPy matrix to expose different memory costs.
- What you can do after
- Apply it via pandas memory measurements and type selection, and diagnose Python container estimates without double-counting shared objects.
Read how Fast Python: High performance techniques for large datasets explains memory usage →
Troubleshooting Java, Second Edition
Memory usage is the amount of runtime memory occupied or allocated by a process or its components.
- How it is used
- Guides investigation of slow, frozen, or failing Java applications by relating memory consumption to allocation, retention, and garbage collection.
- How deep it goes
- Explains monitoring, sampling, leak patterns, and heap pressure, while deferring detailed memory instrumentation and GC-log analysis.
- Examples
- Uses VisualVM widgets, sampled Product objects, large video files, memory graphs, and repeated full-GC log entries.
- What you can do after
- Diagnose memory pressure with VisualVM sampling and GC graphs, distinguish leaks from insufficient allocation, and identify optimization targets.
Read how Troubleshooting Java, Second Edition explains memory usage →
Pandas Workout: 200 exercises to make you a stronger data analyst
Memory usage is the amount of computer memory occupied by a data structure or computation.
- How it is used
- Reduces pandas DataFrame resource consumption while preserving the data needed for analysis.
- How deep it goes
- Demonstrates import-time column selection and reports measured savings, while deferring detailed memory-measurement techniques.
- Examples
- Uses a 7.6-million-row New York City taxi dataset and a wide 264-column survey dataset.
- What you can do after
- Operate/configure pandas CSV imports to retain necessary columns and avoid overburdening available memory.
Read how Pandas Workout: 200 exercises to make you a stronger data analyst explains memory usage →
Graph Neural Networks in Action
The amount or proportion of available memory consumed by a computation.
- How it is used
- Uses memory consumption to identify GNN scaling problems alongside epoch time and convergence time.
- How deep it goes
- Defines memory usage, identifies graph storage demands, and shows monitoring methods; stops short of a complete memory budget.
- Examples
- Measures process resident set size with psutil and used NVIDIA GPU memory with pynvml.
- What you can do after
- Diagnose/debug with psutil and pynvml by measuring memory use before choosing a scaling strategy.
Read how Graph Neural Networks in Action explains memory usage →
The Art of Code
Memory footprint is the amount of system memory occupied by a program and its in-memory data during execution.
- How it is used
- Guides greener performance work by reducing resource use and measuring allocation behavior.
- How deep it goes
- Explains allocation-reducing code changes and single-stream collection, while deferring detailed profiler operation.
- Examples
- Compares boxed versus primitive age bounds, preallocated lists, separate streams, and nested teeing collectors over Person records.
- What you can do after
- Apply it by reducing avoidable allocation, preallocating collections, combining stream work, and selecting profiling tools.
Advanced Algorithms and Data Structures
Memory consumption is the amount of memory occupied by a computation or program during execution.
- How it is used
- Uses memory profiling and space estimates to compare data structures and judge practical trade-offs.
- How deep it goes
- Develops node-count reasoning, key-size formulas, profiling comparisons, and structural trade-offs without detailing allocator internals.
- Examples
- Compares BST and treap allocations for integer and string keys, and contrasts trie storage when strings share prefixes or do not.
- What you can do after
- Evaluate data-structure choices by estimating space from node counts, key lengths, overhead, and measured allocations.
Read how Advanced Algorithms and Data Structures explains memory usage →
AI for Everyday IT
Memory utilization is the proportion of available physical memory occupied by a system or workload.
- How it is used
- Calculates database memory utilization and identifies configurations exceeding a physical-memory threshold.
- How deep it goes
- Provides a worked SQL approach with unit conversion, conditional calculation, and threshold filtering rather than database internals.
- Examples
- Uses a configuration SELECT statement, a sample row, and a two-CTE calculation pattern.
- What you can do after
- Build it from scratch by prompting for a query that normalizes memory settings and reports utilization above 75%.
Related concepts
- quantizationaddressed by · Quantization maps numerical values to a finite set of discrete representations, often using reduced precision to lower model storage or computation requirements.
- gradient accumulationaddressed by · Gradient accumulation is a training technique that combines gradients from multiple smaller batches before applying an optimizer update to model parameters.
- pruningaddressed by · Pruning is the selective removal or restriction of parts of a model, structure, or candidate set, typically to reduce complexity or computation while retaining useful results.
- atomic typeconfigured by · An atomic type is a type whose supported accesses and operations are indivisible with respect to concurrent execution and provide defined synchronization semantics.
- categorical dataaddressed by · Categorical data consists of values representing membership in a discrete set of named or ordered categories rather than measured numerical magnitudes.
- BenchmarkDotNetmeasured by · BenchmarkDotNet is a .NET library and tool for measuring and analyzing managed-code performance.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.