Distributed programming
Distributed programming is the design of software whose computation is coordinated across multiple networked machines, processors, or processes.
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
Distributed computing divides computation across multiple networked machines or processors.
- How it is used
- Uses pandas-style data preparation as a practical transition from single-CPU processing toward multiprocessing or distributed computing.
- How deep it goes
- Explains the transition and scalability rationale, but stops short of distributed-system implementation details.
- Examples
- Demonstrates joining three product tables with SQL and pandas merge operations.
- What you can do after
- Apply distributed-computing reasoning by identifying when pandas becomes inefficient or exceeds available memory.
Graph Neural Networks in Action
Distributed processing is the execution of computation across multiple cooperating devices or processes.
- How it is used
- Reduces training time by spreading large graph workloads across processors or machines when available hardware cannot process them efficiently.
- How deep it goes
- Explains workload distribution, training-loop synchronization, entry-point guards, process-group cleanup, and NCCL or Gloo selection.
- Examples
- Uses PyTorch training loops, dist.barrier(), process groups, NCCL, Gloo, processor nodes, and machine clusters as implementation examples.
- What you can do after
- Operate/configure distributed training with PyTorch communication primitives and choose NCCL or Gloo for GPU or CPU execution.
Read how Graph Neural Networks in Action explains distributed programming →
Effective Data Analysis
Distributed computing is the execution of computational work by dividing it across multiple networked computers that process tasks in parallel.
- How it is used
- Handles massive data ingestion and processing when traditional warehouses or local environments become impractical.
- How deep it goes
- Defines cluster-based parallelism, recombination, performance gains, and fault tolerance without detailing framework-specific implementation.
- Examples
- Uses billions- or trillions-record datasets, cluster nodes, parallel task chunks, and failed-node rerouting as examples.
Read how Effective Data Analysis explains distributed programming →
Related concepts
- XGBoostused by · XGBoost is an open-source software library for efficient gradient-boosted decision-tree algorithms used in supervised machine learning.
- model training serviceused by · A service that provides computation resources and job management for executing model-training code, producing trained models, and tracking training execution.
- scalabilityaddresses · Scalability is the ability of a system, process, or method to accommodate increasing workload, size, or demand while maintaining acceptable operation.
- parallelismimplements · Parallelism is the concurrent execution of multiple computational operations, often across multiple processing resources.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.