Dask
Dask is an open-source Python library for parallel and distributed computing using task graphs and scalable data structures for large or partitioned workloads.
Fast Python: High performance techniques for large datasets
- How it is used
- Scales parallel analytics across partitioned data when computation or datasets strain one machine.
- How deep it goes
- Works through lazy graphs, partitions, distributed scheduling, storage, and memory bottlenecks, with examples rather than a complete API reference.
- Examples
- Uses a tax data-frame calculation and a blockwise Mandelbrot image, including measured runtimes with and without persisted intermediates.
- What you can do after
- Apply it via Dask data frames, arrays, and dask.distributed; inspect task graphs and dashboards to diagnose partitioning and memory costs.
Read how Fast Python: High performance techniques for large datasets explains Dask →
Starting Data Analytics with Generative AI and Python
Dask is a Python library for parallel and distributed computing on datasets and computations that may exceed the resources of a single machine.
- How it is used
- Scales data-analysis calculations across CPUs, GPUs, and cloud clusters when growing data or complexity limits timely results.
- How deep it goes
- Explains Dask’s scaling role and cloud-provider pattern, but stops short of detailed cluster setup or implementation code.
- Examples
- Uses Brazilian e-commerce orders, millions of records, and Dask clusters on AWS, Google Cloud Platform, or Microsoft Azure.
- What you can do after
- Apply it via Dask to distribute calculations across cloud resources with minimal modifications to existing analysis code.
Read how Starting Data Analytics with Generative AI and Python explains Dask →
Outlier Detection in Python
Dask is a Python library for parallel and distributed computing with larger-than-memory and partitioned data.
- How it is used
- Supports distributed collection, preprocessing, and basic univariate outlier screening for datasets larger than memory.
- How deep it goes
- Demonstrates a concrete preprocessing and screening workflow but stops short of advanced outlier-detection algorithms.
- Examples
- Uses the KDD Cup dataset to encode features, partition records, scale values robustly, and identify rows with large values.
- What you can do after
- Build a pandas-to-Dask preprocessing and univariate screening pipeline for distributed, larger-than-memory tabular data.
Related concepts
- pandasalternative to · pandas is an open-source Python library that provides labeled data structures and tools for manipulating and analyzing tabular and other structured data.
- lazy evaluationuses · Lazy evaluation is an evaluation strategy that defers computing an expression or value until its result is needed, potentially avoiding the computation entirely.
- parallelismuses · Parallelism is the concurrent execution of multiple computational operations, often across multiple processing resources.
- Sparkalternative to · Apache Spark is an open-source distributed computing engine for large-scale data processing and analytics across clusters, supporting batch and streaming workloads.
- PySparkalternative to · PySpark is the Python interface to Apache Spark for distributed data processing and analytics.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.