dask

1# Dask
2 
3## Overview
4 
5Dask is a Python library for parallel and distributed computing that enables three critical capabilities:
6- **Larger-than-memory execution** on single machines for data exceeding available RAM
7- **Parallel processing** for improved computational speed across multiple cores
8- **Distributed computation** supporting terabyte-scale datasets across multiple machines
9 
10Dask scales from laptops (processing ~100 GiB) to clusters (processing ~100 TiB) while maintaining familiar Python APIs.
11 
12## When to Use This Skill
13 
14This skill should be used when:
15- Process datasets that exceed available RAM
16- Scale pandas or NumPy operations to larger datasets
17- Parallelize computations for performance improvements
18- Process multiple files efficiently (CSVs, Parquet, JSON, text logs)
19- Build custom parallel workflows with task dependencies
20- Distribute workloads across multiple cores or machines
21 
22## Core Capabilities
23 
24Dask provides five main components, each suited to different use cases:
25 
26### 1. DataFrames - Parallel Pandas Operations
27 
28**Purpose**: Scale pandas operations to larger datasets through parallel processing.
29 
30**When to Use**:
31- Tabular data exceeds available RAM
32- Need to process multiple CSV/Parquet files together
33- Pandas operations are slow and need parallelization
34- Scaling from pandas prototype to production
35 
36**Reference Documentation**: For comprehensive guidance on Dask DataFrames, refer to `references/dataframes.md` which includes:
37…