# Dask Task Graph
### Key Points
* Dask schedulers expect arbitrary tasks graphs where each node is a single python function call and each edge is a dependency between two function calls. These are generally stored in flat dictionaries.
* The task graph is a dictionary that stores every pandas-level function call necessary to compute the final result.
* Dask's high level graphs help us explicitly encode this structure by storing our task graphs in layers with dependencies between layers
### Specification
* A **dask graph** is a dictionary mapping **keys** to **computation**
```python
{'x': 1,
'y': 2,
'z': (add, 'x', 'y'),
'w': (sum, ['x', 'y', 'z']),
'v': [(sum, ['w', 'z']), 2]}
```
* A **key** is any hashable value that is not a **task** (for instance, above the keys are `x`, `y`, `z`,...)
* A **task** is a tuple with a callable first element. Tasks represent atomic units of work meant to be run by a single worker. For example:
```python
(add, 'x', 'y')
```
### What makes that a high-level graph? What is a low-level graph?
That graph is “high level” because it comes from user-supplied method calls and operators and, in many cases, it is a bit too abstract for direct Dask scheduling. Recall that Dask tasks, when actually scheduled on workers, are just regular Python functions operating over regular Python objects.
In the first example above, that Dask Array is not a “regular” Python object: it really represents a collection of NumPy Arrays, and translation from the Dask abstraction to the concrete NumPy operations needs to happen prior to task scheduling and execution.
A low-level graph is a graph of operations that are “ready to schedule/execute” — that is, **they represent regular Python functions on regular Python objects**.
---
Date: 20210902
Links to:
Tags:
References:
* [High level graphs docs](https://docs.dask.org/en/latest/high-level-graphs.html)
* [Specification docs](https://docs.dask.org/en/latest/spec.html)
* [Dask under the hood scheduler refactor](https://coiled.io/blog/dask-under-the-hood-scheduler-refactor/)