36 lines
1.5 KiB
Markdown
36 lines
1.5 KiB
Markdown
|
# Taskpools
|
||
|
|
|
||
|
|
## API
|
||
|
|
|
||
|
|
The API spec follows https://github.com/nim-lang/RFCs/issues/347#task-parallelism-api
|
||
|
|
|
||
|
|
## Overview
|
||
|
|
|
||
|
|
This implements a lightweight, energy-efficient, easily auditable multithreaded taskpools.
|
||
|
|
|
||
|
|
This taskpools will be used in a highly security-sensitive blockchain application
|
||
|
|
targeted at resource-restricted devices hence desirable properties are:
|
||
|
|
|
||
|
|
- Ease of auditing and maintenance.
|
||
|
|
- Formally verified synchronization primitives are highly-sought after.
|
||
|
|
- Otherwise primitives are implemented from papers or ported from proven codebases
|
||
|
|
that can serve as reference for auditors.
|
||
|
|
- Resource-efficient. Threads spindown to save power, low memory use.
|
||
|
|
- Decent performance and scalability. The workload to parallelize are cryptography-related
|
||
|
|
and require at least 1ms runtime per thread.
|
||
|
|
This means that only a simple scheduler is required.
|
||
|
|
|
||
|
|
Non-goals:
|
||
|
|
- Supporting task priorities
|
||
|
|
- Being distributed
|
||
|
|
- Supporting GC-ed memory on Nim default GC (sequences and strings)
|
||
|
|
- Have async-awaitable tasks
|
||
|
|
|
||
|
|
In particular compared to [Weave](https://github.com/mratsim/weave), here are the tradeoffs:
|
||
|
|
- Taskpools only provide spawn/sync (task parallelism).\
|
||
|
|
There is no parallel for (data parallelism)\
|
||
|
|
or precise in/out dependencies (dataflow parallelism).
|
||
|
|
- Weave can handle trillions of small tasks that require only 10µs per task. (Load Balancing overhead)
|
||
|
|
- Weave maintains an adaptive memory pool to reduce memory allocation overhead,
|
||
|
|
Taskpools allocations are as-needed. (Scheduler overhead)
|