status-im
/
nim-chronos
mirror of https://github.com/status-im/nim-chronos.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity

move docs to docs (#466) 

* introduce user guide based on `mdbook`
* set up structure for adding simple `chronos` usage examples
* move most readme content to book
* ci deploys book and api guide automatically
* remove most of existing engine docs (obsolete)
This commit is contained in:
Jacek Sieka 2023-11-15 09:06:37 +01:00 committed by GitHub
parent 9c93ab48de
commit 24be151cf3
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
29 changed files with 811 additions and 643 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

40
.appveyor.yml
View File

@ -1,40 +0,0 @@
version: '{build}'
image: Visual Studio 2015
cache:
- NimBinaries
matrix:
# We always want 32 and 64-bit compilation
fast_finish: false
platform:
- x86
- x64
# when multiple CI builds are queued, the tested commit needs to be in the last X commits cloned with "--depth X"
clone_depth: 10
install:
# use the newest versions documented here: https://www.appveyor.com/docs/windows-images-software/#mingw-msys-cygwin
- IF "%PLATFORM%" == "x86" SET PATH=C:\mingw-w64\i686-6.3.0-posix-dwarf-rt_v5-rev1\mingw32\bin;%PATH%
- IF "%PLATFORM%" == "x64" SET PATH=C:\mingw-w64\x86_64-8.1.0-posix-seh-rt_v6-rev0\mingw64\bin;%PATH%
# build nim from our own branch - this to avoid the day-to-day churn and
# regressions of the fast-paced Nim development while maintaining the
# flexibility to apply patches
- curl -O -L -s -S https://raw.githubusercontent.com/status-im/nimbus-build-system/master/scripts/build_nim.sh
- env MAKE="mingw32-make -j2" ARCH_OVERRIDE=%PLATFORM% bash build_nim.sh Nim csources dist/nimble NimBinaries
- SET PATH=%CD%\Nim\bin;%PATH%
build_script:
- cd C:\projects\%APPVEYOR_PROJECT_SLUG%
- nimble install -y --depsOnly
- nimble install -y libbacktrace
test_script:
- nimble test
deploy: off

1
.github/workflows/ci.yml vendored
View File

@ -165,3 +165,4 @@ jobs:
nimble install -y libbacktrace nimble install -y libbacktrace
nimble test nimble test
nimble test_libbacktrace nimble test_libbacktrace
nimble examples

56
.github/workflows/doc.yml vendored
View File

@ -18,6 +18,26 @@ jobs:
uses: actions/checkout@v3 uses: actions/checkout@v3
with: with:
submodules: true submodules: true
- uses: actions-rs/install@v0.1
with:
crate: mdbook
use-tool-cache: true
version: "0.4.35"
- uses: actions-rs/install@v0.1
with:
crate: mdbook-toc
use-tool-cache: true
version: "0.14.1"
- uses: actions-rs/install@v0.1
with:
crate: mdbook-open-on-gh
use-tool-cache: true
version: "2.4.1"
- uses: actions-rs/install@v0.1
with:
crate: mdbook-admonish
use-tool-cache: true
version: "1.13.1"
- uses: jiro4989/setup-nim-action@v1 - uses: jiro4989/setup-nim-action@v1
with: with:
@ -28,35 +48,11 @@ jobs:
nim --version nim --version
nimble --version nimble --version
nimble install -dy nimble install -dy
# nim doc can "fail", but the doc is still generated nimble docs || true
nim doc --git.url:https://github.com/status-im/nim-chronos --git.commit:master --outdir:docs --project chronos || true
# check that the folder exists - name: Deploy
ls docs uses: peaceiris/actions-gh-pages@v3
- name: Clone the gh-pages branch
uses: actions/checkout@v3
with: with:
repository: status-im/nim-chronos github_token: ${{ secrets.GITHUB_TOKEN }}
ref: gh-pages publish_dir: ./docs/book
path: subdoc force_orphan: true
submodules: true
fetch-depth: 0
- name: Commit & push
run: |
cd subdoc
# Update / create this branch doc
rm -rf docs
mv ../docs .
# Remove .idx files
# NOTE: git also uses idx files in his
# internal folder, hence the `*` instead of `.`
find * -name "*.idx" -delete
git add .
git config --global user.email "${{ github.actor }}@users.noreply.github.com"
git config --global user.name = "${{ github.actor }}"
git commit -a -m "update docs"
git push origin gh-pages

1
.gitignore vendored
View File

@ -4,4 +4,3 @@ nimble.develop
nimble.paths nimble.paths
/build/ /build/
nimbledeps nimbledeps
/docs

27
.travis.yml
View File

@ -1,27 +0,0 @@
language: c
# https://docs.travis-ci.com/user/caching/
cache:
directories:
- NimBinaries
git:
# when multiple CI builds are queued, the tested commit needs to be in the last X commits cloned with "--depth X"
depth: 10
os:
- linux
- osx
install:
# build nim from our own branch - this to avoid the day-to-day churn and
# regressions of the fast-paced Nim development while maintaining the
# flexibility to apply patches
- curl -O -L -s -S https://raw.githubusercontent.com/status-im/nimbus-build-system/master/scripts/build_nim.sh
- env MAKE="make -j2" bash build_nim.sh Nim csources dist/nimble NimBinaries
- export PATH="$PWD/Nim/bin:$PATH"
script:
- nimble install -y
- nimble test

453
README.md
View File

@ -14,11 +14,11 @@ Chronos is an efficient [async/await](https://en.wikipedia.org/wiki/Async/await)
* Synchronization primitivies like queues, events and locks * Synchronization primitivies like queues, events and locks
* Cancellation * Cancellation
* Efficient dispatch pipeline with excellent multi-platform support * Efficient dispatch pipeline with excellent multi-platform support
* Exception effect support (see [exception effects](#exception-effects)) * Exceptional error handling features, including `raises` tracking
## Installation ## Getting started
You can use Nim's official package manager Nimble to install Chronos: Install `chronos` using `nimble`:
```text ```text
nimble install chronos nimble install chronos
@ -30,6 +30,30 @@ or add a dependency to your `.nimble` file:
requires "chronos" requires "chronos"
``` ```
and start using it:
```nim
import chronos/apps/http/httpclient
proc retrievePage(uri: string): Future[string] {.async.} =
# Create a new HTTP session
let httpSession = HttpSessionRef.new()
try:
# Fetch page contents
let resp = await httpSession.fetch(parseUri(uri))
# Convert response to a string, assuming its encoding matches the terminal!
bytesToString(resp.data)
finally: # Close the session
await noCancel(httpSession.closeWait())
echo waitFor retrievePage(
"https://raw.githubusercontent.com/status-im/nim-chronos/master/README.md")
```
## Documentation
See the [user guide](https://status-im.github.io/nim-chronos/).
## Projects using `chronos` ## Projects using `chronos`
* [libp2p](https://github.com/status-im/nim-libp2p) - Peer-to-Peer networking stack implemented in many languages * [libp2p](https://github.com/status-im/nim-libp2p) - Peer-to-Peer networking stack implemented in many languages
@ -42,426 +66,7 @@ requires "chronos"
Submit a PR to add yours! Submit a PR to add yours!
## Documentation
### Concepts
Chronos implements the async/await paradigm in a self-contained library using
the macro and closure iterator transformation features provided by Nim.
The event loop is called a "dispatcher" and a single instance per thread is
created, as soon as one is needed.
To trigger a dispatcher's processing step, we need to call `poll()` - either
directly or through a wrapper like `runForever()` or `waitFor()`. Each step
handles any file descriptors, timers and callbacks that are ready to be
processed.
`Future` objects encapsulate the result of an `async` procedure upon successful
completion, and a list of callbacks to be scheduled after any type of
completion - be that success, failure or cancellation.
(These explicit callbacks are rarely used outside Chronos, being replaced by
implicit ones generated by async procedure execution and `await` chaining.)
Async procedures (those using the `{.async.}` pragma) return `Future` objects.
Inside an async procedure, you can `await` the future returned by another async
procedure. At this point, control will be handled to the event loop until that
future is completed.
Future completion is tested with `Future.finished()` and is defined as success,
failure or cancellation. This means that a future is either pending or completed.
To differentiate between completion states, we have `Future.failed()` and
`Future.cancelled()`.
### Dispatcher
You can run the "dispatcher" event loop forever, with `runForever()` which is defined as:
```nim
proc runForever*() =
while true:
poll()
```
You can also run it until a certain future is completed, with `waitFor()` which
will also call `Future.read()` on it:
```nim
proc p(): Future[int] {.async.} =
await sleepAsync(100.milliseconds)
return 1
echo waitFor p() # prints "1"
```
`waitFor()` is defined like this:
```nim
proc waitFor*[T](fut: Future[T]): T =
while not(fut.finished()):
poll()
return fut.read()
```
### Async procedures and methods
The `{.async.}` pragma will transform a procedure (or a method) returning a
specialised `Future` type into a closure iterator. If there is no return type
specified, a `Future[void]` is returned.
```nim
proc p() {.async.} =
await sleepAsync(100.milliseconds)
echo p().type # prints "Future[system.void]"
```
Whenever `await` is encountered inside an async procedure, control is passed
back to the dispatcher for as many steps as it's necessary for the awaited
future to complete successfully, fail or be cancelled. `await` calls the
equivalent of `Future.read()` on the completed future and returns the
encapsulated value.
```nim
proc p1() {.async.} =
await sleepAsync(1.seconds)
proc p2() {.async.} =
await sleepAsync(1.seconds)
proc p3() {.async.} =
let
fut1 = p1()
fut2 = p2()
# Just by executing the async procs, both resulting futures entered the
# dispatcher's queue and their "clocks" started ticking.
await fut1
await fut2
# Only one second passed while awaiting them both, not two.
waitFor p3()
```
Don't let `await`'s behaviour of giving back control to the dispatcher surprise
you. If an async procedure modifies global state, and you can't predict when it
will start executing, the only way to avoid that state changing underneath your
feet, in a certain section, is to not use `await` in it.
### Error handling
Exceptions inheriting from [`CatchableError`](https://nim-lang.org/docs/system.html#CatchableError)
interrupt execution of the `async` procedure. The exception is placed in the
`Future.error` field while changing the status of the `Future` to `Failed`
and callbacks are scheduled.
When a future is awaited, the exception is re-raised, traversing the `async`
execution chain until handled.
```nim
proc p1() {.async.} =
await sleepAsync(1.seconds)
raise newException(ValueError, "ValueError inherits from CatchableError")
proc p2() {.async.} =
await sleepAsync(1.seconds)
proc p3() {.async.} =
let
fut1 = p1()
fut2 = p2()
await fut1
echo "unreachable code here"
await fut2
# `waitFor()` would call `Future.read()` unconditionally, which would raise the
# exception in `Future.error`.
let fut3 = p3()
while not(fut3.finished()):
poll()
echo "fut3.state = ", fut3.state # "Failed"
if fut3.failed():
echo "p3() failed: ", fut3.error.name, ": ", fut3.error.msg
# prints "p3() failed: ValueError: ValueError inherits from CatchableError"
```
You can put the `await` in a `try` block, to deal with that exception sooner:
```nim
proc p3() {.async.} =
let
fut1 = p1()
fut2 = p2()
try:
await fut1
except CachableError:
echo "p1() failed: ", fut1.error.name, ": ", fut1.error.msg
echo "reachable code here"
await fut2
```
Because `chronos` ensures that all exceptions are re-routed to the `Future`,
`poll` will not itself raise exceptions.
`poll` may still panic / raise `Defect` if such are raised in user code due to
undefined behavior.
#### Checked exceptions
By specifying a `raises` list to an async procedure, you can check which
exceptions can be raised by it:
```nim
proc p1(): Future[void] {.async: (raises: [IOError]).} =
assert not (compiles do: raise newException(ValueError, "uh-uh"))
raise newException(IOError, "works") # Or any child of IOError
proc p2(): Future[void] {.async, (raises: [IOError]).} =
await p1() # Works, because await knows that p1
# can only raise IOError
```
Under the hood, the return type of `p1` will be rewritten to an internal type
which will convey raises informations to `await`.
#### The `Exception` type
Exceptions deriving from `Exception` are not caught by default as these may
include `Defect` and other forms undefined or uncatchable behavior.
Because exception effect tracking is turned on for `async` functions, this may
sometimes lead to compile errors around forward declarations, methods and
closures as Nim conservatively asssumes that any `Exception` might be raised
from those.
Make sure to excplicitly annotate these with `{.raises.}`:
```nim
# Forward declarations need to explicitly include a raises list:
proc myfunction() {.raises: [ValueError].}
# ... as do `proc` types
type MyClosure = proc() {.raises: [ValueError].}
proc myfunction() =
raise (ref ValueError)(msg: "Implementation here")
let closure: MyClosure = myfunction
```
For compatibility, `async` functions can be instructed to handle `Exception` as
well, specifying `handleException: true`. `Exception` that is not a `Defect` and
not a `CatchableError` will then be caught and remapped to
`AsyncExceptionError`:
```nim
proc raiseException() {.async: (handleException: true, raises: [AsyncExceptionError]).} =
raise (ref Exception)(msg: "Raising Exception is UB")
proc callRaiseException() {.async: (raises: []).} =
try:
raiseException()
except AsyncExceptionError as exc:
# The original Exception is available from the `parent` field
echo exc.parent.msg
```
This mode can be enabled globally with `-d:chronosHandleException` as a help
when porting code to `chronos` but should generally be avoided as global
configuration settings may interfere with libraries that use `chronos` leading
to unexpected behavior.
### Raw functions
Raw functions are those that interact with `chronos` via the `Future` type but
whose body does not go through the async transformation.
Such functions are created by adding `raw: true` to the `async` parameters:
```nim
proc rawAsync(): Future[void] {.async: (raw: true).} =
let future = newFuture[void]("rawAsync")
future.complete()
return future
```
Raw functions must not raise exceptions directly - they are implicitly declared
as `raises: []` - instead they should store exceptions in the returned `Future`:
```nim
proc rawFailure(): Future[void] {.async: (raw: true).} =
let future = newFuture[void]("rawAsync")
future.fail((ref ValueError)(msg: "Oh no!"))
return future
```
Raw functions can also use checked exceptions:
```nim
proc rawAsyncRaises(): Future[void] {.async: (raw: true, raises: [IOError]).} =
let fut = newFuture[void]()
assert not (compiles do: fut.fail((ref ValueError)(msg: "uh-uh")))
fut.fail((ref IOError)(msg: "IO"))
return fut
```
### Callbacks and closures
Callback/closure types are declared using the `async` annotation as usual:
```nim
type MyCallback = proc(): Future[void] {.async.}
proc runCallback(cb: MyCallback) {.async: (raises: []).} =
try:
await cb()
except CatchableError:
discard # handle errors as usual
```
When calling a callback, it is important to remember that the given function
may raise and exceptions need to be handled.
Checked exceptions can be used to limit the exceptions that a callback can
raise:
```nim
type MyEasyCallback = proc: Future[void] {.async: (raises: []).}
proc runCallback(cb: MyEasyCallback) {.async: (raises: [])} =
await cb()
```
### Platform independence
Several functions in `chronos` are backed by the operating system, such as
waiting for network events, creating files and sockets etc. The specific
exceptions that are raised by the OS is platform-dependent, thus such functions
are declared as raising `CatchableError` but will in general raise something
more specific. In particular, it's possible that some functions that are
annotated as raising `CatchableError` only raise on _some_ platforms - in order
to work on all platforms, calling code must assume that they will raise even
when they don't seem to do so on one platform.
### Cancellation support
Any running `Future` can be cancelled. This can be used for timeouts,
to let a user cancel a running task, to start multiple futures in parallel
and cancel them as soon as one finishes, etc.
```nim
import chronos/apps/http/httpclient
proc cancellationExample() {.async.} =
# Simple cancellation
let future = sleepAsync(10.minutes)
future.cancelSoon()
# `cancelSoon` will not wait for the cancellation
# to be finished, so the Future could still be
# pending at this point.
# Wait for cancellation
let future2 = sleepAsync(10.minutes)
await future2.cancelAndWait()
# Using `cancelAndWait`, we know that future2 isn't
# pending anymore. However, it could have completed
# before cancellation happened (in which case, it
# will hold a value)
# Race between futures
proc retrievePage(uri: string): Future[string] {.async.} =
let httpSession = HttpSessionRef.new()
try:
let resp = await httpSession.fetch(parseUri(uri))
return bytesToString(resp.data)
finally:
# be sure to always close the session
# `finally` will run also during cancellation -
# `noCancel` ensures that `closeWait` doesn't get cancelled
await noCancel(httpSession.closeWait())
let
futs =
@[
retrievePage("https://duckduckgo.com/?q=chronos"),
retrievePage("https://www.google.fr/search?q=chronos")
]
let finishedFut = await one(futs)
for fut in futs:
if not fut.finished:
fut.cancelSoon()
echo "Result: ", await finishedFut
waitFor(cancellationExample())
```
Even if cancellation is initiated, it is not guaranteed that
the operation gets cancelled - the future might still be completed
or fail depending on the ordering of events and the specifics of
the operation.
If the future indeed gets cancelled, `await` will raise a
`CancelledError` as is likely to happen in the following example:
```nim
proc c1 {.async.} =
echo "Before sleep"
try:
await sleepAsync(10.minutes)
echo "After sleep" # not reach due to cancellation
except CancelledError as exc:
echo "We got cancelled!"
raise exc
proc c2 {.async.} =
await c1()
echo "Never reached, since the CancelledError got re-raised"
let work = c2()
waitFor(work.cancelAndWait())
```
The `CancelledError` will now travel up the stack like any other exception.
It can be caught and handled (for instance, freeing some resources)
### Multiple async backend support
Thanks to its powerful macro support, Nim allows `async`/`await` to be
implemented in libraries with only minimal support from the language - as such,
multiple `async` libraries exist, including `chronos` and `asyncdispatch`, and
more may come to be developed in the futures.
Libraries built on top of `async`/`await` may wish to support multiple async
backends - the best way to do so is to create separate modules for each backend
that may be imported side-by-side - see [nim-metrics](https://github.com/status-im/nim-metrics/blob/master/metrics/)
for an example.
An alternative way is to select backend using a global compile flag - this
method makes it diffucult to compose applications that use both backends as may
happen with transitive dependencies, but may be appropriate in some cases -
libraries choosing this path should call the flag `asyncBackend`, allowing
applications to choose the backend with `-d:asyncBackend=<backend_name>`.
Known `async` backends include:
* `chronos` - this library (`-d:asyncBackend=chronos`)
* `asyncdispatch` the standard library `asyncdispatch` [module](https://nim-lang.org/docs/asyncdispatch.html) (`-d:asyncBackend=asyncdispatch`)
* `none` - ``-d:asyncBackend=none`` - disable ``async`` support completely
``none`` can be used when a library supports both a synchronous and
asynchronous API, to disable the latter.
### Compile-time configuration
`chronos` contains several compile-time [configuration options](./chronos/config.nim) enabling stricter compile-time checks and debugging helpers whose runtime cost may be significant.
Strictness options generally will become default in future chronos releases and allow adapting existing code without changing the new version - see the [`config.nim`](./chronos/config.nim) module for more information.
## TODO ## TODO
* Pipe/Subprocess Transports.
* Multithreading Stream/Datagram servers * Multithreading Stream/Datagram servers
## Contributing ## Contributing
@ -470,10 +75,6 @@ When submitting pull requests, please add test cases for any new features or fix
`chronos` follows the [Status Nim Style Guide](https://status-im.github.io/nim-style-guide/). `chronos` follows the [Status Nim Style Guide](https://status-im.github.io/nim-style-guide/).
## Other resources
* [Historical differences with asyncdispatch](https://github.com/status-im/nim-chronos/wiki/AsyncDispatch-comparison)
## License ## License
Licensed and distributed under either of Licensed and distributed under either of

5
chronos.nim
View File

@ -5,5 +5,10 @@
# Licensed under either of # Licensed under either of
# Apache License, version 2.0, (LICENSE-APACHEv2) # Apache License, version 2.0, (LICENSE-APACHEv2)
# MIT license (LICENSE-MIT) # MIT license (LICENSE-MIT)
## `async`/`await` framework for [Nim](https://nim-lang.org)
##
## See https://status-im.github.io/nim-chronos/ for documentation
import chronos/[asyncloop, asyncsync, handles, transport, timer, debugutils] import chronos/[asyncloop, asyncsync, handles, transport, timer, debugutils]
export asyncloop, asyncsync, handles, transport, timer, debugutils export asyncloop, asyncsync, handles, transport, timer, debugutils

13
chronos.nimble
View File

@ -14,7 +14,7 @@ requires "nim >= 1.6.0",
"httputils", "httputils",
"unittest2" "unittest2"
import os import os, strutils
let nimc = getEnv("NIMC", "nim") # Which nim compiler to use let nimc = getEnv("NIMC", "nim") # Which nim compiler to use
let lang = getEnv("NIMLANG", "c") # Which backend (c/cpp/js) let lang = getEnv("NIMLANG", "c") # Which backend (c/cpp/js)
@ -46,12 +46,19 @@ proc run(args, path: string) =
build args, path build args, path
exec "build/" & path.splitPath[1] exec "build/" & path.splitPath[1]
task examples, "Build examples":
# Build book examples
for file in listFiles("docs/examples"):
if file.endsWith(".nim"):
build "", file
task test, "Run all tests": task test, "Run all tests":
for args in testArguments: for args in testArguments:
run args, "tests/testall" run args, "tests/testall"
if (NimMajor, NimMinor) > (1, 6): if (NimMajor, NimMinor) > (1, 6):
run args & " --mm:refc", "tests/testall" run args & " --mm:refc", "tests/testall"
task test_libbacktrace, "test with libbacktrace": task test_libbacktrace, "test with libbacktrace":
var allArgs = @[ var allArgs = @[
"-d:release --debugger:native -d:chronosStackTrace -d:nimStackTraceOverride --import:libbacktrace", "-d:release --debugger:native -d:chronosStackTrace -d:nimStackTraceOverride --import:libbacktrace",
@ -59,3 +66,7 @@ task test_libbacktrace, "test with libbacktrace":
for args in allArgs: for args in allArgs:
run args, "tests/testall" run args, "tests/testall"
task docs, "Generate API documentation":
exec "mdbook build docs"
exec nimc & " doc " & "--git.url:https://github.com/status-im/nim-chronos --git.commit:master --outdir:docs/book/api --project chronos"

118
chronos/asyncloop.nim
View File

@ -10,124 +10,6 @@
{.push raises: [].} {.push raises: [].}
## Chronos
## *************
##
## This module implements asynchronous IO. This includes a dispatcher,
## a ``Future`` type implementation, and an ``async`` macro which allows
## asynchronous code to be written in a synchronous style with the ``await``
## keyword.
##
## The dispatcher acts as a kind of event loop. You must call ``poll`` on it
## (or a function which does so for you such as ``waitFor`` or ``runForever``)
## in order to poll for any outstanding events. The underlying implementation
## is based on epoll on Linux, IO Completion Ports on Windows and select on
## other operating systems.
##
## The ``poll`` function will not, on its own, return any events. Instead
## an appropriate ``Future`` object will be completed. A ``Future`` is a
## type which holds a value which is not yet available, but which *may* be
## available in the future. You can check whether a future is finished
## by using the ``finished`` function. When a future is finished it means that
## either the value that it holds is now available or it holds an error instead.
## The latter situation occurs when the operation to complete a future fails
## with an exception. You can distinguish between the two situations with the
## ``failed`` function.
##
## Future objects can also store a callback procedure which will be called
## automatically once the future completes.
##
## Futures therefore can be thought of as an implementation of the proactor
## pattern. In this
## pattern you make a request for an action, and once that action is fulfilled
## a future is completed with the result of that action. Requests can be
## made by calling the appropriate functions. For example: calling the ``recv``
## function will create a request for some data to be read from a socket. The
## future which the ``recv`` function returns will then complete once the
## requested amount of data is read **or** an exception occurs.
##
## Code to read some data from a socket may look something like this:
##
## .. code-block::nim
## var future = socket.recv(100)
## future.addCallback(
## proc () =
## echo(future.read)
## )
##
## All asynchronous functions returning a ``Future`` will not block. They
## will not however return immediately. An asynchronous function will have
## code which will be executed before an asynchronous request is made, in most
## cases this code sets up the request.
##
## In the above example, the ``recv`` function will return a brand new
## ``Future`` instance once the request for data to be read from the socket
## is made. This ``Future`` instance will complete once the requested amount
## of data is read, in this case it is 100 bytes. The second line sets a
## callback on this future which will be called once the future completes.
## All the callback does is write the data stored in the future to ``stdout``.
## The ``read`` function is used for this and it checks whether the future
## completes with an error for you (if it did it will simply raise the
## error), if there is no error however it returns the value of the future.
##
## Asynchronous procedures
## -----------------------
##
## Asynchronous procedures remove the pain of working with callbacks. They do
## this by allowing you to write asynchronous code the same way as you would
## write synchronous code.
##
## An asynchronous procedure is marked using the ``{.async.}`` pragma.
## When marking a procedure with the ``{.async.}`` pragma it must have a
## ``Future[T]`` return type or no return type at all. If you do not specify
## a return type then ``Future[void]`` is assumed.
##
## Inside asynchronous procedures ``await`` can be used to call any
## procedures which return a
## ``Future``; this includes asynchronous procedures. When a procedure is
## "awaited", the asynchronous procedure it is awaited in will
## suspend its execution
## until the awaited procedure's Future completes. At which point the
## asynchronous procedure will resume its execution. During the period
## when an asynchronous procedure is suspended other asynchronous procedures
## will be run by the dispatcher.
##
## The ``await`` call may be used in many contexts. It can be used on the right
## hand side of a variable declaration: ``var data = await socket.recv(100)``,
## in which case the variable will be set to the value of the future
## automatically. It can be used to await a ``Future`` object, and it can
## be used to await a procedure returning a ``Future[void]``:
## ``await socket.send("foobar")``.
##
## If an awaited future completes with an error, then ``await`` will re-raise
## this error.
##
## Handling Exceptions
## -------------------
##
## The ``async`` procedures also offer support for the try statement.
##
## .. code-block:: Nim
## try:
## let data = await sock.recv(100)
## echo("Received ", data)
## except CancelledError as exc:
## # Handle exc
##
## Discarding futures
## ------------------
##
## Futures should **never** be discarded. This is because they may contain
## errors. If you do not care for the result of a Future then you should
## use the ``asyncSpawn`` procedure instead of the ``discard`` keyword.
## ``asyncSpawn`` will transform any exception thrown by the called procedure
## to a Defect
##
## Limitations/Bugs
## ----------------
##
## * The effect system (``raises: []``) does not work with async procedures.
import ./internal/[asyncengine, asyncfutures, asyncmacro, errors] import ./internal/[asyncengine, asyncfutures, asyncmacro, errors]
export asyncfutures, asyncengine, errors export asyncfutures, asyncengine, errors

4
chronos/internal/asyncengine.nim
View File

@ -10,6 +10,10 @@
{.push raises: [].} {.push raises: [].}
## This module implements the core asynchronous engine / dispatcher.
##
## For more information, see the `Concepts` chapter of the guide.
from nativesockets import Port from nativesockets import Port
import std/[tables, heapqueue, deques] import std/[tables, heapqueue, deques]
import results import results

1
docs/.gitignore vendored Normal file
View File

@ -0,0 +1 @@
book

20
docs/book.toml Normal file
View File

@ -0,0 +1,20 @@
[book]
authors = ["Jacek Sieka"]
language = "en"
multilingual = false
src = "src"
title = "Chronos"
[preprocessor.toc]
command = "mdbook-toc"
renderer = ["html"]
max-level = 2
[preprocessor.open-on-gh]
command = "mdbook-open-on-gh"
renderer = ["html"]
[output.html]
git-repository-url = "https://github.com/status-im/nim-chronos/"
git-branch = "master"
additional-css = ["open-in.css"]

21
docs/examples/cancellation.nim Normal file
View File

@ -0,0 +1,21 @@
## Simple cancellation example
import chronos
proc someTask() {.async.} = await sleepAsync(10.minutes)
proc cancellationExample() {.async.} =
# Start a task but don't wait for it to finish
let future = someTask()
future.cancelSoon()
# `cancelSoon` schedules but does not wait for the future to get cancelled -
# it might still be pending here
let future2 = someTask() # Start another task concurrently
await future2.cancelAndWait()
# Using `cancelAndWait`, we can be sure that `future2` is either
# complete, failed or cancelled at this point. `future` could still be
# pending!
assert future2.finished()
waitFor(cancellationExample())

28
docs/examples/discards.nim Normal file
View File

@ -0,0 +1,28 @@
## The peculiarities of `discard` in `async` procedures
import chronos
proc failingOperation() {.async.} =
echo "Raising!"
raise (ref ValueError)(msg: "My error")
proc myApp() {.async.} =
# This style of discard causes the `ValueError` to be discarded, hiding the
# failure of the operation - avoid!
discard failingOperation()
proc runAsTask(fut: Future[void]): Future[void] {.async: (raises: []).} =
# runAsTask uses `raises: []` to ensure at compile-time that no exceptions
# escape it!
try:
await fut
except CatchableError as exc:
echo "The task failed! ", exc.msg
# asyncSpawn ensures that errors don't leak unnoticed from tasks without
# blocking:
asyncSpawn runAsTask(failingOperation())
# If we didn't catch the exception with `runAsTask`, the program will crash:
asyncSpawn failingOperation()
waitFor myApp()

15
docs/examples/httpget.nim Normal file
View File

@ -0,0 +1,15 @@
import chronos/apps/http/httpclient
proc retrievePage*(uri: string): Future[string] {.async.} =
# Create a new HTTP session
let httpSession = HttpSessionRef.new()
try:
# Fetch page contents
let resp = await httpSession.fetch(parseUri(uri))
# Convert response to a string, assuming its encoding matches the terminal!
bytesToString(resp.data)
finally: # Close the session
await noCancel(httpSession.closeWait())
echo waitFor retrievePage(
"https://raw.githubusercontent.com/status-im/nim-chronos/master/README.md")

1
docs/examples/nim.cfg Normal file
View File

@ -0,0 +1 @@
path = "../.."

25
docs/examples/timeoutcomposed.nim Normal file
View File

@ -0,0 +1,25 @@
## Single timeout for several operations
import chronos
proc shortTask {.async.} =
try:
await sleepAsync(1.seconds)
except CancelledError as exc:
echo "Short task was cancelled!"
raise exc # Propagate cancellation to the next operation
proc composedTimeout() {.async.} =
let
# Common timout for several sub-tasks
timeout = sleepAsync(10.seconds)
while not timeout.finished():
let task = shortTask() # Start a task but don't `await` it
if (await race(task, timeout)) == task:
echo "Ran one more task"
else:
# This cancellation may or may not happen as task might have finished
# right at the timeout!
task.cancelSoon()
waitFor composedTimeout()

20
docs/examples/timeoutsimple.nim Normal file
View File

@ -0,0 +1,20 @@
## Simple timeouts
import chronos
proc longTask {.async.} =
try:
await sleepAsync(10.minutes)
except CancelledError as exc:
echo "Long task was cancelled!"
raise exc # Propagate cancellation to the next operation
proc simpleTimeout() {.async.} =
let
task = longTask() # Start a task but don't `await` it
if not await task.withTimeout(1.seconds):
echo "Timeout reached - withTimeout should have cancelled the task"
else:
echo "Task completed"
waitFor simpleTimeout()

24
docs/examples/twogets.nim Normal file
View File

@ -0,0 +1,24 @@
## Make two http requests concurrently and output the one that wins
import chronos
import ./httpget
proc twoGets() {.async.} =
let
futs = @[
# Both pages will start downloading concurrently...
httpget.retrievePage("https://duckduckgo.com/?q=chronos"),
httpget.retrievePage("https://www.google.fr/search?q=chronos")
]
# Wait for at least one request to finish..
let winner = await one(futs)
# ..and cancel the others since we won't need them
for fut in futs:
# Trying to cancel an already-finished future is harmless
fut.cancelSoon()
# An exception could be raised here if the winning request failed!
echo "Result: ", winner.read()
waitFor(twoGets())

7
docs/open-in.css Normal file
View File

@ -0,0 +1,7 @@
footer {
font-size: 0.8em;
text-align: center;
border-top: 1px solid black;
padding: 5px 0;
}

10
docs/src/SUMMARY.md Normal file
View File

@ -0,0 +1,10 @@
- [Introduction](./introduction.md)
- [Getting started](./getting_started.md)
# User guide
- [Core concepts](./concepts.md)
- [`async` functions](async_procs.md)
- [Errors and exceptions](./error_handling.md)
- [Tips, tricks and best practices](./tips.md)
- [Porting code to `chronos`](./porting.md)

112
docs/src/async_procs.md Normal file
View File

@ -0,0 +1,112 @@
# Async procedures
<!-- toc -->
## The `async` pragma
The `{.async.}` pragma will transform a procedure (or a method) returning a
`Future` into a closure iterator. If there is no return type specified,
`Future[void]` is returned.
```nim
proc p() {.async.} =
await sleepAsync(100.milliseconds)
echo p().type # prints "Future[system.void]"
```
## `await` keyword
Whenever `await` is encountered inside an async procedure, control is given
back to the dispatcher for as many steps as it's necessary for the awaited
future to complete, fail or be cancelled. `await` calls the
equivalent of `Future.read()` on the completed future and returns the
encapsulated value.
```nim
proc p1() {.async.} =
await sleepAsync(1.seconds)
proc p2() {.async.} =
await sleepAsync(1.seconds)
proc p3() {.async.} =
let
fut1 = p1()
fut2 = p2()
# Just by executing the async procs, both resulting futures entered the
# dispatcher queue and their "clocks" started ticking.
await fut1
await fut2
# Only one second passed while awaiting them both, not two.
waitFor p3()
```
```admonition warning
Because `async` procedures are executed concurrently, they are subject to many
of the same risks that typically accompany multithreaded programming
In particular, if two `async` procedures have access to the same mutable state,
the value before and after `await` might not be the same as the order of execution is not guaranteed!
```
## Raw functions
Raw functions are those that interact with `chronos` via the `Future` type but
whose body does not go through the async transformation.
Such functions are created by adding `raw: true` to the `async` parameters:
```nim
proc rawAsync(): Future[void] {.async: (raw: true).} =
let fut = newFuture[void]("rawAsync")
fut.complete()
fut
```
Raw functions must not raise exceptions directly - they are implicitly declared
as `raises: []` - instead they should store exceptions in the returned `Future`:
```nim
proc rawFailure(): Future[void] {.async: (raw: true).} =
let fut = newFuture[void]("rawAsync")
fut.fail((ref ValueError)(msg: "Oh no!"))
fut
```
Raw functions can also use checked exceptions:
```nim
proc rawAsyncRaises(): Future[void] {.async: (raw: true, raises: [IOError]).} =
let fut = newFuture[void]()
assert not (compiles do: fut.fail((ref ValueError)(msg: "uh-uh")))
fut.fail((ref IOError)(msg: "IO"))
fut
```
## Callbacks and closures
Callback/closure types are declared using the `async` annotation as usual:
```nim
type MyCallback = proc(): Future[void] {.async.}
proc runCallback(cb: MyCallback) {.async: (raises: []).} =
try:
await cb()
except CatchableError:
discard # handle errors as usual
```
When calling a callback, it is important to remember that it may raise exceptions that need to be handled.
Checked exceptions can be used to limit the exceptions that a callback can
raise:
```nim
type MyEasyCallback = proc(): Future[void] {.async: (raises: []).}
proc runCallback(cb: MyEasyCallback) {.async: (raises: [])} =
await cb()
```

126
docs/src/concepts.md Normal file
View File

@ -0,0 +1,126 @@
# Concepts
<!-- toc -->
## The dispatcher
Async/await programming relies on cooperative multitasking to coordinate the
concurrent execution of procedures, using event notifications from the operating system to resume execution.
The event handler loop is called a "dispatcher" and a single instance per
thread is created, as soon as one is needed.
Scheduling is done by calling [async procedures](./async_procs.md) that return
`Future` objects - each time a procedure is unable to make further
progress, for example because it's waiting for some data to arrive, it hands
control back to the dispatcher which ensures that the procedure is resumed when
ready.
## The `Future` type
`Future` objects encapsulate the outcome of executing an `async` procedure. The
`Future` may be `pending` meaning that the outcome is not yet known or
`finished` meaning that the return value is available, the operation failed
with an exception or was cancelled.
Inside an async procedure, you can `await` the outcome of another async
procedure - if the `Future` representing that operation is still `pending`, a
callback representing where to resume execution will be added to it and the
dispatcher will be given back control to deal with other tasks.
When a `Future` is `finished`, all its callbacks are scheduled to be run by
the dispatcher, thus continuing any operations that were waiting for an outcome.
## The `poll` call
To trigger the processing step of the dispatcher, we need to call `poll()` -
either directly or through a wrapper like `runForever()` or `waitFor()`.
Each call to poll handles any file descriptors, timers and callbacks that are
ready to be processed.
Using `waitFor`, the result of a single asynchronous operation can be obtained:
```nim
proc myApp() {.async.} =
echo "Waiting for a second..."
await sleepAsync(1.seconds)
echo "done!"
waitFor myApp()
```
It is also possible to keep running the event loop forever using `runForever`:
```nim
proc myApp() {.async.} =
while true:
await sleepAsync(1.seconds)
echo "A bit more than a second passed!"
let future = myApp()
runForever()
```
Such an application never terminates, thus it is rare that applications are
structured this way.
```admonish warning
Both `waitFor` and `runForever` call `poll` which offers fine-grained
control over the event loop steps.
Nested calls to `poll`, `waitFor` and `runForever` are not allowed.
```
## Cancellation
Any pending `Future` can be cancelled. This can be used for timeouts, to start
multiple operations in parallel and cancel the rest as soon as one finishes,
to initiate the orderely shutdown of an application etc.
```nim
{{#include ../examples/cancellation.nim}}
```
Even if cancellation is initiated, it is not guaranteed that the operation gets
cancelled - the future might still be completed or fail depending on the
order of events in the dispatcher and the specifics of the operation.
If the future indeed gets cancelled, `await` will raise a
`CancelledError` as is likely to happen in the following example:
```nim
proc c1 {.async.} =
echo "Before sleep"
try:
await sleepAsync(10.minutes)
echo "After sleep" # not reach due to cancellation
except CancelledError as exc:
echo "We got cancelled!"
# `CancelledError` is typically re-raised to notify the caller that the
# operation is being cancelled
raise exc
proc c2 {.async.} =
await c1()
echo "Never reached, since the CancelledError got re-raised"
let work = c2()
waitFor(work.cancelAndWait())
```
The `CancelledError` will now travel up the stack like any other exception.
It can be caught and handled (for instance, freeing some resources)
Cancelling an already-finished `Future` has no effect, as the following example
of downloading two web pages concurrently shows:
```nim
{{#include ../examples/twogets.nim}}
```
## Compile-time configuration
`chronos` contains several compile-time [configuration options](./chronos/config.nim) enabling stricter compile-time checks and debugging helpers whose runtime cost may be significant.
Strictness options generally will become default in future chronos releases and allow adapting existing code without changing the new version - see the [`config.nim`](./chronos/config.nim) module for more information.

134
docs/src/error_handling.md Normal file
View File

@ -0,0 +1,134 @@
# Errors and exceptions
<!-- toc -->
## Exceptions
Exceptions inheriting from [`CatchableError`](https://nim-lang.org/docs/system.html#CatchableError)
interrupt execution of an `async` procedure. The exception is placed in the
`Future.error` field while changing the status of the `Future` to `Failed`
and callbacks are scheduled.
When a future is read or awaited the exception is re-raised, traversing the
`async` execution chain until handled.
```nim
proc p1() {.async.} =
await sleepAsync(1.seconds)
raise newException(ValueError, "ValueError inherits from CatchableError")
proc p2() {.async.} =
await sleepAsync(1.seconds)
proc p3() {.async.} =
let
fut1 = p1()
fut2 = p2()
await fut1
echo "unreachable code here"
await fut2
# `waitFor()` would call `Future.read()` unconditionally, which would raise the
# exception in `Future.error`.
let fut3 = p3()
while not(fut3.finished()):
poll()
echo "fut3.state = ", fut3.state # "Failed"
if fut3.failed():
echo "p3() failed: ", fut3.error.name, ": ", fut3.error.msg
# prints "p3() failed: ValueError: ValueError inherits from CatchableError"
```
You can put the `await` in a `try` block, to deal with that exception sooner:
```nim
proc p3() {.async.} =
let
fut1 = p1()
fut2 = p2()
try:
await fut1
except CachableError:
echo "p1() failed: ", fut1.error.name, ": ", fut1.error.msg
echo "reachable code here"
await fut2
```
Because `chronos` ensures that all exceptions are re-routed to the `Future`,
`poll` will not itself raise exceptions.
`poll` may still panic / raise `Defect` if such are raised in user code due to
undefined behavior.
## Checked exceptions
By specifying a `raises` list to an async procedure, you can check which
exceptions can be raised by it:
```nim
proc p1(): Future[void] {.async: (raises: [IOError]).} =
assert not (compiles do: raise newException(ValueError, "uh-uh"))
raise newException(IOError, "works") # Or any child of IOError
proc p2(): Future[void] {.async, (raises: [IOError]).} =
await p1() # Works, because await knows that p1
# can only raise IOError
```
Under the hood, the return type of `p1` will be rewritten to an internal type
which will convey raises informations to `await`.
```admonition note
Most `async` include `CancelledError` in the list of `raises`, indicating that
the operation they implement might get cancelled resulting in neither value nor
error!
```
## The `Exception` type
Exceptions deriving from `Exception` are not caught by default as these may
include `Defect` and other forms undefined or uncatchable behavior.
Because exception effect tracking is turned on for `async` functions, this may
sometimes lead to compile errors around forward declarations, methods and
closures as Nim conservatively asssumes that any `Exception` might be raised
from those.
Make sure to excplicitly annotate these with `{.raises.}`:
```nim
# Forward declarations need to explicitly include a raises list:
proc myfunction() {.raises: [ValueError].}
# ... as do `proc` types
type MyClosure = proc() {.raises: [ValueError].}
proc myfunction() =
raise (ref ValueError)(msg: "Implementation here")
let closure: MyClosure = myfunction
```
For compatibility, `async` functions can be instructed to handle `Exception` as
well, specifying `handleException: true`. `Exception` that is not a `Defect` and
not a `CatchableError` will then be caught and remapped to
`AsyncExceptionError`:
```nim
proc raiseException() {.async: (handleException: true, raises: [AsyncExceptionError]).} =
raise (ref Exception)(msg: "Raising Exception is UB")
proc callRaiseException() {.async: (raises: []).} =
try:
raiseException()
except AsyncExceptionError as exc:
# The original Exception is available from the `parent` field
echo exc.parent.msg
```
This mode can be enabled globally with `-d:chronosHandleException` as a help
when porting code to `chronos` but should generally be avoided as global
configuration settings may interfere with libraries that use `chronos` leading
to unexpected behavior.

19
docs/src/getting_started.md Normal file
View File

@ -0,0 +1,19 @@
## Getting started
Install `chronos` using `nimble`:
```text
nimble install chronos
```
or add a dependency to your `.nimble` file:
```text
requires "chronos"
```
and start using it:
```nim
{{#include ../examples/httpget.nim}}
```

32
docs/src/introduction.md Normal file
View File

@ -0,0 +1,32 @@
# Introduction
Chronos implements the [async/await](https://en.wikipedia.org/wiki/Async/await)
paradigm in a self-contained library using macro and closure iterator
transformation features provided by Nim.
Features include:
* Asynchronous socket and process I/O
* HTTP server with SSL/TLS support out of the box (no OpenSSL needed)
* Synchronization primitivies like queues, events and locks
* Cancellation
* Efficient dispatch pipeline with excellent multi-platform support
* Exception [effect support](./guide.md#error-handling)
## Platform support
Several platforms are supported, with different backend [options](./concepts.md#compile-time-configuration):
* Windows: [`IOCP`](https://learn.microsoft.com/en-us/windows/win32/fileio/i-o-completion-ports)
* Linux: [`epoll`](https://en.wikipedia.org/wiki/Epoll) / `poll`
* OSX / BSD: [`kqueue`](https://en.wikipedia.org/wiki/Kqueue) / `poll`
* Android / Emscripten / posix: `poll`
## Examples
Examples are available in the [`docs/examples/`](https://github.com/status-im/nim-chronos/docs/examples) folder.
## API documentation
This guide covers basic usage of chronos - for details, see the
[API reference](./api/chronos.html).

54
docs/src/porting.md Normal file
View File

@ -0,0 +1,54 @@
# Porting code to `chronos` v4
<!-- toc -->
Thanks to its macro support, Nim allows `async`/`await` to be implemented in
libraries with only minimal support from the language - as such, multiple
`async` libraries exist, including `chronos` and `asyncdispatch`, and more may
come to be developed in the futures.
## Chronos v3
Chronos v4 introduces new features for IPv6, exception effects, a stand-alone
`Future` type as well as several other changes - when upgrading from chronos v3,
here are several things to consider:
* Exception handling is now strict by default - see the [error handling](./error_handling.md)
chapter for how to deal with `raises` effects
* `AsyncEventBus` was removed - use `AsyncEventQueue` instead
## `asyncdispatch`
Projects written for `asyncdispatch` and `chronos` look similar but there are
several differences to be aware of:
* `chronos` has its own dispatch loop - you can typically not mix `chronos` and
`asyncdispatch` in the same thread
* `import chronos` instead of `import asyncdispatch`
* cleanup is important - make sure to use `closeWait` to release any resources
you're using or file descript leaks and other
* cancellation support means that `CancelledError` may be raised from most
`{.async.}` functions
* Calling `yield` directly in tasks is not supported - instead, use `awaitne`.
## Supporting multiple backends
Libraries built on top of `async`/`await` may wish to support multiple async
backends - the best way to do so is to create separate modules for each backend
that may be imported side-by-side - see [nim-metrics](https://github.com/status-im/nim-metrics/blob/master/metrics/)
for an example.
An alternative way is to select backend using a global compile flag - this
method makes it diffucult to compose applications that use both backends as may
happen with transitive dependencies, but may be appropriate in some cases -
libraries choosing this path should call the flag `asyncBackend`, allowing
applications to choose the backend with `-d:asyncBackend=<backend_name>`.
Known `async` backends include:
* `chronos` - this library (`-d:asyncBackend=chronos`)
* `asyncdispatch` the standard library `asyncdispatch` [module](https://nim-lang.org/docs/asyncdispatch.html) (`-d:asyncBackend=asyncdispatch`)
* `none` - ``-d:asyncBackend=none`` - disable ``async`` support completely
``none`` can be used when a library supports both a synchronous and
asynchronous API, to disable the latter.

34
docs/src/tips.md Normal file
View File

@ -0,0 +1,34 @@
# Tips, tricks and best practices
## Timeouts
To prevent a single task from taking too long, `withTimeout` can be used:
```nim
{{#include ../examples/timeoutsimple.nim}}
```
When several tasks should share a single timeout, a common timer can be created
with `sleepAsync`:
```nim
{{#include ../examples/timeoutcomposed.nim}}
```
## `discard`
When calling an asynchronous procedure without `await`, the operation is started
but its result is not processed until corresponding `Future` is `read`.
It is therefore important to never `discard` futures directly - instead, one
can discard the result of awaiting the future or use `asyncSpawn` to monitor
the outcome of the future as if it were running in a separate thread.
Similar to threads, tasks managed by `asyncSpawn` may causes the application to
crash if any exceptions leak out of it - use
[checked exceptions](./error_handling.md#checked-exceptions) to avoid this
problem.
```nim
{{#include ../examples/discards.nim}}
```

53
docs/theme/highlight.js vendored Normal file
View File

File diff suppressed because one or more lines are too long