nim-chronos/chronos/asyncloop.nim

1293 lines
46 KiB
Nim
Raw Normal View History

2018-05-16 08:22:34 +00:00
#
# Chronos
2018-05-16 08:22:34 +00:00
#
# (c) Copyright 2015 Dominik Picheta
# (c) Copyright 2018-Present Status Research & Development GmbH
2018-05-16 08:22:34 +00:00
#
# Licensed under either of
# Apache License, version 2.0, (LICENSE-APACHEv2)
# MIT license (LICENSE-MIT)
when (NimMajor, NimMinor) < (1, 4):
{.push raises: [Defect].}
else:
{.push raises: [].}
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
from nativesockets import Port
import std/[tables, strutils, heapqueue, options, deques]
import stew/results
import "."/[config, osdefs, osutils, timer]
2018-05-16 08:22:34 +00:00
export Port
export timer, results
2018-05-16 08:22:34 +00:00
#{.injectStmt: newGcInvariant().}
## Chronos
2018-05-16 08:22:34 +00:00
## *************
##
## This module implements asynchronous IO. This includes a dispatcher,
## a ``Future`` type implementation, and an ``async`` macro which allows
## asynchronous code to be written in a synchronous style with the ``await``
## keyword.
##
## The dispatcher acts as a kind of event loop. You must call ``poll`` on it
## (or a function which does so for you such as ``waitFor`` or ``runForever``)
## in order to poll for any outstanding events. The underlying implementation
## is based on epoll on Linux, IO Completion Ports on Windows and select on
## other operating systems.
##
## The ``poll`` function will not, on its own, return any events. Instead
## an appropriate ``Future`` object will be completed. A ``Future`` is a
## type which holds a value which is not yet available, but which *may* be
## available in the future. You can check whether a future is finished
## by using the ``finished`` function. When a future is finished it means that
## either the value that it holds is now available or it holds an error instead.
## The latter situation occurs when the operation to complete a future fails
## with an exception. You can distinguish between the two situations with the
## ``failed`` function.
##
## Future objects can also store a callback procedure which will be called
## automatically once the future completes.
##
## Futures therefore can be thought of as an implementation of the proactor
## pattern. In this
## pattern you make a request for an action, and once that action is fulfilled
## a future is completed with the result of that action. Requests can be
## made by calling the appropriate functions. For example: calling the ``recv``
## function will create a request for some data to be read from a socket. The
## future which the ``recv`` function returns will then complete once the
## requested amount of data is read **or** an exception occurs.
##
## Code to read some data from a socket may look something like this:
##
## .. code-block::nim
## var future = socket.recv(100)
## future.addCallback(
## proc () =
## echo(future.read)
## )
##
## All asynchronous functions returning a ``Future`` will not block. They
## will not however return immediately. An asynchronous function will have
## code which will be executed before an asynchronous request is made, in most
## cases this code sets up the request.
##
## In the above example, the ``recv`` function will return a brand new
## ``Future`` instance once the request for data to be read from the socket
## is made. This ``Future`` instance will complete once the requested amount
## of data is read, in this case it is 100 bytes. The second line sets a
## callback on this future which will be called once the future completes.
## All the callback does is write the data stored in the future to ``stdout``.
## The ``read`` function is used for this and it checks whether the future
## completes with an error for you (if it did it will simply raise the
## error), if there is no error however it returns the value of the future.
##
## Asynchronous procedures
## -----------------------
##
## Asynchronous procedures remove the pain of working with callbacks. They do
## this by allowing you to write asynchronous code the same way as you would
## write synchronous code.
##
## An asynchronous procedure is marked using the ``{.async.}`` pragma.
## When marking a procedure with the ``{.async.}`` pragma it must have a
## ``Future[T]`` return type or no return type at all. If you do not specify
## a return type then ``Future[void]`` is assumed.
##
## Inside asynchronous procedures ``await`` can be used to call any
## procedures which return a
## ``Future``; this includes asynchronous procedures. When a procedure is
## "awaited", the asynchronous procedure it is awaited in will
## suspend its execution
## until the awaited procedure's Future completes. At which point the
## asynchronous procedure will resume its execution. During the period
## when an asynchronous procedure is suspended other asynchronous procedures
## will be run by the dispatcher.
##
## The ``await`` call may be used in many contexts. It can be used on the right
## hand side of a variable declaration: ``var data = await socket.recv(100)``,
## in which case the variable will be set to the value of the future
## automatically. It can be used to await a ``Future`` object, and it can
## be used to await a procedure returning a ``Future[void]``:
## ``await socket.send("foobar")``.
##
## If an awaited future completes with an error, then ``await`` will re-raise
## this error.
2018-05-16 08:22:34 +00:00
##
## Handling Exceptions
## -------------------
2018-05-16 08:22:34 +00:00
##
## The ``async`` procedures also offer support for the try statement.
2018-05-16 08:22:34 +00:00
##
## .. code-block:: Nim
## try:
## let data = await sock.recv(100)
## echo("Received ", data)
## except CancelledError as exc:
## # Handle exc
2018-05-16 08:22:34 +00:00
##
## Discarding futures
## ------------------
##
## Futures should **never** be discarded. This is because they may contain
## errors. If you do not care for the result of a Future then you should
## use the ``asyncSpawn`` procedure instead of the ``discard`` keyword.
## ``asyncSpawn`` will transform any exception thrown by the called procedure
## to a Defect
2018-05-16 08:22:34 +00:00
##
## Limitations/Bugs
## ----------------
##
## * The effect system (``raises: []``) does not work with async procedures.
# TODO: Check if yielded future is nil and throw a more meaningful exception
const
MaxEventsCount* = 64
when defined(windows):
import std/[sets, hashes]
elif defined(macosx) or defined(freebsd) or defined(netbsd) or
defined(openbsd) or defined(dragonfly) or defined(macos) or
defined(linux) or defined(android) or defined(solaris):
import "."/selectors2
import "."/oserrno
from posix import MSG_PEEK, MSG_NOSIGNAL,
SIGHUP, SIGINT, SIGQUIT, SIGILL, SIGTRAP, SIGABRT,
SIGBUS, SIGFPE, SIGKILL, SIGUSR1, SIGSEGV, SIGUSR2,
SIGPIPE, SIGALRM, SIGTERM, SIGPIPE
export SIGHUP, SIGINT, SIGQUIT, SIGILL, SIGTRAP, SIGABRT,
SIGBUS, SIGFPE, SIGKILL, SIGUSR1, SIGSEGV, SIGUSR2,
SIGPIPE, SIGALRM, SIGTERM, SIGPIPE
export oserrno
2018-05-16 08:22:34 +00:00
type
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
CallbackFunc* = proc (arg: pointer) {.gcsafe, raises: [Defect].}
2020-01-08 17:06:56 +00:00
AsyncCallback* = object
function*: CallbackFunc
udata*: pointer
2019-03-26 10:29:45 +00:00
AsyncError* = object of CatchableError
## Generic async exception
AsyncTimeoutError* = object of AsyncError
## Timeout exception
2020-01-07 05:26:18 +00:00
TimerCallback* = ref object
finishAt*: Moment
2018-05-16 08:22:34 +00:00
function*: AsyncCallback
TrackerBase* = ref object of RootRef
id*: string
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
dump*: proc(): string {.gcsafe, raises: [Defect].}
isLeaked*: proc(): bool {.gcsafe, raises: [Defect].}
2018-05-16 08:22:34 +00:00
PDispatcherBase = ref object of RootRef
timers*: HeapQueue[TimerCallback]
callbacks*: Deque[AsyncCallback]
idlers*: Deque[AsyncCallback]
trackers*: Table[string, TrackerBase]
2018-05-16 08:22:34 +00:00
proc sentinelCallbackImpl(arg: pointer) {.gcsafe.} =
raiseAssert "Sentinel callback MUST not be scheduled"
const
SentinelCallback = AsyncCallback(function: sentinelCallbackImpl,
udata: nil)
proc isSentinel(acb: AsyncCallback): bool =
acb == SentinelCallback
2018-05-16 08:22:34 +00:00
proc `<`(a, b: TimerCallback): bool =
result = a.finishAt < b.finishAt
func getAsyncTimestamp*(a: Duration): auto {.inline.} =
## Return rounded up value of duration with milliseconds resolution.
##
## This function also take care on int32 overflow, because Linux and Windows
## accepts signed 32bit integer as timeout.
let milsec = Millisecond.nanoseconds()
let nansec = a.nanoseconds()
var res = nansec div milsec
let mid = nansec mod milsec
when defined(windows):
res = min(int64(high(int32) - 1), res)
result = cast[DWORD](res)
result += DWORD(min(1'i32, cast[int32](mid)))
else:
res = min(int64(high(int32) - 1), res)
result = cast[int32](res)
result += min(1, cast[int32](mid))
2018-06-02 23:03:48 +00:00
template processTimersGetTimeout(loop, timeout: untyped) =
2020-01-08 16:03:34 +00:00
var lastFinish = curTime
while loop.timers.len > 0:
if loop.timers[0].function.function.isNil:
2020-01-08 16:03:34 +00:00
discard loop.timers.pop()
continue
2020-01-07 05:26:18 +00:00
2020-01-08 16:03:34 +00:00
lastFinish = loop.timers[0].finishAt
if curTime < lastFinish:
break
2020-01-07 05:26:18 +00:00
2020-01-08 16:03:34 +00:00
loop.callbacks.addLast(loop.timers.pop().function)
2020-01-08 16:03:34 +00:00
if loop.timers.len > 0:
timeout = (lastFinish - curTime).getAsyncTimestamp()
2018-06-02 23:03:48 +00:00
if timeout == 0:
if (len(loop.callbacks) == 0) and (len(loop.idlers) == 0):
2018-06-02 23:03:48 +00:00
when defined(windows):
timeout = INFINITE
else:
timeout = -1
else:
if (len(loop.callbacks) != 0) or (len(loop.idlers) != 0):
2018-06-02 23:03:48 +00:00
timeout = 0
template processTimers(loop: untyped) =
var curTime = Moment.now()
2020-01-08 16:03:34 +00:00
while loop.timers.len > 0:
if loop.timers[0].function.function.isNil:
2020-01-08 16:03:34 +00:00
discard loop.timers.pop()
continue
2020-01-07 05:26:18 +00:00
2020-01-08 16:03:34 +00:00
if curTime < loop.timers[0].finishAt:
break
loop.callbacks.addLast(loop.timers.pop().function)
2018-06-02 23:03:48 +00:00
template processIdlers(loop: untyped) =
if len(loop.idlers) > 0:
loop.callbacks.addLast(loop.idlers.popFirst())
2018-06-02 23:03:48 +00:00
template processCallbacks(loop: untyped) =
while true:
let callable = loop.callbacks.popFirst() # len must be > 0 due to sentinel
if isSentinel(callable):
break
if not(isNil(callable.function)):
callable.function(callable.udata)
2018-06-02 23:03:48 +00:00
proc raiseAsDefect*(exc: ref Exception, msg: string) {.noreturn, noinline.} =
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
# Reraise an exception as a Defect, where it's unexpected and can't be handled
# We include the stack trace in the message because otherwise, it's easily
# lost - Nim doesn't print it for `parent` exceptions for example (!)
raise (ref Defect)(
msg: msg & "\n" & exc.msg & "\n" & exc.getStackTrace(), parent: exc)
proc raiseOsDefect*(error: OSErrorCode, msg = "") {.noreturn, noinline.} =
# Reraise OS error code as a Defect, where it's unexpected and can't be
# handled. We include the stack trace in the message because otherwise,
# it's easily lost.
raise (ref Defect)(msg: msg & "\n[" & $int(error) & "] " & osErrorMsg(error) &
"\n" & getStackTrace())
func toPointer(error: OSErrorCode): pointer =
when sizeof(int) == 8:
cast[pointer](uint64(uint32(error)))
else:
cast[pointer](uint32(error))
func toException*(v: OSErrorCode): ref OSError = newOSError(v)
# This helper will allow to use `tryGet()` and raise OSError for
# Result[T, OSErrorCode] values.
2022-01-04 17:00:17 +00:00
when defined(windows):
2018-05-16 08:22:34 +00:00
type
CompletionKey = ULONG_PTR
CompletionData* = object
cb*: CallbackFunc
errCode*: OSErrorCode
bytesCount*: uint32
2018-05-16 08:22:34 +00:00
udata*: pointer
CustomOverlapped* = object of OVERLAPPED
data*: CompletionData
OVERLAPPED_ENTRY* = object
lpCompletionKey*: ULONG_PTR
lpOverlapped*: ptr CustomOverlapped
internal: ULONG_PTR
dwNumberOfBytesTransferred: DWORD
2018-05-16 08:22:34 +00:00
PDispatcher* = ref object of PDispatcherBase
ioPort: HANDLE
2018-05-16 08:22:34 +00:00
handles: HashSet[AsyncFD]
connectEx*: WSAPROC_CONNECTEX
acceptEx*: WSAPROC_ACCEPTEX
getAcceptExSockAddrs*: WSAPROC_GETACCEPTEXSOCKADDRS
transmitFile*: WSAPROC_TRANSMITFILE
getQueuedCompletionStatusEx*: LPFN_GETQUEUEDCOMPLETIONSTATUSEX
2018-05-16 08:22:34 +00:00
2018-05-21 21:52:57 +00:00
PtrCustomOverlapped* = ptr CustomOverlapped
2018-05-16 08:22:34 +00:00
RefCustomOverlapped* = ref CustomOverlapped
AsyncFD* = distinct int
proc hash(x: AsyncFD): Hash {.borrow.}
2021-11-20 00:30:23 +00:00
proc `==`*(x: AsyncFD, y: AsyncFD): bool {.borrow, gcsafe.}
2018-05-16 08:22:34 +00:00
proc getFunc(s: SocketHandle, fun: var pointer, guid: GUID): bool =
var bytesRet: DWORD
fun = nil
result = wsaIoctl(s, SIO_GET_EXTENSION_FUNCTION_POINTER, unsafeAddr(guid),
sizeof(GUID).DWORD, addr fun, sizeof(pointer).DWORD,
addr(bytesRet), nil, nil) == 0
proc globalInit() =
var wsa = WSAData()
let res = wsaStartup(0x0202'u16, addr wsa)
if res != 0:
raiseOsDefect(osLastError(),
"globalInit(): Unable to initialize Windows Sockets API")
2019-09-23 17:24:26 +00:00
proc initAPI(loop: PDispatcher) =
var funcPointer: pointer = nil
let kernel32 = getModuleHandle(newWideCString("kernel32.dll"))
loop.getQueuedCompletionStatusEx = cast[LPFN_GETQUEUEDCOMPLETIONSTATUSEX](
getProcAddress(kernel32, "GetQueuedCompletionStatusEx"))
let sock = osdefs.socket(osdefs.AF_INET, 1, 6)
if sock == osdefs.INVALID_SOCKET:
raiseOsDefect(osLastError(), "initAPI(): Unable to create control socket")
block:
let res = getFunc(sock, funcPointer, WSAID_CONNECTEX)
if not(res):
raiseOsDefect(osLastError(), "initAPI(): Unable to initialize " &
"dispatcher's ConnectEx()")
loop.connectEx = cast[WSAPROC_CONNECTEX](funcPointer)
block:
let res = getFunc(sock, funcPointer, WSAID_ACCEPTEX)
if not(res):
raiseOsDefect(osLastError(), "initAPI(): Unable to initialize " &
"dispatcher's AcceptEx()")
loop.acceptEx = cast[WSAPROC_ACCEPTEX](funcPointer)
block:
let res = getFunc(sock, funcPointer, WSAID_GETACCEPTEXSOCKADDRS)
if not(res):
raiseOsDefect(osLastError(), "initAPI(): Unable to initialize " &
"dispatcher's GetAcceptExSockAddrs()")
loop.getAcceptExSockAddrs =
cast[WSAPROC_GETACCEPTEXSOCKADDRS](funcPointer)
block:
let res = getFunc(sock, funcPointer, WSAID_TRANSMITFILE)
if not(res):
raiseOsDefect(osLastError(), "initAPI(): Unable to initialize " &
"dispatcher's TransmitFile()")
loop.transmitFile = cast[WSAPROC_TRANSMITFILE](funcPointer)
if closeFd(sock) != 0:
raiseOsDefect(osLastError(), "initAPI(): Unable to close control socket")
proc newDispatcher*(): PDispatcher =
2018-05-16 08:22:34 +00:00
## Creates a new Dispatcher instance.
let port = createIoCompletionPort(osdefs.INVALID_HANDLE_VALUE,
HANDLE(0), 0, 1)
if port == osdefs.INVALID_HANDLE_VALUE:
raiseOsDefect(osLastError(), "newDispatcher(): Unable to create " &
"IOCP port")
var res = PDispatcher(
ioPort: port,
handles: initHashSet[AsyncFD](),
timers: initHeapQueue[TimerCallback](),
callbacks: initDeque[AsyncCallback](64),
idlers: initDeque[AsyncCallback](),
trackers: initTable[string, TrackerBase]()
)
res.callbacks.addLast(SentinelCallback)
initAPI(res)
res
2018-05-16 08:22:34 +00:00
var gDisp{.threadvar.}: PDispatcher ## Global dispatcher
proc setThreadDispatcher*(disp: PDispatcher) {.gcsafe, raises: [Defect].}
proc getThreadDispatcher*(): PDispatcher {.gcsafe, raises: [Defect].}
2018-05-16 08:22:34 +00:00
proc getIoHandler*(disp: PDispatcher): HANDLE =
2018-05-16 08:22:34 +00:00
## Returns the underlying IO Completion Port handle (Windows) or selector
## (Unix) for the specified dispatcher.
disp.ioPort
2018-05-16 08:22:34 +00:00
proc register2*(fd: AsyncFD): Result[void, OSErrorCode] =
## Register file descriptor ``fd`` in thread's dispatcher.
let loop = getThreadDispatcher()
if createIoCompletionPort(HANDLE(fd), loop.ioPort, cast[CompletionKey](fd),
1) == osdefs.INVALID_HANDLE_VALUE:
return err(osLastError())
loop.handles.incl(fd)
ok()
2018-05-16 08:22:34 +00:00
proc register*(fd: AsyncFD) {.raises: [Defect, OSError].} =
## Register file descriptor ``fd`` in thread's dispatcher.
register2(fd).tryGet()
proc unregister*(fd: AsyncFD) =
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
## Unregisters ``fd``.
getThreadDispatcher().handles.excl(fd)
proc poll*() =
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
## Perform single asynchronous step, processing timers and completing
## tasks. Blocks until at least one event has completed.
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
##
## Exceptions raised here indicate that waiting for tasks to be unblocked
## failed - exceptions from within tasks are instead propagated through
## their respective futures and not allowed to interrrupt the poll call.
let loop = getThreadDispatcher()
var
curTime = Moment.now()
curTimeout = DWORD(0)
events: array[MaxEventsCount, osdefs.OVERLAPPED_ENTRY]
# On reentrant `poll` calls from `processCallbacks`, e.g., `waitFor`,
# complete pending work of the outer `processCallbacks` call.
# On non-reentrant `poll` calls, this only removes sentinel element.
processCallbacks(loop)
2018-05-16 08:22:34 +00:00
# Moving expired timers to `loop.callbacks` and calculate timeout
2018-06-02 23:03:48 +00:00
loop.processTimersGetTimeout(curTimeout)
let networkEventsCount =
if isNil(loop.getQueuedCompletionStatusEx):
let res = getQueuedCompletionStatus(
loop.ioPort,
addr events[0].dwNumberOfBytesTransferred,
addr events[0].lpCompletionKey,
cast[ptr POVERLAPPED](addr events[0].lpOverlapped),
curTimeout
)
if res == FALSE:
let errCode = osLastError()
if not(isNil(events[0].lpOverlapped)):
1
else:
if uint32(errCode) != WAIT_TIMEOUT:
raiseOsDefect(errCode, "poll(): Unable to get OS events")
0
else:
1
else:
var eventsReceived = ULONG(0)
let res = loop.getQueuedCompletionStatusEx(
loop.ioPort,
addr events[0],
ULONG(len(events)),
eventsReceived,
curTimeout,
WINBOOL(0)
)
if res == FALSE:
let errCode = osLastError()
if uint32(errCode) != WAIT_TIMEOUT:
raiseOsDefect(errCode, "poll(): Unable to get OS events")
0
else:
int(eventsReceived)
for i in 0 ..< networkEventsCount:
var customOverlapped = PtrCustomOverlapped(events[i].lpOverlapped)
customOverlapped.data.errCode =
block:
let res = cast[uint64](customOverlapped.internal)
if res == 0'u64:
OSErrorCode(-1)
else:
OSErrorCode(rtlNtStatusToDosError(res))
customOverlapped.data.bytesCount = events[i].dwNumberOfBytesTransferred
2018-05-16 08:22:34 +00:00
let acb = AsyncCallback(function: customOverlapped.data.cb,
udata: cast[pointer](customOverlapped))
loop.callbacks.addLast(acb)
# Moving expired timers to `loop.callbacks`.
2018-06-02 23:03:48 +00:00
loop.processTimers()
2018-05-16 08:22:34 +00:00
# We move idle callbacks to `loop.callbacks` only if there no pending
# network events.
if networkEventsCount == 0:
loop.processIdlers()
# All callbacks which will be added during `processCallbacks` will be
# scheduled after the sentinel and are processed on next `poll()` call.
loop.callbacks.addLast(SentinelCallback)
processCallbacks(loop)
# All callbacks done, skip `processCallbacks` at start.
loop.callbacks.addFirst(SentinelCallback)
2018-05-16 08:22:34 +00:00
proc closeSocket*(fd: AsyncFD, aftercb: CallbackFunc = nil) =
2018-05-16 08:22:34 +00:00
## Closes a socket and ensures that it is unregistered.
let loop = getThreadDispatcher()
loop.handles.excl(fd)
let
param = toPointer(
if closeFd(SocketHandle(fd)) == 0:
OSErrorCode(0)
else:
osLastError()
)
if not(isNil(aftercb)):
loop.callbacks.addLast(AsyncCallback(function: aftercb, udata: param))
2018-05-16 08:22:34 +00:00
proc closeHandle*(fd: AsyncFD, aftercb: CallbackFunc = nil) =
## Closes a (pipe/file) handle and ensures that it is unregistered.
let loop = getThreadDispatcher()
loop.handles.excl(fd)
let
param = toPointer(
if closeFd(HANDLE(fd)) == 0:
OSErrorCode(0)
else:
osLastError()
)
if not(isNil(aftercb)):
loop.callbacks.addLast(AsyncCallback(function: aftercb, udata: param))
2018-05-16 08:22:34 +00:00
proc contains*(disp: PDispatcher, fd: AsyncFD): bool =
## Returns ``true`` if ``fd`` is registered in thread's dispatcher.
fd in disp.handles
2018-05-16 08:22:34 +00:00
elif defined(macosx) or defined(freebsd) or defined(netbsd) or
defined(openbsd) or defined(dragonfly) or defined(macos) or
defined(linux) or defined(android) or defined(solaris):
const
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
SIG_IGN = cast[proc(x: cint) {.raises: [], noconv, gcsafe.}](1)
2018-05-16 08:22:34 +00:00
type
AsyncFD* = distinct cint
SelectorData* = object
reader*: AsyncCallback
writer*: AsyncCallback
PDispatcher* = ref object of PDispatcherBase
selector: Selector[SelectorData]
keys: seq[ReadyKey]
2021-11-20 00:30:23 +00:00
proc `==`*(x, y: AsyncFD): bool {.borrow, gcsafe.}
2018-05-16 08:22:34 +00:00
2019-09-23 17:24:26 +00:00
proc globalInit() =
# We are ignoring SIGPIPE signal, because we are working with EPIPE.
posix.signal(cint(SIGPIPE), SIG_IGN)
proc initAPI(disp: PDispatcher) =
2019-09-23 17:24:26 +00:00
discard
proc newDispatcher*(): PDispatcher =
## Create new dispatcher.
let selector =
block:
let res = Selector.new(SelectorData)
if res.isErr(): raiseOsDefect(res.error(),
"Could not initialize selector")
res.get()
var res = PDispatcher(
selector: selector,
timers: initHeapQueue[TimerCallback](),
callbacks: initDeque[AsyncCallback](asyncEventsCount),
idlers: initDeque[AsyncCallback](),
keys: newSeq[ReadyKey](asyncEventsCount),
trackers: initTable[string, TrackerBase]()
)
res.callbacks.addLast(SentinelCallback)
initAPI(res)
res
2018-05-16 08:22:34 +00:00
var gDisp{.threadvar.}: PDispatcher ## Global dispatcher
proc setThreadDispatcher*(disp: PDispatcher) {.gcsafe, raises: [Defect].}
proc getThreadDispatcher*(): PDispatcher {.gcsafe, raises: [Defect].}
2018-05-16 08:22:34 +00:00
proc getIoHandler*(disp: PDispatcher): Selector[SelectorData] =
## Returns system specific OS queue.
disp.selector
2018-05-16 08:22:34 +00:00
proc contains*(disp: PDispatcher, fd: AsyncFD): bool {.inline.} =
## Returns ``true`` if ``fd`` is registered in thread's dispatcher.
cint(fd) in disp.selector
proc register2*(fd: AsyncFD): Result[void, OSErrorCode] =
## Register file descriptor ``fd`` in thread's dispatcher.
var data: SelectorData
getThreadDispatcher().selector.registerHandle2(cint(fd), {}, data)
2018-05-16 08:22:34 +00:00
proc unregister2*(fd: AsyncFD): Result[void, OSErrorCode] =
## Unregister file descriptor ``fd`` from thread's dispatcher.
getThreadDispatcher().selector.unregister2(cint(fd))
2018-05-16 08:22:34 +00:00
proc addReader2*(fd: AsyncFD, cb: CallbackFunc,
udata: pointer = nil): Result[void, OSErrorCode] =
## Start watching the file descriptor ``fd`` for read availability and then
## call the callback ``cb`` with specified argument ``udata``.
let loop = getThreadDispatcher()
2018-05-16 08:22:34 +00:00
var newEvents = {Event.Read}
withData(loop.selector, cint(fd), adata) do:
let acb = AsyncCallback(function: cb, udata: udata)
2018-05-16 08:22:34 +00:00
adata.reader = acb
2020-01-08 17:06:56 +00:00
if not(isNil(adata.writer.function)):
2020-01-08 01:06:27 +00:00
newEvents.incl(Event.Write)
2018-05-16 08:22:34 +00:00
do:
return err(OSErrorCode(osdefs.EBADF))
loop.selector.updateHandle2(cint(fd), newEvents)
2018-05-16 08:22:34 +00:00
proc removeReader2*(fd: AsyncFD): Result[void, OSErrorCode] =
## Stop watching the file descriptor ``fd`` for read availability.
let loop = getThreadDispatcher()
2018-05-16 08:22:34 +00:00
var newEvents: set[Event]
withData(loop.selector, cint(fd), adata) do:
# We need to clear `reader` data, because `selectors` don't do it
adata.reader = default(AsyncCallback)
2020-01-08 17:06:56 +00:00
if not(isNil(adata.writer.function)):
2020-01-08 01:06:27 +00:00
newEvents.incl(Event.Write)
2018-05-16 08:22:34 +00:00
do:
return err(OSErrorCode(osdefs.EBADF))
loop.selector.updateHandle2(cint(fd), newEvents)
proc addWriter2*(fd: AsyncFD, cb: CallbackFunc,
udata: pointer = nil): Result[void, OSErrorCode] =
## Start watching the file descriptor ``fd`` for write availability and then
## call the callback ``cb`` with specified argument ``udata``.
let loop = getThreadDispatcher()
2018-05-16 08:22:34 +00:00
var newEvents = {Event.Write}
withData(loop.selector, cint(fd), adata) do:
let acb = AsyncCallback(function: cb, udata: udata)
2018-05-16 08:22:34 +00:00
adata.writer = acb
2020-01-08 17:06:56 +00:00
if not(isNil(adata.reader.function)):
2020-01-08 01:06:27 +00:00
newEvents.incl(Event.Read)
2018-05-16 08:22:34 +00:00
do:
return err(OSErrorCode(osdefs.EBADF))
loop.selector.updateHandle2(cint(fd), newEvents)
2018-05-16 08:22:34 +00:00
proc removeWriter2*(fd: AsyncFD): Result[void, OSErrorCode] =
## Stop watching the file descriptor ``fd`` for write availability.
let loop = getThreadDispatcher()
2018-05-16 08:22:34 +00:00
var newEvents: set[Event]
withData(loop.selector, cint(fd), adata) do:
# We need to clear `writer` data, because `selectors` don't do it
adata.writer = default(AsyncCallback)
2020-01-08 17:06:56 +00:00
if not(isNil(adata.reader.function)):
2020-01-08 01:06:27 +00:00
newEvents.incl(Event.Read)
2018-05-16 08:22:34 +00:00
do:
return err(OSErrorCode(osdefs.EBADF))
loop.selector.updateHandle2(cint(fd), newEvents)
proc register*(fd: AsyncFD) {.raises: [Defect, OSError].} =
## Register file descriptor ``fd`` in thread's dispatcher.
register2(fd).tryGet()
proc unregister*(fd: AsyncFD) {.raises: [Defect, OSError].} =
## Unregister file descriptor ``fd`` from thread's dispatcher.
unregister2(fd).tryGet()
proc addReader*(fd: AsyncFD, cb: CallbackFunc, udata: pointer = nil) {.
raises: [Defect, OSError].} =
## Start watching the file descriptor ``fd`` for read availability and then
## call the callback ``cb`` with specified argument ``udata``.
addReader2(fd, cb, udata).tryGet()
proc removeReader*(fd: AsyncFD) {.raises: [Defect, OSError].} =
## Stop watching the file descriptor ``fd`` for read availability.
removeReader2(fd).tryGet()
proc addWriter*(fd: AsyncFD, cb: CallbackFunc, udata: pointer = nil) {.
raises: [Defect, OSError].} =
## Start watching the file descriptor ``fd`` for write availability and then
## call the callback ``cb`` with specified argument ``udata``.
addWriter2(fd, cb, udata).tryGet()
proc removeWriter*(fd: AsyncFD) {.raises: [Defect, OSError].} =
## Stop watching the file descriptor ``fd`` for write availability.
removeWriter2(fd).tryGet()
proc unregisterAndCloseFd*(fd: AsyncFD): Result[void, OSErrorCode] =
## Unregister from system queue and close asynchronous socket.
##
## NOTE: Use this function to close temporary sockets/pipes only (which
## are not exposed to the public and not supposed to be used/reused).
## Please use closeSocket(AsyncFD) and closeHandle(AsyncFD) instead.
doAssert(fd != AsyncFD(osdefs.INVALID_SOCKET))
? unregister2(fd)
if closeFd(cint(fd)) != 0:
err(osLastError())
else:
ok()
proc closeSocket*(fd: AsyncFD, aftercb: CallbackFunc = nil) =
## Close asynchronous socket.
##
## Please note, that socket is not closed immediately. To avoid bugs with
## closing socket, while operation pending, socket will be closed as
## soon as all pending operations will be notified.
## You can execute ``aftercb`` before actual socket close operation.
let loop = getThreadDispatcher()
proc continuation(udata: pointer) =
let
param = toPointer(
if SocketHandle(fd) in loop.selector:
let ures = unregister2(fd)
if ures.isErr():
discard closeFd(cint(fd))
ures.error()
else:
if closeFd(cint(fd)) != 0:
osLastError()
else:
OSErrorCode(0)
else:
OSErrorCode(osdefs.EBADF)
)
if not(isNil(aftercb)): aftercb(param)
withData(loop.selector, cint(fd), adata) do:
# We are scheduling reader and writer callbacks to be called
# explicitly, so they can get an error and continue work.
# Callbacks marked as deleted so we don't need to get REAL notifications
# from system queue for this reader and writer.
2020-01-08 17:06:56 +00:00
if not(isNil(adata.reader.function)):
loop.callbacks.addLast(adata.reader)
adata.reader = default(AsyncCallback)
2020-01-08 17:06:56 +00:00
if not(isNil(adata.writer.function)):
loop.callbacks.addLast(adata.writer)
adata.writer = default(AsyncCallback)
# We can't unregister file descriptor from system queue here, because
# in such case processing queue will stuck on poll() call, because there
# can be no file descriptors registered in system queue.
var acb = AsyncCallback(function: continuation)
loop.callbacks.addLast(acb)
2018-05-16 08:22:34 +00:00
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
proc closeHandle*(fd: AsyncFD, aftercb: CallbackFunc = nil) =
## Close asynchronous file/pipe handle.
##
## Please note, that socket is not closed immediately. To avoid bugs with
## closing socket, while operation pending, socket will be closed as
## soon as all pending operations will be notified.
## You can execute ``aftercb`` before actual socket close operation.
closeSocket(fd, aftercb)
when asyncEventEngine in ["epoll", "kqueue"]:
proc addSignal2*(signal: int, cb: CallbackFunc,
udata: pointer = nil): Result[int, OSErrorCode] =
## Start watching signal ``signal``, and when signal appears, call the
## callback ``cb`` with specified argument ``udata``. Returns signal
## identifier code, which can be used to remove signal callback
## via ``removeSignal``.
let loop = getThreadDispatcher()
var data: SelectorData
let sigfd = ? loop.selector.registerSignal(signal, data)
withData(loop.selector, sigfd, adata) do:
adata.reader = AsyncCallback(function: cb, udata: udata)
do:
return err(OSErrorCode(osdefs.EBADF))
ok(sigfd)
proc addProcess2*(pid: int, cb: CallbackFunc,
udata: pointer = nil): Result[int, OSErrorCode] =
## Registers callback ``cb`` to be called when process with process
## identifier ``pid`` exited. Returns process' descriptor, which can be
## used to clear process callback via ``removeProcess``.
let loop = getThreadDispatcher()
var data: SelectorData
let procfd = ? loop.selector.registerProcess(pid, data)
withData(loop.selector, procfd, adata) do:
adata.reader = AsyncCallback(function: cb, udata: udata)
do:
return err(OSErrorCode(osdefs.EBADF))
ok(procfd)
proc removeSignal2*(sigfd: int): Result[void, OSErrorCode] =
## Remove watching signal ``signal``.
getThreadDispatcher().selector.unregister2(cint(sigfd))
proc removeProcess2*(procfd: int): Result[void, OSErrorCode] =
## Remove process' watching using process' descriptor ``procfd``.
getThreadDispatcher().selector.unregister2(cint(procfd))
proc addSignal*(signal: int, cb: CallbackFunc,
udata: pointer = nil): int {.raises: [Defect, OSError].} =
## Start watching signal ``signal``, and when signal appears, call the
## callback ``cb`` with specified argument ``udata``. Returns signal
## identifier code, which can be used to remove signal callback
## via ``removeSignal``.
addSignal2(signal, cb, udata).tryGet()
proc removeSignal*(sigfd: int) {.raises: [Defect, OSError].} =
## Remove watching signal ``signal``.
removeSignal2(sigfd).tryGet()
proc removeProcess*(procfd: int) {.raises: [Defect, OSError].} =
## Remove process' watching using process' descriptor ``procfd``.
removeProcess2(procfd).tryGet()
proc poll*() {.gcsafe.} =
## Perform single asynchronous step.
let loop = getThreadDispatcher()
var curTime = Moment.now()
2018-05-16 08:22:34 +00:00
var curTimeout = 0
# On reentrant `poll` calls from `processCallbacks`, e.g., `waitFor`,
# complete pending work of the outer `processCallbacks` call.
# On non-reentrant `poll` calls, this only removes sentinel element.
processCallbacks(loop)
2018-05-25 20:00:32 +00:00
# Moving expired timers to `loop.callbacks` and calculate timeout.
2018-06-02 23:03:48 +00:00
loop.processTimersGetTimeout(curTimeout)
2018-05-16 08:22:34 +00:00
2018-05-25 20:00:32 +00:00
# Processing IO descriptors and all hardware events.
let count =
block:
let res = loop.selector.selectInto2(curTimeout, loop.keys)
if res.isErr():
raiseOsDefect(res.error(), "poll(): Unable to get OS events")
res.get()
for i in 0 ..< count:
2018-05-16 08:22:34 +00:00
let fd = loop.keys[i].fd
let events = loop.keys[i].events
withData(loop.selector, cint(fd), adata) do:
if (Event.Read in events) or (events == {Event.Error}):
if not isNil(adata.reader.function):
loop.callbacks.addLast(adata.reader)
2018-05-16 08:22:34 +00:00
if (Event.Write in events) or (events == {Event.Error}):
if not isNil(adata.writer.function):
loop.callbacks.addLast(adata.writer)
2018-05-16 08:22:34 +00:00
if Event.User in events:
if not isNil(adata.reader.function):
loop.callbacks.addLast(adata.reader)
2018-05-16 08:22:34 +00:00
when asyncEventEngine in ["epoll", "kqueue"]:
let customSet = {Event.Timer, Event.Signal, Event.Process,
Event.Vnode}
if customSet * events != {}:
if not isNil(adata.reader.function):
loop.callbacks.addLast(adata.reader)
2018-05-16 08:22:34 +00:00
# Moving expired timers to `loop.callbacks`.
2018-06-02 23:03:48 +00:00
loop.processTimers()
2018-05-16 08:22:34 +00:00
# We move idle callbacks to `loop.callbacks` only if there no pending
# network events.
if count == 0:
loop.processIdlers()
# All callbacks which will be added during `processCallbacks` will be
# scheduled after the sentinel and are processed on next `poll()` call.
loop.callbacks.addLast(SentinelCallback)
processCallbacks(loop)
# All callbacks done, skip `processCallbacks` at start.
loop.callbacks.addFirst(SentinelCallback)
2018-05-16 08:22:34 +00:00
else:
proc initAPI() = discard
2019-09-23 17:24:26 +00:00
proc globalInit() = discard
proc setThreadDispatcher*(disp: PDispatcher) =
## Set current thread's dispatcher instance to ``disp``.
if not(gDisp.isNil()):
doAssert gDisp.callbacks.len == 0
gDisp = disp
proc getThreadDispatcher*(): PDispatcher =
## Returns current thread's dispatcher instance.
if gDisp.isNil():
setThreadDispatcher(newDispatcher())
gDisp
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
proc setGlobalDispatcher*(disp: PDispatcher) {.
gcsafe, deprecated: "Use setThreadDispatcher() instead".} =
setThreadDispatcher(disp)
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
proc getGlobalDispatcher*(): PDispatcher {.
gcsafe, deprecated: "Use getThreadDispatcher() instead".} =
getThreadDispatcher()
proc setTimer*(at: Moment, cb: CallbackFunc,
udata: pointer = nil): TimerCallback =
2020-01-07 05:26:18 +00:00
## Arrange for the callback ``cb`` to be called at the given absolute
## timestamp ``at``. You can also pass ``udata`` to callback.
let loop = getThreadDispatcher()
2020-01-08 01:06:27 +00:00
result = TimerCallback(finishAt: at,
function: AsyncCallback(function: cb, udata: udata))
2020-01-08 01:06:27 +00:00
loop.timers.push(result)
2020-01-07 05:26:18 +00:00
proc clearTimer*(timer: TimerCallback) {.inline.} =
timer.function = default(AsyncCallback)
2020-01-07 05:26:18 +00:00
proc addTimer*(at: Moment, cb: CallbackFunc, udata: pointer = nil) {.
inline, deprecated: "Use setTimer/clearTimer instead".} =
## Arrange for the callback ``cb`` to be called at the given absolute
## timestamp ``at``. You can also pass ``udata`` to callback.
discard setTimer(at, cb, udata)
2018-05-16 08:22:34 +00:00
proc addTimer*(at: int64, cb: CallbackFunc, udata: pointer = nil) {.
inline, deprecated: "Use addTimer(Duration, cb, udata)".} =
discard setTimer(Moment.init(at, Millisecond), cb, udata)
proc addTimer*(at: uint64, cb: CallbackFunc, udata: pointer = nil) {.
inline, deprecated: "Use addTimer(Duration, cb, udata)".} =
discard setTimer(Moment.init(int64(at), Millisecond), cb, udata)
proc removeTimer*(at: Moment, cb: CallbackFunc, udata: pointer = nil) =
## Remove timer callback ``cb`` with absolute timestamp ``at`` from waiting
## queue.
let loop = getThreadDispatcher()
2018-05-16 08:22:34 +00:00
var list = cast[seq[TimerCallback]](loop.timers)
var index = -1
for i in 0..<len(list):
if list[i].finishAt == at and list[i].function.function == cb and
list[i].function.udata == udata:
index = i
break
if index != -1:
loop.timers.del(index)
proc removeTimer*(at: int64, cb: CallbackFunc, udata: pointer = nil) {.
inline, deprecated: "Use removeTimer(Duration, cb, udata)".} =
removeTimer(Moment.init(at, Millisecond), cb, udata)
proc removeTimer*(at: uint64, cb: CallbackFunc, udata: pointer = nil) {.
inline, deprecated: "Use removeTimer(Duration, cb, udata)".} =
removeTimer(Moment.init(int64(at), Millisecond), cb, udata)
proc callSoon*(acb: AsyncCallback) =
## Schedule `cbproc` to be called as soon as possible.
## The callback is called when control returns to the event loop.
getThreadDispatcher().callbacks.addLast(acb)
proc callSoon*(cbproc: CallbackFunc, data: pointer) {.
gcsafe.} =
## Schedule `cbproc` to be called as soon as possible.
## The callback is called when control returns to the event loop.
doAssert(not isNil(cbproc))
callSoon(AsyncCallback(function: cbproc, udata: data))
proc callSoon*(cbproc: CallbackFunc) =
callSoon(cbproc, nil)
proc callIdle*(acb: AsyncCallback) =
## Schedule ``cbproc`` to be called when there no pending network events
## available.
##
## **WARNING!** Despite the name, "idle" callbacks called on every loop
## iteration if there no network events available, not when the loop is
## actually "idle".
getThreadDispatcher().idlers.addLast(acb)
proc callIdle*(cbproc: CallbackFunc, data: pointer) =
## Schedule ``cbproc`` to be called when there no pending network events
## available.
##
## **WARNING!** Despite the name, "idle" callbacks called on every loop
## iteration if there no network events available, not when the loop is
## actually "idle".
doAssert(not isNil(cbproc))
callIdle(AsyncCallback(function: cbproc, udata: data))
proc callIdle*(cbproc: CallbackFunc) =
callIdle(cbproc, nil)
include asyncfutures2
when not(defined(windows)):
when asyncEventEngine in ["epoll", "kqueue"]:
proc waitSignal*(signal: int): Future[void] {.raises: [Defect].} =
var retFuture = newFuture[void]("chronos.waitSignal()")
var sigfd: int = -1
template getSignalException(e: OSErrorCode): untyped =
newException(AsyncError, "Could not manipulate signal handler, " &
"reason [" & $int(e) & "]: " & osErrorMsg(e))
proc continuation(udata: pointer) {.gcsafe.} =
if not(retFuture.finished()):
if sigfd != -1:
let res = removeSignal2(sigfd)
if res.isErr():
retFuture.fail(getSignalException(res.error()))
else:
retFuture.complete()
proc cancellation(udata: pointer) {.gcsafe.} =
if not(retFuture.finished()):
if sigfd != -1:
let res = removeSignal2(sigfd)
if res.isErr():
retFuture.fail(getSignalException(res.error()))
sigfd =
block:
let res = addSignal2(signal, continuation)
if res.isErr():
retFuture.fail(getSignalException(res.error()))
res.get()
retFuture.cancelCallback = cancellation
retFuture
proc sleepAsync*(duration: Duration): Future[void] =
2018-05-16 08:22:34 +00:00
## Suspends the execution of the current async procedure for the next
## ``duration`` time.
var retFuture = newFuture[void]("chronos.sleepAsync(Duration)")
2019-06-20 20:30:41 +00:00
let moment = Moment.fromNow(duration)
2020-01-08 01:06:27 +00:00
var timer: TimerCallback
2019-06-20 20:30:41 +00:00
proc completion(data: pointer) {.gcsafe.} =
if not(retFuture.finished()):
retFuture.complete()
2019-06-20 20:30:41 +00:00
proc cancellation(udata: pointer) {.gcsafe.} =
if not(retFuture.finished()):
clearTimer(timer)
2019-06-20 20:30:41 +00:00
retFuture.cancelCallback = cancellation
2020-01-07 05:26:18 +00:00
timer = setTimer(moment, completion, cast[pointer](retFuture))
2018-05-16 08:22:34 +00:00
return retFuture
proc sleepAsync*(ms: int): Future[void] {.
inline, deprecated: "Use sleepAsync(Duration)".} =
result = sleepAsync(ms.milliseconds())
proc stepsAsync*(number: int): Future[void] =
## Suspends the execution of the current async procedure for the next
## ``number`` of asynchronous steps (``poll()`` calls).
##
## This primitive can be useful when you need to create more deterministic
## tests and cases.
##
## WARNING! Do not use this primitive to perform switch between tasks, because
## this can lead to 100% CPU load in the moments when there are no I/O
## events. Usually when there no I/O events CPU consumption should be near 0%.
var retFuture = newFuture[void]("chronos.stepsAsync(int)")
var counter = 0
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
var continuation: proc(data: pointer) {.gcsafe, raises: [Defect].}
continuation = proc(data: pointer) {.gcsafe, raises: [Defect].} =
if not(retFuture.finished()):
inc(counter)
if counter < number:
callSoon(continuation, nil)
else:
retFuture.complete()
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
proc cancellation(udata: pointer) =
discard
if number <= 0:
retFuture.complete()
else:
retFuture.cancelCallback = cancellation
callSoon(continuation, nil)
retFuture
proc idleAsync*(): Future[void] =
## Suspends the execution of the current asynchronous task until "idle" time.
##
## "idle" time its moment of time, when no network events were processed by
## ``poll()`` call.
var retFuture = newFuture[void]("chronos.idleAsync()")
proc continuation(data: pointer) {.gcsafe.} =
if not(retFuture.finished()):
retFuture.complete()
proc cancellation(udata: pointer) {.gcsafe.} =
discard
retFuture.cancelCallback = cancellation
callIdle(continuation, nil)
retFuture
proc withTimeout*[T](fut: Future[T], timeout: Duration): Future[bool] =
2018-05-16 08:22:34 +00:00
## Returns a future which will complete once ``fut`` completes or after
## ``timeout`` milliseconds has elapsed.
##
## If ``fut`` completes first the returned future will hold true,
## otherwise, if ``timeout`` milliseconds has elapsed first, the returned
## future will hold false.
2019-06-20 20:30:41 +00:00
var retFuture = newFuture[bool]("chronos.`withTimeout`")
var moment: Moment
2020-01-08 01:06:27 +00:00
var timer: TimerCallback
var cancelling = false
2019-06-20 20:30:41 +00:00
# TODO: raises annotation shouldn't be needed, but likely similar issue as
# https://github.com/nim-lang/Nim/issues/17369
proc continuation(udata: pointer) {.gcsafe, raises: [Defect].} =
2019-06-20 20:30:41 +00:00
if not(retFuture.finished()):
if not(cancelling):
if not(fut.finished()):
# Timer exceeded first, we going to cancel `fut` and wait until it
# not completes.
cancelling = true
fut.cancel()
else:
# Future `fut` completed/failed/cancelled first.
if not(isNil(timer)):
clearTimer(timer)
retFuture.complete(true)
2018-05-16 08:22:34 +00:00
else:
retFuture.complete(false)
2019-06-20 20:30:41 +00:00
# TODO: raises annotation shouldn't be needed, but likely similar issue as
# https://github.com/nim-lang/Nim/issues/17369
proc cancellation(udata: pointer) {.gcsafe, raises: [Defect].} =
if not isNil(timer):
clearTimer(timer)
if not(fut.finished()):
fut.removeCallback(continuation)
fut.cancel()
2019-06-20 20:30:41 +00:00
if fut.finished():
retFuture.complete(true)
else:
if timeout.isZero():
retFuture.complete(false)
elif timeout.isInfinite():
retFuture.cancelCallback = cancellation
2019-06-20 20:30:41 +00:00
fut.addCallback(continuation)
else:
moment = Moment.fromNow(timeout)
retFuture.cancelCallback = cancellation
2020-01-07 05:26:18 +00:00
timer = setTimer(moment, continuation, nil)
2019-06-20 20:30:41 +00:00
fut.addCallback(continuation)
2018-05-16 08:22:34 +00:00
return retFuture
proc withTimeout*[T](fut: Future[T], timeout: int): Future[bool] {.
inline, deprecated: "Use withTimeout(Future[T], Duration)".} =
result = withTimeout(fut, timeout.milliseconds())
proc wait*[T](fut: Future[T], timeout = InfiniteDuration): Future[T] =
## Returns a future which will complete once future ``fut`` completes
## or if timeout of ``timeout`` milliseconds has been expired.
##
2018-07-20 09:11:38 +00:00
## If ``timeout`` is ``-1``, then statement ``await wait(fut)`` is
## equal to ``await fut``.
2019-06-20 20:30:41 +00:00
##
## TODO: In case when ``fut`` got cancelled, what result Future[T]
## should return, because it can't be cancelled too.
var retFuture = newFuture[T]("chronos.wait()")
var moment: Moment
2020-01-08 01:06:27 +00:00
var timer: TimerCallback
var cancelling = false
2019-06-20 20:30:41 +00:00
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
proc continuation(udata: pointer) {.raises: [Defect].} =
2019-06-20 20:30:41 +00:00
if not(retFuture.finished()):
if not(cancelling):
if not(fut.finished()):
# Timer exceeded first.
cancelling = true
fut.cancel()
2019-03-27 22:56:17 +00:00
else:
# Future `fut` completed/failed/cancelled first.
if not isNil(timer):
clearTimer(timer)
if fut.failed():
retFuture.fail(fut.error)
2019-06-20 20:30:41 +00:00
else:
when T is void:
retFuture.complete()
else:
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
retFuture.complete(fut.value)
else:
retFuture.fail(newException(AsyncTimeoutError, "Timeout exceeded!"))
2019-06-20 20:30:41 +00:00
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
var cancellation: proc(udata: pointer) {.gcsafe, raises: [Defect].}
cancellation = proc(udata: pointer) {.gcsafe, raises: [Defect].} =
if not isNil(timer):
clearTimer(timer)
if not(fut.finished()):
fut.removeCallback(continuation)
fut.cancel()
2019-06-20 20:30:41 +00:00
if fut.finished():
if fut.failed():
retFuture.fail(fut.error)
else:
2019-06-20 20:30:41 +00:00
when T is void:
retFuture.complete()
else:
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
retFuture.complete(fut.value)
else:
2019-06-20 20:30:41 +00:00
if timeout.isZero():
retFuture.fail(newException(AsyncTimeoutError, "Timeout exceeded!"))
elif timeout.isInfinite():
retFuture.cancelCallback = cancellation
2019-06-20 20:30:41 +00:00
fut.addCallback(continuation)
else:
moment = Moment.fromNow(timeout)
retFuture.cancelCallback = cancellation
2020-01-07 05:26:18 +00:00
timer = setTimer(moment, continuation, nil)
2019-06-20 20:30:41 +00:00
fut.addCallback(continuation)
return retFuture
proc wait*[T](fut: Future[T], timeout = -1): Future[T] {.
inline, deprecated: "Use wait(Future[T], Duration)".} =
if timeout == -1:
wait(fut, InfiniteDuration)
elif timeout == 0:
wait(fut, ZeroDuration)
else:
wait(fut, timeout.milliseconds())
2018-05-16 08:22:34 +00:00
include asyncmacro2
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
proc runForever*() {.raises: [Defect, CatchableError].} =
2018-05-16 08:22:34 +00:00
## Begins a never ending global dispatcher poll loop.
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
## Raises different exceptions depending on the platform.
2018-05-16 08:22:34 +00:00
while true:
poll()
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
proc waitFor*[T](fut: Future[T]): T {.raises: [Defect, CatchableError].} =
2018-05-16 08:22:34 +00:00
## **Blocks** the current thread until the specified future completes.
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
## There's no way to tell if poll or read raised the exception
2019-06-20 20:30:41 +00:00
while not(fut.finished()):
2018-05-16 08:22:34 +00:00
poll()
2019-06-20 20:30:41 +00:00
fut.read()
proc addTracker*[T](id: string, tracker: T) =
## Add new ``tracker`` object to current thread dispatcher with identifier
## ``id``.
let loop = getThreadDispatcher()
loop.trackers[id] = tracker
proc getTracker*(id: string): TrackerBase =
## Get ``tracker`` from current thread dispatcher using identifier ``id``.
let loop = getThreadDispatcher()
result = loop.trackers.getOrDefault(id, nil)
2019-09-23 17:24:26 +00:00
when chronosFutureTracking:
iterator pendingFutures*(): FutureBase =
## Iterates over the list of pending Futures (Future[T] objects which not
## yet completed, cancelled or failed).
var slider = futureList.head
while not(isNil(slider)):
yield slider
slider = slider.next
proc pendingFuturesCount*(): uint =
## Returns number of pending Futures (Future[T] objects which not yet
## completed, cancelled or failed).
futureList.count
2019-09-23 17:24:26 +00:00
# Perform global per-module initialization.
globalInit()