nim-chronos/chronos/handles.nim

233 lines
8.2 KiB
Nim
Raw Normal View History

2018-05-16 08:22:34 +00:00
#
# Chronos Handles
# (c) Copyright 2018-Present
2018-05-16 08:22:34 +00:00
# Status Research & Development GmbH
#
# Licensed under either of
# Apache License, version 2.0, (LICENSE-APACHEv2)
# MIT license (LICENSE-MIT)
when (NimMajor, NimMinor) < (1, 4):
{.push raises: [Defect].}
else:
{.push raises: [].}
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
import "."/[asyncloop, osdefs, osutils]
import stew/results
from nativesockets import Domain, Protocol, SockType, toInt
Add facility to emulate signals on Windows. (#357) * Initial Windows asyncproc part. Deprecate usage of net/nativesockets in handles, asyncloop. Introduce primitives with inheritable flag. * Some posix fixes. * More duplicate fixes. * Fix AsyncProcessOption.EchoCommand option. Remove isMainModule code. Fix execCommand to use AsyncProcessOption.EvalCommand. * Initial asyncproc tests commit. * Some Posix fixes. * Fix Posix crash. * Add testsuite. Fix osnet.nim to use proper declarations. Fix timer.nim to use proper declarations. Add Windows environment handling procedures. Fix createAsyncPipe. Add leaks tracking for AsyncProcessRef. * Fix O_CLOEXEC constant value. * Add file descriptors leak test. * Remove commented code. Refactor exceptions. Fix compilation warnings. * No exception ioselectors_kqueue initial commit. * Some BSD fixes. Linux refactoring of selectors.nim. * Some fixes to move further. * Last Linux fixes. * Fixes for asyncloop to use 2nd version of selectors api. * Add osutils.nim. * Some fixes. * Hardening resumeRead(), resumeWrite() and consumers. Add ESRCH handling. Introduce no-exception fromPipe2. * Make Windows part exception-free and fix zombie race issue. * createStreamServer() fixes. * Upgrade asyncproc to use non-exception primitives. Fix ioselectors_kqueue to use closeFd(). * Refactor accept() and acceptLoop() to be exception free. * Deprecated some `result` usage. Last fixes to make stream.nim exception free. Use closeFd(). Refactor some loops to use handleEintr(). * Fix connect() forgot to unregister socket on error. * All createAsyncSocket() sockets should be closed with unregisterAndCloseFd(). * Attempt to fix posix bug with incomplete output. * Change number of runs in big test. * Refactoring pipes creation. Attempt to fix "write error: Resource temporarily unavailable". * Fix waitForExit(duration) code. Fix test exit code. * Fix Windows missing SIGTERM. * Fix mistype. * Fix compilation error. * Attempt to fix Nim 1.6 issue. * Eliminate Nim's WideCString usage to avoid any problems in future. * Deprecate posix usage in osnet. * Eliminate unused imports. * Some debugging statements for investigation. * Remove all the debugging statements. * Remove testhelpers in favor of unittest2/asynctests. * Fix flaky test. * Make test more resilient to timings. * Add memory size dump to CI. * Refactor some blocks to use valueOr. Make waitForExit to use kill() instead of terminate(). * Remove memory dumps. * Fix peekProcessExitCode() blocks issue. * Fix Windows issue. * Add some debugging echoes. * Fix compilation error. * Add some debugging echoes. * Add more helpers. * Fix compilation error. * waitForExit() is primary suspect. * Remove fast-path for waitForExit. Remove loop for reading signal events. * Remove all debugging echoes. * Return one debugging echo. * Fix futures tests. * Add test for multiple processes waiting to attempt stress test. * Refactor ioselectors_epoll for better signalfd and process monitoring. Add more race condition fixes to waitForExit. Fix some Nim 1.6 warnings. * Fix after rebase issues and warnings. * Fix style issues. Fix different Nim versions issues. Workaround `signalfd` style issues. * Add one more Linux signals workaround. Add one more multiple processes test. * Windows fixes. * Remove unixPlatform define. Fix WSAECONNABORTED for devel. * Temporarily disable rate limit tests. Fix more devel issues. * Deprecate `hasThreadSupport` for ioselectors_kqueue. Fix verifySelectParams issue. Add exit codes test for multiple processes. Fix osnet PtrToCstringConv warning. * ioselectors_kqueue refactoring. * Initial commit. * Fix 1.2-1.4 compilation issue. * Fix unused warning for testCtrlC() test. * Post-rebase fixes. * Restore master files. * More fixes. * Remove duplicates. * Fix style mistake. * Add more flexible pragmas.
2023-06-01 22:53:20 +00:00
export Domain, Protocol, SockType, results, osutils
2018-05-16 08:22:34 +00:00
const
asyncInvalidSocket* = AsyncFD(osdefs.INVALID_SOCKET)
asyncInvalidPipe* = asyncInvalidSocket
2018-05-16 08:22:34 +00:00
proc setSocketBlocking*(s: SocketHandle, blocking: bool): bool =
## Sets blocking mode on socket.
2022-01-04 17:00:17 +00:00
when defined(windows) or defined(nimdoc):
2018-05-16 08:22:34 +00:00
var mode = clong(ord(not blocking))
if osdefs.ioctlsocket(s, osdefs.FIONBIO, addr(mode)) == -1:
false
else:
true
2018-05-16 08:22:34 +00:00
else:
let x: int = osdefs.fcntl(s, osdefs.F_GETFL, 0)
2018-05-16 08:22:34 +00:00
if x == -1:
false
2018-05-16 08:22:34 +00:00
else:
let mode =
if blocking: x and not osdefs.O_NONBLOCK else: x or osdefs.O_NONBLOCK
if osdefs.fcntl(s, osdefs.F_SETFL, mode) == -1:
false
else:
true
2018-05-16 08:22:34 +00:00
2018-05-21 21:52:57 +00:00
proc setSockOpt*(socket: AsyncFD, level, optname, optval: int): bool =
2018-05-16 08:22:34 +00:00
## `setsockopt()` for integer options.
## Returns ``true`` on success, ``false`` on error.
var value = cint(optval)
osdefs.setsockopt(SocketHandle(socket), cint(level), cint(optname),
addr(value), SockLen(sizeof(value))) >= cint(0)
proc setSockOpt*(socket: AsyncFD, level, optname: int, value: pointer,
valuelen: int): bool =
## `setsockopt()` for custom options (pointer and length).
## Returns ``true`` on success, ``false`` on error.
osdefs.setsockopt(SocketHandle(socket), cint(level), cint(optname), value,
SockLen(valuelen)) >= cint(0)
2018-05-21 21:52:57 +00:00
proc getSockOpt*(socket: AsyncFD, level, optname: int, value: var int): bool =
2018-05-16 08:22:34 +00:00
## `getsockopt()` for integer options.
## Returns ``true`` on success, ``false`` on error.
2018-05-16 08:22:34 +00:00
var res: cint
2019-05-10 05:27:05 +00:00
var size = SockLen(sizeof(res))
if osdefs.getsockopt(SocketHandle(socket), cint(level), cint(optname),
addr(res), addr(size)) >= cint(0):
2019-05-10 05:27:05 +00:00
value = int(res)
true
else:
false
2018-05-16 08:22:34 +00:00
proc getSockOpt*(socket: AsyncFD, level, optname: int, value: pointer,
valuelen: var int): bool =
## `getsockopt()` for custom options (pointer and length).
## Returns ``true`` on success, ``false`` on error.
osdefs.getsockopt(SocketHandle(socket), cint(level), cint(optname),
value, cast[ptr SockLen](addr valuelen)) >= cint(0)
2018-05-21 21:52:57 +00:00
proc getSocketError*(socket: AsyncFD, err: var int): bool =
## Recover error code associated with socket handle ``socket``.
getSockOpt(socket, cint(osdefs.SOL_SOCKET), cint(osdefs.SO_ERROR), err)
proc createAsyncSocket2*(domain: Domain, sockType: SockType,
protocol: Protocol,
inherit = true): Result[AsyncFD, OSErrorCode] =
## Creates new asynchronous socket.
when defined(windows):
let flags =
if inherit:
osdefs.WSA_FLAG_OVERLAPPED
else:
osdefs.WSA_FLAG_OVERLAPPED or osdefs.WSA_FLAG_NO_HANDLE_INHERIT
let fd = wsaSocket(toInt(domain), toInt(sockType), toInt(protocol),
nil, GROUP(0), flags)
if fd == osdefs.INVALID_SOCKET:
return err(osLastError())
let bres = setDescriptorBlocking(fd, false)
if bres.isErr():
discard closeFd(fd)
return err(bres.error())
let res = register2(AsyncFD(fd))
if res.isErr():
discard closeFd(fd)
return err(res.error())
ok(AsyncFD(fd))
else:
when declared(SOCK_NONBLOCK) and declared(SOCK_CLOEXEC):
let socketType =
if inherit:
toInt(sockType) or osdefs.SOCK_NONBLOCK
else:
toInt(sockType) or osdefs.SOCK_NONBLOCK or osdefs.SOCK_CLOEXEC
let fd = osdefs.socket(toInt(domain), socketType, toInt(protocol))
if fd == -1:
return err(osLastError())
let res = register2(AsyncFD(fd))
if res.isErr():
discard closeFd(fd)
return err(res.error())
ok(AsyncFD(fd))
else:
let fd = osdefs.socket(toInt(domain), toInt(sockType), toInt(protocol))
if fd == -1:
return err(osLastError())
let bres = setDescriptorFlags(cint(fd), true, true)
if bres.isErr():
discard closeFd(fd)
return err(bres.error())
let res = register2(AsyncFD(fd))
if res.isErr():
discard closeFd(fd)
return err(bres.error())
ok(AsyncFD(fd))
proc wrapAsyncSocket2*(sock: cint|SocketHandle): Result[AsyncFD, OSErrorCode] =
## Wraps socket to asynchronous socket handle.
let fd =
when defined(windows):
sock
else:
when sock is cint: sock else: cint(sock)
? setDescriptorFlags(fd, true, true)
? register2(AsyncFD(fd))
ok(AsyncFD(fd))
2018-05-16 08:22:34 +00:00
proc createAsyncSocket*(domain: Domain, sockType: SockType,
protocol: Protocol,
inherit = true): AsyncFD =
2018-05-16 08:22:34 +00:00
## Creates new asynchronous socket.
## Returns ``asyncInvalidSocket`` on error.
createAsyncSocket2(domain, sockType, protocol, inherit).valueOr:
2018-05-16 08:22:34 +00:00
return asyncInvalidSocket
proc wrapAsyncSocket*(sock: cint|SocketHandle): AsyncFD {.
exception tracking (#166) * exception tracking This PR adds minimal exception tracking to chronos, moving the goalpost one step further. In particular, it becomes invalid to raise exceptions from `callSoon` callbacks: this is critical for writing correct error handling because there's no reasonable way that a user of chronos can possibly _reason_ about exceptions coming out of there: the event loop will be in an indeterminite state when the loop is executing an _random_ callback. As expected, there are several issues in the error handling of chronos: in particular, it will end up in an inconsistent internal state whenever the selector loop operations fail, because the internal state update functions are not written in an exception-safe way. This PR turns this into a Defect, which probably is not the optimal way of handling things - expect more work to be done here. Some API have no way of reporting back errors to callers - for example, when something fails in the accept loop, there's not much it can do, and no way to report it back to the user of the API - this has been fixed with the new accept flow - the old one should be deprecated. Finally, there is information loss in the API: in composite operations like `poll` and `waitFor` there's no way to differentiate internal errors from user-level errors originating from callbacks. * store `CatchableError` in future * annotate proc's with correct raises information * `selectors2` to avoid non-CatchableError IOSelectorsException * `$` should never raise * remove unnecessary gcsafe annotations * fix exceptions leaking out of timer waits * fix some imports * functions must signal raising the union of all exceptions across all platforms to enable cross-platform code * switch to unittest2 * add `selectors2` which supercedes the std library version and fixes several exception handling issues in there * fixes * docs, platform-independent eh specifiers for some functions * add feature flag for strict exception mode also bump version to 3.0.0 - _most_ existing code should be compatible with this version of exception handling but some things might need fixing - callbacks, existing raises specifications etc. * fix AsyncCheck for non-void T
2021-03-24 09:08:33 +00:00
raises: [Defect, CatchableError].} =
2018-05-21 21:52:57 +00:00
## Wraps socket to asynchronous socket handle.
2018-05-16 08:22:34 +00:00
## Return ``asyncInvalidSocket`` on error.
wrapAsyncSocket2(sock).valueOr:
2018-05-16 08:22:34 +00:00
return asyncInvalidSocket
proc getMaxOpenFiles2*(): Result[int, OSErrorCode] =
## Returns maximum file descriptor number that can be opened by this process.
##
## Note: On Windows its impossible to obtain such number, so getMaxOpenFiles()
## will return constant value of 16384. You can get more information on this
## link https://docs.microsoft.com/en-us/archive/blogs/markrussinovich/pushing-the-limits-of-windows-handles
2022-01-04 17:00:17 +00:00
when defined(windows) or defined(nimdoc):
ok(16384)
else:
var limits: RLimit
if osdefs.getrlimit(osdefs.RLIMIT_NOFILE, limits) != 0:
return err(osLastError())
ok(int(limits.rlim_cur))
proc setMaxOpenFiles2*(count: int): Result[void, OSErrorCode] =
## Set maximum file descriptor number that can be opened by this process.
##
## Note: On Windows its impossible to set this value, so it just a nop call.
2022-01-04 17:00:17 +00:00
when defined(windows) or defined(nimdoc):
ok()
else:
var limits: RLimit
if getrlimit(osdefs.RLIMIT_NOFILE, limits) != 0:
return err(osLastError())
limits.rlim_cur = count
if setrlimit(osdefs.RLIMIT_NOFILE, limits) != 0:
return err(osLastError())
ok()
proc getMaxOpenFiles*(): int {.raises: [Defect, OSError].} =
## Returns maximum file descriptor number that can be opened by this process.
##
## Note: On Windows its impossible to obtain such number, so getMaxOpenFiles()
## will return constant value of 16384. You can get more information on this
## link https://docs.microsoft.com/en-us/archive/blogs/markrussinovich/pushing-the-limits-of-windows-handles
let res = getMaxOpenFiles2()
if res.isErr():
raiseOSError(res.error())
res.get()
proc setMaxOpenFiles*(count: int) {.raises: [Defect, OSError].} =
## Set maximum file descriptor number that can be opened by this process.
##
## Note: On Windows its impossible to set this value, so it just a nop call.
let res = setMaxOpenFiles2(count)
if res.isErr():
raiseOSError(res.error())
proc getInheritable*(fd: AsyncFD): Result[bool, OSErrorCode] =
## Returns ``true`` if ``fd`` is inheritable handle.
when defined(windows):
var flags = 0'u32
if getHandleInformation(HANDLE(fd), flags) == FALSE:
return err(osLastError())
ok((flags and HANDLE_FLAG_INHERIT) == HANDLE_FLAG_INHERIT)
else:
let flags = osdefs.fcntl(cint(fd), osdefs.F_GETFD)
if flags == -1:
return err(osLastError())
ok((flags and osdefs.FD_CLOEXEC) == osdefs.FD_CLOEXEC)
proc createAsyncPipe*(): tuple[read: AsyncFD, write: AsyncFD] =
## Create new asynchronouse pipe.
## Returns tuple of read pipe handle and write pipe handle``asyncInvalidPipe``
## on error.
let res = createOsPipe(AsyncDescriptorDefault, AsyncDescriptorDefault)
if res.isErr():
(read: asyncInvalidPipe, write: asyncInvalidPipe)
else:
let pipes = res.get()
(read: AsyncFD(pipes.read), write: AsyncFD(pipes.write))