Commit Graph

69 Commits

Author SHA1 Message Date
Eugene Kabanov 148ddf49c2
Asyncproc (Part 3/3) (#374)
* Initial commit.

* Some Linux fixes.

* Address review comments on Windows.

* Fix issues on Linux.

* Fix 1.2 issue and Windows warnings.

* Fix posix compilation issues.
2023-05-23 12:39:35 +02:00
Eugene Kabanov 956ae5af55
Fix import poison regression. (#388)
* Initial commit.

* One more linux fix.

* Eliminate posix from asyncloop.

* Fix Linux warnings.

* Fix MacOS issues.

* Fix BSD issues.

* Fix comma.

* Fix Windows issues.
2023-05-15 19:45:26 +03:00
Eugene Kabanov 3118f8c1b2
Fix cast[pointer] issues on nim-devel. (#381)
* Fix cast[pointer] issues on nim-devel.

* More cast[T] fixes.

* Fix compilation error.

* Add oserrno.nim
Further fixes for Windows and Linux.

* MacOS fixes.

* More Windows fixes and attempt to fix 1.2, 1.4 branches.

* Implicitly import/export oserrno.

* Replace oserrno with osdefs.

* Return back oserrno.

* epoll to oserrno.

* datagram/stream to oserrno.

* common to oserrno.

* test to oserrno.
2023-04-30 08:20:08 +02:00
Jacek Sieka 229de5f842
Compile-time configuration (#371)
This PR moves all compile-time configuration to a single module,
simplifying documentation and access to these features.

Upcomfing features may be enabled either individually, or through a new
`chronosPreviewV4` catch-all designed to allow code to be prepared for
increased strictness in future chronos releases.

`-d:chronosDebug` may be used to enable the existing debugging helpers
together.
2023-03-31 07:35:04 +02:00
Eugene Kabanov 1394c9e049
IOSelectors refactoring to properly support signals and processes. (AsyncProc 2) (#366)
* ioselectors_epoll() refactoring.

* ioselectors_kqueue() refactoring.

* ioselectors_poll() initial refactor.

* Remove `s.count` because it inconsistent and not used in `chronos`.

* Remove Windows version of select() engine.

* Add ability to switch event queue engine via `asyncEventEngine` command line option.

* Make it possible to switch between engines.

* Fix epoll regression.

* Fix poll() engine issues.

* Address review comments.

* Add proper trick.

* Address review comments.

* Bump version to 3.1.0.
2023-03-24 17:52:55 +02:00
Eugene Kabanov 0f70a6b8ee
OS definitions and exceptions changes. (AsyncProc part 1). (#361)
* Initial commit.

* Finalize Windows part.

* Finalize Posix part.

* Fix style issues.

* Move osnet declarations to osdefs.

* Move osnet declarations to osdefs 2.

* Fix Nim 1.2 issues.

* Fix 1.2 compilation issues 2.

* Address review comments.
Fix SignalFdInfo issue.
2023-02-21 11:48:36 +01:00
Tanguy a50ac4f642
Documentation (#311)
* Fix & update asyncloop documentation

* Add cancellation documentation

* Add documentation generation workflow

* pin nim version for doc gen

* update readme

* fix example

* Simplify examples
2023-02-16 21:27:31 +01:00
Jacek Sieka c65cc4c136
`async` `proc` types (#346)
Allow `type = proc(...) {.async.}` for pre-declaring async callback
types - in this case, the macro applies the correct transformation to
the return type and adds the appropriate exception / gcsafe annotations
2023-01-18 15:54:39 +01:00
Eugene Kabanov 6525f4ce1d
Fix some warnings. (#310)
* Fix some warnings messages.

* More warning fixes.

* Address review comments.
Fix MacOS issues.

* More warning fixes.

* More Windows specific fixes.

* Fix macos and windows warnings.

* Fix warnings in timer.nim
Refactor to remove `result`.
Improve performance and behavior of timer to string procedure.
Add tests.
2022-11-02 08:09:15 +01:00
Eugene Kabanov be2352027e
Fix nested waitFor (IndexError defect crash) bug. (#309)
Add IndexError crash test for Linux/MacOS.
Add Sentinel version of fix.
Add GetQueuedCompletionStatusEx() support for Windows, which allows to capture more then one event in single `poll()` call.
2022-09-16 22:34:18 +02:00
Jacek Sieka 746832384a
`Option` -> `Opt` (#307) 2022-09-05 11:08:52 +00:00
Miran 93800879fd
make it more Nim 1.4+ compatible (#296) 2022-08-06 13:56:06 +03:00
Eugene Kabanov 377e197417
waitSignal() helper and tests for it. (#289)
* Add waitForSignal() implementation and tests.

* Fix compilation issues.

* Fix mistype.

* Rename to waitSignal().

* Fix Windows compilation issue.

* Re-export posix signals.

* Remove signal handler on continuation too.
2022-06-29 00:53:09 +03:00
Jacek Sieka bb4c3298f5
fix crash after reading from fd > 1024 (#267)
The socket selector holds a `seq` of per-descriptor data. When a reader
is registered, a pointer to a seq item is stored - when the `seq` grows,
this pointer becomes dangling and causes crashes like
https://github.com/status-im/nimbus-eth2/issues/3521.

It turns out that there already exist two mechanisms for passing user
data around - this PR simply removes one of them, saving on memory usage
and removing the need to store pointers to the `seq` data that become
dangling on resize.
2022-04-11 11:56:30 +02:00
Ștefan Talpalaru 17fed89c99
style fixes (#253)
* style fixes

and disable the cron CI job
2022-01-04 23:14:30 +01:00
Tanguy d926415f42
Fixes for nimdoc (#252) 2022-01-04 18:00:17 +01:00
Zahary Karadjov 7ca85ddadc
Fix compilation errors with --styleCheck:usages --styleCheck:error 2021-12-08 16:58:24 +02:00
Ștefan Talpalaru 4aaaa3379a
Nim-1.6 fixes (#239) 2021-11-20 01:30:23 +01:00
Eugene Kabanov 3bc0bc36f7
Change Future identifier type from `int` to `uint`. (#228)
* Change Future identifier type from `int` to `uint64`.

* Fix compilation error and tests.

* Add integer overflow test, to avoid confusion with next Nim versions, we expect that unsigned integers do not raise any exceptions if overflow happens.

* Switch from `uint64` to `uint` type.

* Add `uint` overflow tests.
2021-10-21 17:22:11 +03:00
Eugene Kabanov d270dba8a3
Fix unused warnings, result, asyncCheck and 80 cpl (#185)
* Fix sources to follow 80 characters per line.
Fix unused compilation warnings.
Refactor (remove result) handles.nim.
Fix tests to use asyncSpawn instead of asyncCheck.

* Fix handles for Unix platforms.
2021-05-07 23:52:24 +03:00
Jacek Sieka 895fc53193
add explicit gcsafe annotations (fixes #174) (#175)
Seems like a variation of https://github.com/nim-lang/Nim/issues/17369
2021-04-03 10:05:08 +02:00
markspanbroek e5d3ecdd01
Ensures that asyncmacro2 does not raise exceptions (#172)
* Ensures that asyncmacro2 does not raise exceptions

* Remove unnecessary include
2021-03-30 16:48:20 +02:00
Kim De Mey c206d2bc19
Add raises annotation to withTimeout callbacks (#168) 2021-03-25 14:20:26 +01:00
Jacek Sieka 4abd7a5645
exception tracking (#166)
* exception tracking

This PR adds minimal exception tracking to chronos, moving the goalpost
one step further.

In particular, it becomes invalid to raise exceptions from `callSoon`
callbacks: this is critical for writing correct error handling because
there's no reasonable way that a user of chronos can possibly _reason_
about exceptions coming out of there: the event loop will be in an
indeterminite state when the loop is executing an _random_ callback.

As expected, there are several issues in the error handling of chronos:
in particular, it will end up in an inconsistent internal state whenever
the selector loop operations fail, because the internal state update
functions are not written in an exception-safe way. This PR turns this
into a Defect, which probably is not the optimal way of handling things
- expect more work to be done here.

Some API have no way of reporting back errors to callers - for example,
when something fails in the accept loop, there's not much it can do, and
no way to report it back to the user of the API - this has been fixed
with the new accept flow - the old one should be deprecated.

Finally, there is information loss in the API: in composite operations
like `poll` and `waitFor` there's no way to differentiate internal
errors from user-level errors originating from callbacks.

* store `CatchableError` in future
* annotate proc's with correct raises information
* `selectors2` to avoid non-CatchableError IOSelectorsException
* `$` should never raise
* remove unnecessary gcsafe annotations
* fix exceptions leaking out of timer waits
* fix some imports
* functions must signal raising the union of all exceptions across all
platforms to enable cross-platform code
* switch to unittest2
* add `selectors2` which supercedes the std library version and fixes
several exception handling issues in there

* fixes

* docs, platform-independent eh specifiers for some functions

* add feature flag for strict exception mode

also bump version to 3.0.0 - _most_ existing code should be compatible
with this version of exception handling but some things might need
fixing - callbacks, existing raises specifications etc.

* fix AsyncCheck for non-void T
2021-03-24 10:08:33 +01:00
cheatfate 13eddf382d Simplification and fixes for TLSStream state machine. 2021-02-18 22:16:04 +02:00
cheatfate e8d2a3ca0a Attempt #5. 2021-02-18 22:16:04 +02:00
cheatfate 49fd70f504 Attempt #4. 2021-02-18 22:16:04 +02:00
cheatfate cf42a5455e Investigation of Linux freezes #2. 2021-02-18 22:16:04 +02:00
Eugene Kabanov 491213dfa0
Add callIdle() primitive. (#148)
* Add callIdle primitive.

* Make single idle callback to be processed by single poll() step.
Add idleAsync() primitive to allow wait for "idle" time.
Refactor some `result` usage.
2021-01-19 13:48:39 +01:00
Jacek Sieka 9a420c6b05
work around overload resolution issues (#146)
it seems that due to a naming conflict from asyncdispatch, callSoon is
deduced to raise exceptions even if it doesn't in modules that import
both, even indirectly - this patch randomly works around the issue with
some more overloads
2021-01-11 22:15:21 +01:00
Eugene Kabanov 0933feaa35
Fix callSoon to raise Defect only. (#145)
* Initial commit.

* Move pragmas to forward declaration.
Add raises to callSoon declaration.

* Fix for IOSelectorsException.
2021-01-11 18:15:23 +01:00
cheatfate 493cb1dbfd Fix behavior of wait() and withTimeout() calls to cancel and wait for result of cancelled Future[T].
Add tests.
2020-11-17 19:28:52 +02:00
Eugene Kabanov b5915ecd29
Small fixes (#134)
* Fix cancellation behavior.
Fix some compilation warnings.

* Fix cancelAndWait() and add proper documentation.

* Add completed(Future) and done(Future) calls to check if Future[T] completed without an error.

* Add stepsAsync() and tests.

* Fix comments.

* Fix new primitive comment, to avoid usage for task switches.
2020-11-13 14:22:58 +02:00
Jacek Sieka 1ffd1cd3dc
release callback memory early (#130)
* release callback memory early

this fixes a memory leak where a deleted callback may keep references
alive until the future is finished.

In particular, when using helpers like `or` which try to remove
themselves from the callback list when a dependent future is completed,
create a reference chain between all futures in the expression - in the
pathological case where one of the futures is completes only rarely (for
example a timeout or a cancellation task), the buildup will be
significant.

* Removing unnecessary asserts, and place comments instead.
2020-09-15 10:55:43 +03:00
Zahary Karadjov e6d50b7736 Revert "some metrics for monitoring futures (#85)"
This reverts commit e45ef32b5b.

Metrics implemented this way, with a lock inside the otherwise tight
event loop are not consistent with the chronos architecture that for
good or bad uses thread local variables to avoid them - the solution
does not have rough consensus behind it, and other avenues should be
explored for this generally useful functionality.
2020-08-16 01:48:25 +03:00
Zahary Karadjov 03f4a26829 Revert "yieldAsync() (#120)"
This reverts commit 284d677815.

There is no rough consensus for how this should be implemented and
whether it's needed indeed - a use case that cannot be solved with
a queue or a lock/event should be identified before pursuing this
functionality.
2020-08-16 01:48:25 +03:00
Ștefan Talpalaru 284d677815
yieldAsync() (#120) 2020-08-10 15:31:21 +02:00
Ștefan Talpalaru e45ef32b5b
some metrics for monitoring futures (#85) 2020-08-06 19:30:53 +02:00
Eugene Kabanov ce6e7d17b1
Make Future tracking and stack traces optional (#108)
* Make Future tracking optional via -d:chronosDutureTracking compilation flag.
* Stack traces is now optional, use -d:chronosStackTraces.
* Fix mistypes and add test for chronosStackTrace option.
2020-07-08 19:48:01 +03:00
Eugene Kabanov 5629b3c41f
[WIP] Zero-cost unattended Future[T] tracking mechanism. (#106)
* Zero-cost unattended Future[T] tracking mechanism with tests and tracking of test suite.
2020-07-06 09:33:13 +03:00
Eugene Kabanov 02b8da986b
Add accept() call (#103)
* Add accept() call and tests.
* Fix rare fd leaks on Windows.
* Fix compilation warnings.
* Add fd leak test.
* Bump version to 2.4.0.
2020-06-24 11:21:52 +03:00
Eugene Kabanov bedd1ded5e
Properly fix cancellation race and not introduce FD leaks. (#102) 2020-06-05 19:11:51 +03:00
cheatfate 74700fdcab Fix bug cancellation handlers not called in wait() and withTimeout().
Fix double completion bug because of callback race.
Fix deprecation warnings.
Rename some internal procedures.
Bump version to 2.3.5.
2020-01-27 22:32:08 +02:00
Dmitriy Ryajov 1c176c173d fix: restore `removeCallback` 2020-01-08 19:32:53 -06:00
Dmitriy Ryajov 0d84f273c9 make `clearTimer` inline 2020-01-08 11:23:01 -06:00
Dmitriy Ryajov 96e0206c27 reverting AsyncCallback to object 2020-01-08 11:06:56 -06:00
Dmitriy Ryajov 78953f8fc3 consolidating timers loop 2020-01-08 10:03:34 -06:00
Dmitriy Ryajov d8dd09a358 don't loop over timers heap 2020-01-07 19:06:27 -06:00
Dmitriy Ryajov d979770263 clear timers under windown as well 2020-01-07 10:22:38 -06:00
Dmitriy Ryajov b9c534724c don't scan timers heap on each remove 2020-01-06 23:26:18 -06:00