This PR moves all compile-time configuration to a single module,
simplifying documentation and access to these features.
Upcomfing features may be enabled either individually, or through a new
`chronosPreviewV4` catch-all designed to allow code to be prepared for
increased strictness in future chronos releases.
`-d:chronosDebug` may be used to enable the existing debugging helpers
together.
* ioselectors_epoll() refactoring.
* ioselectors_kqueue() refactoring.
* ioselectors_poll() initial refactor.
* Remove `s.count` because it inconsistent and not used in `chronos`.
* Remove Windows version of select() engine.
* Add ability to switch event queue engine via `asyncEventEngine` command line option.
* Make it possible to switch between engines.
* Fix epoll regression.
* Fix poll() engine issues.
* Address review comments.
* Add proper trick.
* Address review comments.
* Bump version to 3.1.0.
* Allow passing in trustAnchors to newTLSClientAsyncStream
* Store X509TrustAnchors in a memory-secure, but memory-friendly way
* Remove forgotten import
* Change some waitFor to await in tests; add comment about trustAnchors reuse
* Remove use of result in newTrustAnchorStore
Remove `result` usage.
Remove cast[T] usage.
Add more helpers for TransportAddress checks.
Add isGlobal() helper to check if TransportAddress is globally accessed IP address.
Add tests.
Add more tests for sub.
Add TransportAddress.toHex().
Allow `type = proc(...) {.async.}` for pre-declaring async callback
types - in this case, the macro applies the correct transformation to
the return type and adds the appropriate exception / gcsafe annotations
* Address #329.
* Refactor AsyncStreamReader/Writer `state` helper functions.
Fix some compilation warnings in `debugutils`.
Add multiple writes into test to check consistency.
* Fix Linux issue.
Fix warnings.
* Fix some warnings messages.
* More warning fixes.
* Address review comments.
Fix MacOS issues.
* More warning fixes.
* More Windows specific fixes.
* Fix macos and windows warnings.
* Fix warnings in timer.nim
Refactor to remove `result`.
Improve performance and behavior of timer to string procedure.
Add tests.
Add IndexError crash test for Linux/MacOS.
Add Sentinel version of fix.
Add GetQueuedCompletionStatusEx() support for Windows, which allows to capture more then one event in single `poll()` call.
* AsyncEventQueue (AsyncEventBus replacement) initial commit.
* Add closeWait() and remove assertion test.
* Fix "possible" memory leak.
* Add limits to AsyncEventQueue[T].
Add tests for AsyncEventQueue[T] limits.
Rebase AsyncSync errors to be children of AsyncError.
* Adopt AsyncEventQueue to garbage collect events on emit() too.
Add test from review comment.
* Remove AsyncEventBus test suite.
* Remove unneeded [T] usage.
* Optimize memory usage in 1m test.
* Lower number of events in test from 1m to 100k.
* Deprecate AsyncEventBus.
Add some GC debugging for tests.
* Fix mistype.
* One more attempt to fix crash.
* Add some echo debugging.
* More echo debugging.
* More closer debug echoes.
* Attempt to workaround crash place.
* More debugging echoes in exception handlers.
* Convert suspected test into async procedure.
* Make multiple consumers test async.
* Remove GC debugging.
* Disable added tests.
* Disable AsyncEventQueue tests to confirm that this is an issue.
* Enable all the tests.
* Tentative fix for the HTTP client connection state assertion failures
I've traced the problem to a HTTP connection being closed while there
are outstanding requests that still go through the motions (the crash
occurs when a requests reaches its `finish` processing step, but it
was already put in a Closed state by the connection that owns it).
* Use distinct error exception instead of cancellation error.
Co-authored-by: cheatfate <eugene.kabanov@status.im>
When calling the HTTP `send` / `open` functions, `acquireConnection` is
called to obtain a connection in state `Ready`. In the next code block,
the connection state is advanced to `RequestHeadersSending`. However,
returning from chronos `async` procs yields control, similar to `await`.
This means that a connection may be added to the pool in `Ready` state,
and then a different `acquireConnection` call may obtain a second ref.
Introducing a new `Acquired` state ensures consistency in this case.
No tests added due to this being scheduler dependent; ran manual tests
by adding a `doAssert item.state != HttpClientConnectionState.Acquired`
between `if not(isNil(conn)):` and `return conn`. Eventually, the assert
got hit after several hours of repeated tests, confirming the edge case
to be solved by applying this fix.
Not sure if it is by design that returning from an `async` proc yields.
Even if it's not, this should solve current HTTP issues in nimbus-eth2.
The socket selector holds a `seq` of per-descriptor data. When a reader
is registered, a pointer to a seq item is stored - when the `seq` grows,
this pointer becomes dangling and causes crashes like
https://github.com/status-im/nimbus-eth2/issues/3521.
It turns out that there already exist two mechanisms for passing user
data around - this PR simply removes one of them, saving on memory usage
and removing the need to store pointers to the `seq` data that become
dangling on resize.
`nim doc --project --index:on chronos.nim` was failing with the following error:
/path/to/nim-chronos/chronos/transports/common.nim(519, 21) Error: '*' expected
...which in turned caused errors in all files that import this one.
Now the entire docs can be built successfully, with only a few warnings.
Fixes#186.
Fix of the following bug:
In case of multiple accept headers with same preference
`preferredContentType` used to select the first match within content types
provided by the application. For example, if user specifies accept headers
`application/octet-stream, application/json` and application provides
`application/json, application/octet-stream`, `application/octet-stream`
will be returned, and that is a bug.
Now the procedure returns the most suitable match according
to the application preferences.
When `write` is called on a `StreamTransport`, the current sequence of
operations is:
* copy data to queue
* register for "write" event notification
* return unfinished future to `write` caller
* wait for "write" notification (in `poll`)
* perform one `send`
* wait for notification again if there's more data to write
* complete the future
In this PR, we introduce a fast path for writing:
* If the queue is empty, try to send as much data as possible
* If all data is sent, return completed future without `poll` round
* If there's more data to send than can be sent in one go, add the
rest to queue
* If the queue is not empty, enqueue as above
* When notified that write is possible, keep writing until OS buffer is
full before waiting for event again
The fast path provides significant performance benefits when there are
many small writes, such as when sending gossip to many peers, by
avoiding the poll loop and data copy on each send.
Also fixes an issue where the socket would not be removed from the
writer set if there were pending writes on close.
* Add more accept() call error handlers.
Fix issue when HTTP server silently stops accepting new connections.
Remove unneeded MacOS syscall to disable SIG_PIPE on socket, we already mask this signal in process.
* User `if` instead `case` because constants are actually variables.
* Fix mistypes.
* Do not use case for posix constants.
* Fix Linux compilation error.
* Change Future identifier type from `int` to `uint64`.
* Fix compilation error and tests.
* Add integer overflow test, to avoid confusion with next Nim versions, we expect that unsigned integers do not raise any exceptions if overflow happens.
* Switch from `uint64` to `uint` type.
* Add `uint` overflow tests.
Fix async streams issue with replacing state.
Add `closing` states to HTTP's server connection, request and bodyrw.
Fix Http server cancellation leaks.
* Initial commit.
* Some refactoring.
* Allow boundstream to accept uint64.
Fix httpserver and asyncstream tests to follow new uint64 requirement.
* send() and getBodyBytes() implementations.
* Add closeWait for response and request.
Refactor finish/close flow.
* Changes in state machine
Add first test.
* Missing test file.
* Fixed tests
Add http leaking trackers and tests.
* Some fixes in multipart.
Fix automatic Content-Length header for requests with body.
Fix getBodyBytes() assertions.
Merging tests to main suite.
* Post rebase fixes.
* Fix tests big message generation.
* Fix response state management and leaks for getBodyXXX() procedures.
* Add redirection support to client and server.
Add fetch(url) procedure with redirection support.
Add tests for redirection.
* Restore functionality of zero-sized bounded reader/writer streams.
Adding tests for it.
* run build_nim.sh unconditionally
Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
* Refactor common.nim to remove `result` usage.
Fix comparison of TransportAddress issue.
Add resolveTAddress procedures for both IPv4 and IPv6 addresses.
Fix tests.
* Bump version to 3.0.2.
* exception tracking
This PR adds minimal exception tracking to chronos, moving the goalpost
one step further.
In particular, it becomes invalid to raise exceptions from `callSoon`
callbacks: this is critical for writing correct error handling because
there's no reasonable way that a user of chronos can possibly _reason_
about exceptions coming out of there: the event loop will be in an
indeterminite state when the loop is executing an _random_ callback.
As expected, there are several issues in the error handling of chronos:
in particular, it will end up in an inconsistent internal state whenever
the selector loop operations fail, because the internal state update
functions are not written in an exception-safe way. This PR turns this
into a Defect, which probably is not the optimal way of handling things
- expect more work to be done here.
Some API have no way of reporting back errors to callers - for example,
when something fails in the accept loop, there's not much it can do, and
no way to report it back to the user of the API - this has been fixed
with the new accept flow - the old one should be deprecated.
Finally, there is information loss in the API: in composite operations
like `poll` and `waitFor` there's no way to differentiate internal
errors from user-level errors originating from callbacks.
* store `CatchableError` in future
* annotate proc's with correct raises information
* `selectors2` to avoid non-CatchableError IOSelectorsException
* `$` should never raise
* remove unnecessary gcsafe annotations
* fix exceptions leaking out of timer waits
* fix some imports
* functions must signal raising the union of all exceptions across all
platforms to enable cross-platform code
* switch to unittest2
* add `selectors2` which supercedes the std library version and fixes
several exception handling issues in there
* fixes
* docs, platform-independent eh specifiers for some functions
* add feature flag for strict exception mode
also bump version to 3.0.0 - _most_ existing code should be compatible
with this version of exception handling but some things might need
fixing - callbacks, existing raises specifications etc.
* fix AsyncCheck for non-void T
1. Implement all read() primitives using readLoop() like it was done in streams.
2. Fix readLine() bug.
3. Add readMessage() primitive.
4. Fixing exception hierarchy, handling code and simplification of (break/continue + exception).
5. Fix TLSStream closure procedure.
6. Add BoundedStream stream and tests.
7. Remove `result` usage from the code.
IPV6_MULTICAST_HOPS already declared in posix.nim while IP_MULTICAST_TTL
has a value that will stay constant on linux/x86_64 - better codegen and
nlvm compat like this.
Jacek Sieka
Eugene Kabanov
Etan Kissling
Matt Haggard
Tanguy
Bung
Miran
Zahary Karadjov
Derek 呆
Thomas Jost
Emil
Ștefan Talpalaru
Andreas Rumpf
Michael Bradley
Johann Bauer
markspanbroek
Kim De Mey