Commit Graph

324 Commits

Author SHA1 Message Date
Jacek Sieka 157ca4fea5
Support implicit returns (#401)
* based on https://github.com/nim-lang/Nim/pull/21898
* also fixes generic Future[T] where T ends up being `void`
2023-06-05 13:02:13 +02:00
Eugene Kabanov 6e80f16e0c
Fix recently introduced imports poisoning. (#400)
* Fix import poison introduced.

* Move export to handles, and annotate it.
2023-06-03 19:36:06 +03:00
Eugene Kabanov 02fda01bf2
Add facility to emulate signals on Windows. (#357)
* Initial Windows asyncproc part.
Deprecate usage of net/nativesockets in handles, asyncloop.
Introduce primitives with inheritable flag.

* Some posix fixes.

* More duplicate fixes.

* Fix AsyncProcessOption.EchoCommand option.
Remove isMainModule code.
Fix execCommand to use AsyncProcessOption.EvalCommand.

* Initial asyncproc tests commit.

* Some Posix fixes.

* Fix Posix crash.

* Add testsuite.
Fix osnet.nim to use proper declarations.
Fix timer.nim to use proper declarations.
Add Windows environment handling procedures.
Fix createAsyncPipe.
Add leaks tracking for AsyncProcessRef.

* Fix O_CLOEXEC constant value.

* Add file descriptors leak test.

* Remove commented code.
Refactor exceptions.
Fix compilation warnings.

* No exception ioselectors_kqueue initial commit.

* Some BSD fixes.
Linux refactoring of selectors.nim.

* Some fixes to move further.

* Last Linux fixes.

* Fixes for asyncloop to use 2nd version of selectors api.

* Add osutils.nim.

* Some fixes.

* Hardening resumeRead(), resumeWrite() and consumers.
Add ESRCH handling.
Introduce no-exception fromPipe2.

* Make Windows part exception-free and fix zombie race issue.

* createStreamServer() fixes.

* Upgrade asyncproc to use non-exception primitives.
Fix ioselectors_kqueue to use closeFd().

* Refactor accept() and acceptLoop() to be exception free.

* Deprecated some `result` usage.
Last fixes to make stream.nim exception free.
Use closeFd().
Refactor some loops to use handleEintr().

* Fix connect() forgot to unregister socket on error.

* All createAsyncSocket() sockets should be closed with unregisterAndCloseFd().

* Attempt to fix posix bug with incomplete output.

* Change number of runs in big test.

* Refactoring pipes creation. Attempt to fix "write error: Resource temporarily unavailable".

* Fix waitForExit(duration) code.
Fix test exit code.

* Fix Windows missing SIGTERM.

* Fix mistype.

* Fix compilation error.

* Attempt to fix Nim 1.6 issue.

* Eliminate Nim's WideCString usage to avoid any problems in future.

* Deprecate posix usage in osnet.

* Eliminate unused imports.

* Some debugging statements for investigation.

* Remove all the debugging statements.

* Remove testhelpers in favor of unittest2/asynctests.

* Fix flaky test.

* Make test more resilient to timings.

* Add memory size dump to CI.

* Refactor some blocks to use valueOr.
Make waitForExit to use kill() instead of terminate().

* Remove memory dumps.

* Fix peekProcessExitCode() blocks issue.

* Fix Windows issue.

* Add some debugging echoes.

* Fix compilation error.

* Add some debugging echoes.

* Add more helpers.

* Fix compilation error.

* waitForExit() is primary suspect.

* Remove fast-path for waitForExit.
Remove loop for reading signal events.

* Remove all debugging echoes.

* Return one debugging echo.

* Fix futures tests.

* Add test for multiple processes waiting to attempt stress test.

* Refactor ioselectors_epoll for better signalfd and process monitoring.
Add more race condition fixes to waitForExit.
Fix some Nim 1.6 warnings.

* Fix after rebase issues and warnings.

* Fix style issues.
Fix different Nim versions issues.
Workaround `signalfd` style issues.

* Add one more Linux signals workaround.
Add one more multiple processes test.

* Windows fixes.

* Remove unixPlatform define.
Fix WSAECONNABORTED for devel.

* Temporarily disable rate limit tests.
Fix more devel issues.

* Deprecate `hasThreadSupport` for ioselectors_kqueue.
Fix verifySelectParams issue.
Add exit codes test for multiple processes.
Fix osnet PtrToCstringConv warning.

* ioselectors_kqueue refactoring.

* Initial commit.

* Fix 1.2-1.4 compilation issue.

* Fix unused warning for testCtrlC() test.

* Post-rebase fixes.

* Restore master files.

* More fixes.

* Remove duplicates.

* Fix style mistake.

* Add more flexible pragmas.
2023-06-02 01:53:20 +03:00
Eugene Kabanov 315a27236c
Recover cancellation cleanup for AsyncEvent wait(). (#398)
* Recover cancellation cleanup for AsyncEvent wait().

* Address review comments.
2023-06-01 18:09:49 +03:00
gpicron 5be223e92c
allow reuse of port for UDP (#379) 2023-06-01 17:02:33 +03:00
Jacek Sieka e436f20b33
Memory cleanups (#395)
* Avoid `FutureContinue` bloat by moving type punning into iterator
(which is typed anyway)
* clear closure iterator from future when iteration is done also on
cancellation / exception
* remove some redundant local variables in `await`
* document `futureContinue` flow
2023-05-31 07:24:25 +02:00
Eugene Kabanov 6523f741a6
Assertion and bugfix. (#397) 2023-05-30 17:45:16 +02:00
Eugene Kabanov 36ab9c8e4a
Fix sigprocmask declaration missing when threads are disabled. (#396) 2023-05-30 07:57:57 +02:00
Jacek Sieka b65b85533a
Future cleanup (#393)
* FutureState.Finished -> FutureState.Completed (to avoid name clash
with `proc finished` which means not-pending)
* deprecate `done` - to avoid additional confusion over completed vs
finished
* remove ad leftovers in stack trace formatting
* avoid some generic bloat
* avoid unnecessary allocations in `race`/`one`
2023-05-25 17:31:35 +02:00
Jacek Sieka 2fa6df0880
asyncmacro: code cleanups (#392)
* prefer `let`
* prefer expressions
* renames
2023-05-23 19:45:12 +02:00
Eugene Kabanov 148ddf49c2
Asyncproc (Part 3/3) (#374)
* Initial commit.

* Some Linux fixes.

* Address review comments on Windows.

* Fix issues on Linux.

* Fix 1.2 issue and Windows warnings.

* Fix posix compilation issues.
2023-05-23 12:39:35 +02:00
Eugene Kabanov f748387462
Add HTTP client helper to read `text/event-stream` streaming response. (#375) 2023-05-19 17:25:22 +03:00
Bung 38cc233700
fix missing sigprocmask import (#390) 2023-05-19 13:43:33 +03:00
Eugene Kabanov 61d52b1ef8
Export SocketFlags for HTTP client. (#389) 2023-05-18 10:10:01 +03:00
Eugene Kabanov 956ae5af55
Fix import poison regression. (#388)
* Initial commit.

* One more linux fix.

* Eliminate posix from asyncloop.

* Fix Linux warnings.

* Fix MacOS issues.

* Fix BSD issues.

* Fix comma.

* Fix Windows issues.
2023-05-15 19:45:26 +03:00
Eugene Kabanov 8aa8ee8822
Add flag to completely disable HTTP/1.1 pipelining support. (#387)
* Add flag for HTTP/1.1 pipeline disable.

* Switch from NoHttp11Pipeline to Http11Pipeline.
2023-05-12 17:00:37 +03:00
Eugene Kabanov 8563c93673
Reexport osdefs where Sockaddr_storage was re-declared. (#385) 2023-05-12 07:11:05 +03:00
Eugene Kabanov ef94d75d61
Eliminate last 2 macos warnings. (#384) 2023-05-01 16:02:59 +03:00
Eugene Kabanov 5755d79b67
Eliminate warnings (#383)
* Eliminate warnings on Windows.

* Deprecated toHex() in osnet.
2023-05-01 00:46:15 +03:00
Eugene Kabanov 9ba418cd3b
Eliminate warnings on Windows. (#382) 2023-04-30 19:09:36 +02:00
Eugene Kabanov 3118f8c1b2
Fix cast[pointer] issues on nim-devel. (#381)
* Fix cast[pointer] issues on nim-devel.

* More cast[T] fixes.

* Fix compilation error.

* Add oserrno.nim
Further fixes for Windows and Linux.

* MacOS fixes.

* More Windows fixes and attempt to fix 1.2, 1.4 branches.

* Implicitly import/export oserrno.

* Replace oserrno with osdefs.

* Return back oserrno.

* epoll to oserrno.

* datagram/stream to oserrno.

* common to oserrno.

* test to oserrno.
2023-04-30 08:20:08 +02:00
Eugene Kabanov e05d2f8e96
Fix some compilation warnings and expose tracker names. (#376)
* Fix asyncsync compilation warnings.

* Fix tracker names should be public.
2023-04-08 19:34:57 +03:00
Tanguy ab5a8c2e0f
Add `localAddress` support to `stream.connect` (#362)
* Add `localAddress` support to `stream.connect`

* fix windows

* TransportAddress() instead of AnyAddress

* tweak flags

* Better flags

* try to workaround nim 1.2 issue

* Handle ReusePort in createStreamServer and improve tests

* Rename ClientFlags to SocketFlags

---------

Co-authored-by: Diego <diego@status.im>
2023-04-03 15:34:35 +03:00
Jacek Sieka 229de5f842
Compile-time configuration (#371)
This PR moves all compile-time configuration to a single module,
simplifying documentation and access to these features.

Upcomfing features may be enabled either individually, or through a new
`chronosPreviewV4` catch-all designed to allow code to be prepared for
increased strictness in future chronos releases.

`-d:chronosDebug` may be used to enable the existing debugging helpers
together.
2023-03-31 07:35:04 +02:00
Eugene Kabanov 1394c9e049
IOSelectors refactoring to properly support signals and processes. (AsyncProc 2) (#366)
* ioselectors_epoll() refactoring.

* ioselectors_kqueue() refactoring.

* ioselectors_poll() initial refactor.

* Remove `s.count` because it inconsistent and not used in `chronos`.

* Remove Windows version of select() engine.

* Add ability to switch event queue engine via `asyncEventEngine` command line option.

* Make it possible to switch between engines.

* Fix epoll regression.

* Fix poll() engine issues.

* Address review comments.

* Add proper trick.

* Address review comments.

* Bump version to 3.1.0.
2023-03-24 17:52:55 +02:00
Eugene Kabanov b0af576c7c
Address #320 issue. (#372) 2023-03-24 17:34:45 +02:00
Eugene Kabanov 0688d2ef8f
Add idle connection timeouts for HTTP client's connections pool. (#324)
* Add idle connection timeouts for HTTP client's connections pool.
Add timestamps and duration for both HTTP client requests/responses.
Add test.

* Add comments on `connectionFlag` decisions.

* Address review comments.
Adjust default idle connection timeout to 60 seconds.

* Increase timeout for test.

* Adjust timeout to lower value.

* Address review comments.
2023-03-21 15:10:35 +02:00
Etan Kissling 30c839bac7
fix `hasOverflow` for multiple streams (#370)
Instead of repeatedly checking stream 1, properly check all of them.
2023-03-16 11:46:24 +02:00
Eugene Kabanov f7835a192b
Fix osnet to not raise defect in ipnet. (#364)
* Debugging commit.

* Fix compilation issue.

* Remove debugging echoes.
2023-02-23 09:58:58 +02:00
Matt Haggard 1b69b5e808
Allow passing in trustAnchors to newTLSClientAsyncStream (#355)
* Allow passing in trustAnchors to newTLSClientAsyncStream

* Store X509TrustAnchors in a memory-secure, but memory-friendly way

* Remove forgotten import

* Change some waitFor to await in tests; add comment about trustAnchors reuse

* Remove use of result in newTrustAnchorStore
2023-02-21 20:38:53 +02:00
Eugene Kabanov 0f70a6b8ee
OS definitions and exceptions changes. (AsyncProc part 1). (#361)
* Initial commit.

* Finalize Windows part.

* Finalize Posix part.

* Fix style issues.

* Move osnet declarations to osdefs.

* Move osnet declarations to osdefs 2.

* Fix Nim 1.2 issues.

* Fix 1.2 compilation issues 2.

* Address review comments.
Fix SignalFdInfo issue.
2023-02-21 11:48:36 +01:00
Eugene Kabanov 77ffff322c
Fix missing decodeUrl() while decoding CommaSeparatedArray URL query parameters. (#360)
Fix unused `exc` warnings.
Fix unused ContentDisposition warning.
2023-02-17 12:52:54 +02:00
Tanguy a50ac4f642
Documentation (#311)
* Fix & update asyncloop documentation

* Add cancellation documentation

* Add documentation generation workflow

* pin nim version for doc gen

* update readme

* fix example

* Simplify examples
2023-02-16 21:27:31 +01:00
Eugene Kabanov 1e743dc415
IpNet refactoring (#354)
Remove `result` usage.
Remove cast[T] usage.
Add more helpers for TransportAddress checks.
Add isGlobal() helper to check if TransportAddress is globally accessed IP address.
Add tests.
Add more tests for sub.
Add TransportAddress.toHex().
2023-02-16 17:18:05 +01:00
Eugene Kabanov 5d3da66e56
Properly fix SIGBUS crash issue (#358)
* Fix SIGBUS crash.

* Workaround annoying and buggy style checker.

* Fix revealed SIGSEGV error.

* Add comments.
2023-02-09 07:15:22 +02:00
Tanguy 3d532b6d4a
Enable TCP client to enable TCP_NODELAY (#340)
* Enable TCP client to enable TCP_NODELAY

* add missing return

* fix
2023-01-23 11:18:12 +02:00
Tanguy f3b77d8661
Fix `async` `proc` types & small cleanups (#347)
* Fix `async` `proc` types & small cleanups

* review comments
2023-01-19 07:52:11 +01:00
Jacek Sieka c65cc4c136
`async` `proc` types (#346)
Allow `type = proc(...) {.async.}` for pre-declaring async callback
types - in this case, the macro applies the correct transformation to
the return type and adds the appropriate exception / gcsafe annotations
2023-01-18 15:54:39 +01:00
Eugene Kabanov 945c304197
Address #329. (#330)
* Address #329.

* Refactor AsyncStreamReader/Writer `state` helper functions.
Fix some compilation warnings in `debugutils`.
Add multiple writes into test to check consistency.

* Fix Linux issue.
Fix warnings.
2023-01-10 18:08:54 +02:00
Bung e9f8baa6ee
fix int set (#341) 2022-12-29 12:12:12 +01:00
Bung 5d0c725463
fix int set construction (#335) 2022-12-16 10:40:33 +01:00
Tanguy 75d030ff71
Fix async macro "CannotRaise" warnings in nim >1.2 (#332) 2022-12-01 11:36:14 +01:00
Eugene Kabanov 6525f4ce1d
Fix some warnings. (#310)
* Fix some warnings messages.

* More warning fixes.

* Address review comments.
Fix MacOS issues.

* More warning fixes.

* More Windows specific fixes.

* Fix macos and windows warnings.

* Fix warnings in timer.nim
Refactor to remove `result`.
Improve performance and behavior of timer to string procedure.
Add tests.
2022-11-02 08:09:15 +01:00
Tanguy 24146463a3
Token bucket (#279)
- A single sleepAsync per bucket
- Manual replenish with async
- Cancellation of consume

Co-authored-by: Jacek Sieka <jacek@status.im>
2022-11-02 08:03:19 +01:00
Eugene Kabanov 266e2c0ed2
HTTP client: Allow request connection management. (#323)
* Allow per-request connection management.
Fix NewConnectionAlways leak issue.

* Address review comment.
2022-10-13 10:18:13 +00:00
Eugene Kabanov 9df76c39df
Fix default content-type should not be sent if there is no body. (#315)
Make contentLength field publicly available.
2022-09-20 12:59:34 +00:00
Eugene Kabanov be2352027e
Fix nested waitFor (IndexError defect crash) bug. (#309)
Add IndexError crash test for Linux/MacOS.
Add Sentinel version of fix.
Add GetQueuedCompletionStatusEx() support for Windows, which allows to capture more then one event in single `poll()` call.
2022-09-16 22:34:18 +02:00
Eugene Kabanov 8e8263370b
Add more FutureBase helpers. (#304) 2022-09-12 12:11:13 +00:00
Jacek Sieka 746832384a
`Option` -> `Opt` (#307) 2022-09-05 11:08:52 +00:00
Eugene Kabanov 1334cdfebd
getPreferredContentType() deprecate float math (#303)
* Deprecate float math.

* Remove unneeded helper procedures.

* Remove limitation of 99 types and comments.
2022-08-10 14:07:51 +03:00