Commit Graph

704 Commits

Author SHA1 Message Date
Eugene Kabanov d214bcfb4f
Increase backlog defaults to maximum possible values. (#428) 2023-07-31 22:40:00 +03:00
Eugene Kabanov 926956bcbe
Add time used to establish HTTP client connection. (#427) 2023-07-30 12:43:25 +03:00
Eugene Kabanov 53e9f75735
Add some helpers for asyncproc. (#424)
* Initial commit.

* Adjust posix tests.

* Fix compilation issue.

* Attempt to fix flaky addProcess() test.
2023-07-28 11:54:53 +03:00
Eugene Kabanov f91ac169dc
Fix `NoVerifyServerName` do not actually disables SNI extension. (#423)
Fix HTTP client SSL/TLS error information is now part of connection error exception.
2023-07-23 19:40:57 +03:00
Eugene Kabanov 0277b65be2
Asynchronous thread notification mechanism. (#406)
* Initial commit.

* Some fixes.

* More fixes.

* Add first test.

* Further fixes for MacOS/BSD.

* Fixes for Linux.

* Add proper tests.

* Lower number of tests.

* Add threadsync tests to test suite.

* There is no need to run tests when threads are off.

* Address review comments.
Fix the issue with multiple signals.
Add tests.

* Switch to use socketpair() instead of pipes.
Fix semaphoring issue on MacOS/BSD.
Add tests.

* Add single threaded fire/wait tests.
2023-07-21 15:51:36 +03:00
Eugene Kabanov e04c042e8a
Add cross-platform shutdown() call and use it for HTTP server. (#420)
* Add cross-platform shutdown() call and use it for httpserver connection processing.

* Fix Posix compilation issues and warnings.
2023-07-19 20:33:28 +03:00
Eugene Kabanov d652c52142
Fix regression introduced by #416. (#419) 2023-07-17 16:39:41 +03:00
cheatfate 726fcb1915
Revert "Bump version to 3.2.1."
This reverts commit c8f650e651.
2023-07-14 16:40:07 +03:00
cheatfate c8f650e651
Bump version to 3.2.1. 2023-07-14 14:57:24 +03:00
Eugene Kabanov 155d89450e
Trackers refactoring. (#416)
* Refactor chronos trackers to be more simple.

* Refactor trackers.
Add HTTP server trackers.
Refactor HTTP main processing loop.

* Compatibility fixes.
Add checkLeaks().

* Fix posix test issue.

* Add httpdebug module which introduces HTTP connection dumping helpers.
Add tests for it.

* Recover and deprecate old version of Trackers.

* Make public iterators to iterate over all tracker counters available.
Fix asynctests to use public iterators instead private one.
2023-07-14 13:35:08 +03:00
Jacek Sieka 3d80ea9fc7
fix EpollEvent memory layout (#411)
* fix EpollEvent memory layout

on x86_64, `EpollEvent` is packed - the upstream version is correct here

* copy-paste upstream instead

* Adjust lines to be 80 characters per line.

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>
2023-07-06 19:51:59 +03:00
Jacek Sieka 0a6f5854a7
fix unused result warning on implict return (fixes #414) (#417)
* fix unused result warning on implict return

* describe `{.used.}` tradeoffs

* oops
2023-07-04 01:11:08 +03:00
Eugene Kabanov 94ca0c3847
Fix remoteAddress() should raise TransportAbortedError too. (#409)
Add ENOTCONN as TransportAbortedError.
Refactor remoteAddress()/localAddress().
2023-06-26 16:31:09 +03:00
Eugene Kabanov 73fd1206ab
Disable HTTP/1.1 pipeline support for HTTP/S server by default. (#410)
* Disable HTTP/1.1 pipeline support for HTTP/S server by default.
Add tests.

* Fix tests according to new behavior.
Fix tests to use random assigned OS ports instead of predefined.

* Fix flaky tests in testasyncstream.
2023-06-26 16:28:33 +03:00
Tanguy 47016961f5
Less flaky rate limit tests (#408) 2023-06-23 10:11:14 +02:00
Jacek Sieka 0035f4fa66
Introduce `chronos/futures` (#405)
* move `Future[T]` into its own module along with some basic accessors
* mark all fields internal, exposing only read-only versions under the
old names
* introduce `init`/`completed`/etc as a way of creating a future (vs
newFuture)
* introduce `LocationKind` for `SrcLoc` access
* don't expose `FutureList` unless future tracking is enabled
* introduce `chronosStrictFutureAccess` which controls a number of
additional `Defect` being raised when accessing Future fields in the
wrong state - this will become true in a future version

In this version, `Future[T]` backwards compatibility code remains in
`asyncfutures2` meaning that if only `chronos/futures` is imported, only
"new" API is available.

This branch is a refinement / less invasive / minimal version of
https://github.com/status-im/nim-chronos/pull/373.
2023-06-07 20:04:07 +02:00
Jacek Sieka 1d6c309d77
clean up `Defect` (#404) 2023-06-05 22:21:50 +02:00
Jacek Sieka 2ef34c7339
ratelimit: set cancellation earlier (#402)
future may be completed by worker before cancellation is set
2023-06-05 13:47:38 +02:00
Jacek Sieka a6ac5f2213
exception warning fixes (#403) 2023-06-05 13:03:38 +02:00
Jacek Sieka 157ca4fea5
Support implicit returns (#401)
* based on https://github.com/nim-lang/Nim/pull/21898
* also fixes generic Future[T] where T ends up being `void`
2023-06-05 13:02:13 +02:00
tersec 2c7774d982
drop support for Nim 1.2, 1.4, add Nim 2.0 to CI testing matrix (#378)
* add Nim 2.0 to CI testing matrix

* Deprecate 1.2 and 1.4 testing branches.

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>
2023-06-04 09:51:02 +02:00
Jacek Sieka ba143e029f
version 3.2.0 (#399)
last version to support Nim 1.2, 1.4 - may receive critical backports
but otherwise this is EOL for said Nim versions
2023-06-04 09:46:04 +02:00
Eugene Kabanov 6e80f16e0c
Fix recently introduced imports poisoning. (#400)
* Fix import poison introduced.

* Move export to handles, and annotate it.
2023-06-03 19:36:06 +03:00
Eugene Kabanov 02fda01bf2
Add facility to emulate signals on Windows. (#357)
* Initial Windows asyncproc part.
Deprecate usage of net/nativesockets in handles, asyncloop.
Introduce primitives with inheritable flag.

* Some posix fixes.

* More duplicate fixes.

* Fix AsyncProcessOption.EchoCommand option.
Remove isMainModule code.
Fix execCommand to use AsyncProcessOption.EvalCommand.

* Initial asyncproc tests commit.

* Some Posix fixes.

* Fix Posix crash.

* Add testsuite.
Fix osnet.nim to use proper declarations.
Fix timer.nim to use proper declarations.
Add Windows environment handling procedures.
Fix createAsyncPipe.
Add leaks tracking for AsyncProcessRef.

* Fix O_CLOEXEC constant value.

* Add file descriptors leak test.

* Remove commented code.
Refactor exceptions.
Fix compilation warnings.

* No exception ioselectors_kqueue initial commit.

* Some BSD fixes.
Linux refactoring of selectors.nim.

* Some fixes to move further.

* Last Linux fixes.

* Fixes for asyncloop to use 2nd version of selectors api.

* Add osutils.nim.

* Some fixes.

* Hardening resumeRead(), resumeWrite() and consumers.
Add ESRCH handling.
Introduce no-exception fromPipe2.

* Make Windows part exception-free and fix zombie race issue.

* createStreamServer() fixes.

* Upgrade asyncproc to use non-exception primitives.
Fix ioselectors_kqueue to use closeFd().

* Refactor accept() and acceptLoop() to be exception free.

* Deprecated some `result` usage.
Last fixes to make stream.nim exception free.
Use closeFd().
Refactor some loops to use handleEintr().

* Fix connect() forgot to unregister socket on error.

* All createAsyncSocket() sockets should be closed with unregisterAndCloseFd().

* Attempt to fix posix bug with incomplete output.

* Change number of runs in big test.

* Refactoring pipes creation. Attempt to fix "write error: Resource temporarily unavailable".

* Fix waitForExit(duration) code.
Fix test exit code.

* Fix Windows missing SIGTERM.

* Fix mistype.

* Fix compilation error.

* Attempt to fix Nim 1.6 issue.

* Eliminate Nim's WideCString usage to avoid any problems in future.

* Deprecate posix usage in osnet.

* Eliminate unused imports.

* Some debugging statements for investigation.

* Remove all the debugging statements.

* Remove testhelpers in favor of unittest2/asynctests.

* Fix flaky test.

* Make test more resilient to timings.

* Add memory size dump to CI.

* Refactor some blocks to use valueOr.
Make waitForExit to use kill() instead of terminate().

* Remove memory dumps.

* Fix peekProcessExitCode() blocks issue.

* Fix Windows issue.

* Add some debugging echoes.

* Fix compilation error.

* Add some debugging echoes.

* Add more helpers.

* Fix compilation error.

* waitForExit() is primary suspect.

* Remove fast-path for waitForExit.
Remove loop for reading signal events.

* Remove all debugging echoes.

* Return one debugging echo.

* Fix futures tests.

* Add test for multiple processes waiting to attempt stress test.

* Refactor ioselectors_epoll for better signalfd and process monitoring.
Add more race condition fixes to waitForExit.
Fix some Nim 1.6 warnings.

* Fix after rebase issues and warnings.

* Fix style issues.
Fix different Nim versions issues.
Workaround `signalfd` style issues.

* Add one more Linux signals workaround.
Add one more multiple processes test.

* Windows fixes.

* Remove unixPlatform define.
Fix WSAECONNABORTED for devel.

* Temporarily disable rate limit tests.
Fix more devel issues.

* Deprecate `hasThreadSupport` for ioselectors_kqueue.
Fix verifySelectParams issue.
Add exit codes test for multiple processes.
Fix osnet PtrToCstringConv warning.

* ioselectors_kqueue refactoring.

* Initial commit.

* Fix 1.2-1.4 compilation issue.

* Fix unused warning for testCtrlC() test.

* Post-rebase fixes.

* Restore master files.

* More fixes.

* Remove duplicates.

* Fix style mistake.

* Add more flexible pragmas.
2023-06-02 01:53:20 +03:00
Eugene Kabanov 315a27236c
Recover cancellation cleanup for AsyncEvent wait(). (#398)
* Recover cancellation cleanup for AsyncEvent wait().

* Address review comments.
2023-06-01 18:09:49 +03:00
gpicron 5be223e92c
allow reuse of port for UDP (#379) 2023-06-01 17:02:33 +03:00
Jacek Sieka e436f20b33
Memory cleanups (#395)
* Avoid `FutureContinue` bloat by moving type punning into iterator
(which is typed anyway)
* clear closure iterator from future when iteration is done also on
cancellation / exception
* remove some redundant local variables in `await`
* document `futureContinue` flow
2023-05-31 07:24:25 +02:00
Eugene Kabanov 6523f741a6
Assertion and bugfix. (#397) 2023-05-30 17:45:16 +02:00
Eugene Kabanov 36ab9c8e4a
Fix sigprocmask declaration missing when threads are disabled. (#396) 2023-05-30 07:57:57 +02:00
Jacek Sieka b65b85533a
Future cleanup (#393)
* FutureState.Finished -> FutureState.Completed (to avoid name clash
with `proc finished` which means not-pending)
* deprecate `done` - to avoid additional confusion over completed vs
finished
* remove ad leftovers in stack trace formatting
* avoid some generic bloat
* avoid unnecessary allocations in `race`/`one`
2023-05-25 17:31:35 +02:00
Jacek Sieka 2fa6df0880
asyncmacro: code cleanups (#392)
* prefer `let`
* prefer expressions
* renames
2023-05-23 19:45:12 +02:00
Jacek Sieka 4c07da6abb
add test for raise-only patterns (#391)
* add test for raise-only patterns

https://github.com/status-im/nim-chronos/issues/56

* fix
2023-05-23 18:40:52 +02:00
Eugene Kabanov 148ddf49c2
Asyncproc (Part 3/3) (#374)
* Initial commit.

* Some Linux fixes.

* Address review comments on Windows.

* Fix issues on Linux.

* Fix 1.2 issue and Windows warnings.

* Fix posix compilation issues.
2023-05-23 12:39:35 +02:00
Eugene Kabanov f748387462
Add HTTP client helper to read `text/event-stream` streaming response. (#375) 2023-05-19 17:25:22 +03:00
Bung 38cc233700
fix missing sigprocmask import (#390) 2023-05-19 13:43:33 +03:00
Eugene Kabanov 61d52b1ef8
Export SocketFlags for HTTP client. (#389) 2023-05-18 10:10:01 +03:00
Eugene Kabanov 956ae5af55
Fix import poison regression. (#388)
* Initial commit.

* One more linux fix.

* Eliminate posix from asyncloop.

* Fix Linux warnings.

* Fix MacOS issues.

* Fix BSD issues.

* Fix comma.

* Fix Windows issues.
2023-05-15 19:45:26 +03:00
Eugene Kabanov 8aa8ee8822
Add flag to completely disable HTTP/1.1 pipelining support. (#387)
* Add flag for HTTP/1.1 pipeline disable.

* Switch from NoHttp11Pipeline to Http11Pipeline.
2023-05-12 17:00:37 +03:00
Eugene Kabanov 8563c93673
Reexport osdefs where Sockaddr_storage was re-declared. (#385) 2023-05-12 07:11:05 +03:00
Eugene Kabanov ef94d75d61
Eliminate last 2 macos warnings. (#384) 2023-05-01 16:02:59 +03:00
Eugene Kabanov 5755d79b67
Eliminate warnings (#383)
* Eliminate warnings on Windows.

* Deprecated toHex() in osnet.
2023-05-01 00:46:15 +03:00
Eugene Kabanov 9ba418cd3b
Eliminate warnings on Windows. (#382) 2023-04-30 19:09:36 +02:00
Eugene Kabanov 3118f8c1b2
Fix cast[pointer] issues on nim-devel. (#381)
* Fix cast[pointer] issues on nim-devel.

* More cast[T] fixes.

* Fix compilation error.

* Add oserrno.nim
Further fixes for Windows and Linux.

* MacOS fixes.

* More Windows fixes and attempt to fix 1.2, 1.4 branches.

* Implicitly import/export oserrno.

* Replace oserrno with osdefs.

* Return back oserrno.

* epoll to oserrno.

* datagram/stream to oserrno.

* common to oserrno.

* test to oserrno.
2023-04-30 08:20:08 +02:00
Eugene Kabanov e05d2f8e96
Fix some compilation warnings and expose tracker names. (#376)
* Fix asyncsync compilation warnings.

* Fix tracker names should be public.
2023-04-08 19:34:57 +03:00
Tanguy ab5a8c2e0f
Add `localAddress` support to `stream.connect` (#362)
* Add `localAddress` support to `stream.connect`

* fix windows

* TransportAddress() instead of AnyAddress

* tweak flags

* Better flags

* try to workaround nim 1.2 issue

* Handle ReusePort in createStreamServer and improve tests

* Rename ClientFlags to SocketFlags

---------

Co-authored-by: Diego <diego@status.im>
2023-04-03 15:34:35 +03:00
Jacek Sieka 229de5f842
Compile-time configuration (#371)
This PR moves all compile-time configuration to a single module,
simplifying documentation and access to these features.

Upcomfing features may be enabled either individually, or through a new
`chronosPreviewV4` catch-all designed to allow code to be prepared for
increased strictness in future chronos releases.

`-d:chronosDebug` may be used to enable the existing debugging helpers
together.
2023-03-31 07:35:04 +02:00
Eugene Kabanov 1394c9e049
IOSelectors refactoring to properly support signals and processes. (AsyncProc 2) (#366)
* ioselectors_epoll() refactoring.

* ioselectors_kqueue() refactoring.

* ioselectors_poll() initial refactor.

* Remove `s.count` because it inconsistent and not used in `chronos`.

* Remove Windows version of select() engine.

* Add ability to switch event queue engine via `asyncEventEngine` command line option.

* Make it possible to switch between engines.

* Fix epoll regression.

* Fix poll() engine issues.

* Address review comments.

* Add proper trick.

* Address review comments.

* Bump version to 3.1.0.
2023-03-24 17:52:55 +02:00
Eugene Kabanov b0af576c7c
Address #320 issue. (#372) 2023-03-24 17:34:45 +02:00
Eugene Kabanov 0688d2ef8f
Add idle connection timeouts for HTTP client's connections pool. (#324)
* Add idle connection timeouts for HTTP client's connections pool.
Add timestamps and duration for both HTTP client requests/responses.
Add test.

* Add comments on `connectionFlag` decisions.

* Address review comments.
Adjust default idle connection timeout to 60 seconds.

* Increase timeout for test.

* Adjust timeout to lower value.

* Address review comments.
2023-03-21 15:10:35 +02:00
Etan Kissling 30c839bac7
fix `hasOverflow` for multiple streams (#370)
Instead of repeatedly checking stream 1, properly check all of them.
2023-03-16 11:46:24 +02:00