81 Commits

Author SHA1 Message Date
Eugene Kabanov
2e8551b0d9
Cancellation fixes and tests. (#445)
* Add callTick and stream cancellation tests.

* Fix stepsAsync() test.

* Cancellation changes.

* Update and add more cancellation tests.

* Fix Posix shutdown call to handle ENOTCONN error.

* With new changes to to cancellation its now possible.

* Refactor testsoon.nim to not produce artifacts after tests are finished.

* Debugging MacOS issue.

* Adjust flaky test times.

* Fix issue.

* Add test for issue #334 which was also addressed in this PR.
Avoid `break` in problematic test.

* Add noCancelWait() call which prohibits cancellation.
Fix closeWait() calls to use noCancelWait() predicate.
Adding sleep to flaky MacOS test.

* Remove all debugging echoes.

* Fix cancelAndWait() which now could perform multiple attempts to cancel target Future (mustCancel behavior).

* Fix issues revealed by switch to different cancelAndWait().

* Address review comments.

* Fix testutils compilation warning.

* Rename callTick() to internalCallTick().

* Add some documentation comments.

* Disable flaky ratelimit test.

* Rename noCancelWait() to noCancel().
Address review comments.
2023-09-15 19:38:39 +03:00
Eugene Kabanov
00614476c6
Address issue #443. (#447)
* Address issue #443.

* Address review comments.
2023-09-07 16:25:25 +03:00
Eugene Kabanov
300fbaaf09
HttpAddress errors should be not only critical. (#446)
* Distinguish between resolve errors and check errors.

* Fix issues and add test for getHttpAddress() procedure.

* Address review comments.
2023-09-04 21:49:45 +03:00
Eugene Kabanov
6c2ea67512
Unroll defers and remove breaks. (#440)
* Unpack `finally/defer` blocks and introduce explicit cleaning of objects.
Add request query to debug information.

* Unroll one more loop to avoid `break`.
Add test for query debug string.

* Fix cancellation behavior.

* Address review comments.
2023-08-09 10:57:49 +03:00
Eugene Kabanov
d214bcfb4f
Increase backlog defaults to maximum possible values. (#428) 2023-07-31 22:40:00 +03:00
Eugene Kabanov
926956bcbe
Add time used to establish HTTP client connection. (#427) 2023-07-30 12:43:25 +03:00
Eugene Kabanov
f91ac169dc
Fix NoVerifyServerName do not actually disables SNI extension. (#423)
Fix HTTP client SSL/TLS error information is now part of connection error exception.
2023-07-23 19:40:57 +03:00
Eugene Kabanov
e04c042e8a
Add cross-platform shutdown() call and use it for HTTP server. (#420)
* Add cross-platform shutdown() call and use it for httpserver connection processing.

* Fix Posix compilation issues and warnings.
2023-07-19 20:33:28 +03:00
Eugene Kabanov
d652c52142
Fix regression introduced by #416. (#419) 2023-07-17 16:39:41 +03:00
Eugene Kabanov
155d89450e
Trackers refactoring. (#416)
* Refactor chronos trackers to be more simple.

* Refactor trackers.
Add HTTP server trackers.
Refactor HTTP main processing loop.

* Compatibility fixes.
Add checkLeaks().

* Fix posix test issue.

* Add httpdebug module which introduces HTTP connection dumping helpers.
Add tests for it.

* Recover and deprecate old version of Trackers.

* Make public iterators to iterate over all tracker counters available.
Fix asynctests to use public iterators instead private one.
2023-07-14 13:35:08 +03:00
Eugene Kabanov
73fd1206ab
Disable HTTP/1.1 pipeline support for HTTP/S server by default. (#410)
* Disable HTTP/1.1 pipeline support for HTTP/S server by default.
Add tests.

* Fix tests according to new behavior.
Fix tests to use random assigned OS ports instead of predefined.

* Fix flaky tests in testasyncstream.
2023-06-26 16:28:33 +03:00
Jacek Sieka
1d6c309d77
clean up Defect (#404) 2023-06-05 22:21:50 +02:00
Eugene Kabanov
f748387462
Add HTTP client helper to read text/event-stream streaming response. (#375) 2023-05-19 17:25:22 +03:00
Eugene Kabanov
61d52b1ef8
Export SocketFlags for HTTP client. (#389) 2023-05-18 10:10:01 +03:00
Eugene Kabanov
8aa8ee8822
Add flag to completely disable HTTP/1.1 pipelining support. (#387)
* Add flag for HTTP/1.1 pipeline disable.

* Switch from NoHttp11Pipeline to Http11Pipeline.
2023-05-12 17:00:37 +03:00
Eugene Kabanov
0688d2ef8f
Add idle connection timeouts for HTTP client's connections pool. (#324)
* Add idle connection timeouts for HTTP client's connections pool.
Add timestamps and duration for both HTTP client requests/responses.
Add test.

* Add comments on `connectionFlag` decisions.

* Address review comments.
Adjust default idle connection timeout to 60 seconds.

* Increase timeout for test.

* Adjust timeout to lower value.

* Address review comments.
2023-03-21 15:10:35 +02:00
Etan Kissling
30c839bac7
fix hasOverflow for multiple streams (#370)
Instead of repeatedly checking stream 1, properly check all of them.
2023-03-16 11:46:24 +02:00
Eugene Kabanov
77ffff322c
Fix missing decodeUrl() while decoding CommaSeparatedArray URL query parameters. (#360)
Fix unused `exc` warnings.
Fix unused ContentDisposition warning.
2023-02-17 12:52:54 +02:00
Eugene Kabanov
266e2c0ed2
HTTP client: Allow request connection management. (#323)
* Allow per-request connection management.
Fix NewConnectionAlways leak issue.

* Address review comment.
2022-10-13 10:18:13 +00:00
Eugene Kabanov
9df76c39df
Fix default content-type should not be sent if there is no body. (#315)
Make contentLength field publicly available.
2022-09-20 12:59:34 +00:00
Jacek Sieka
746832384a
Option -> Opt (#307) 2022-09-05 11:08:52 +00:00
Eugene Kabanov
1334cdfebd
getPreferredContentType() deprecate float math (#303)
* Deprecate float math.

* Remove unneeded helper procedures.

* Remove limitation of 99 types and comments.
2022-08-10 14:07:51 +03:00
Miran
93800879fd
make it more Nim 1.4+ compatible (#296) 2022-08-06 13:56:06 +03:00
Eugene Kabanov
15d7e0ebb7
Add optimized implementation for preferredContentType() which avoid sorting and double iteration. (#299)
Fix tests to use 80cpl.
2022-08-06 13:53:40 +03:00
Eugene Kabanov
939195626f
Use new Content-Type header value parser. (#302) 2022-08-05 19:59:26 +03:00
Eugene Kabanov
f2e4d447d6
Workaround exception tracking issue. (#293)
* Workaround exception tracking issue.

* Fix SecureHttpServerRef too.
2022-07-04 12:31:18 +03:00
Eugene Kabanov
2a5095505f
Add headers argument to redirect() calls. (#287)
* Add headers argument to redirect() calls.

* Fix mistype.
2022-06-17 12:17:49 +03:00
Eugene Kabanov
b3548583fc
Fix last connection state check. (#278) 2022-05-27 23:03:32 +03:00
Eugene Kabanov
728ab7faa7
Fix one more place of assertion for high concurrency requests. (#277) 2022-05-27 20:28:39 +03:00
zah
875d7d8e6e
Tentative fix for the HTTP client connection state assertion failures (#272)
* Tentative fix for the HTTP client connection state assertion failures

I've traced the problem to a HTTP connection being closed while there
are outstanding requests that still go through the motions (the crash
occurs when a requests reaches its `finish` processing step, but it
was already put in a Closed state by the connection that owns it).

* Use distinct error exception instead of cancellation error.

Co-authored-by: cheatfate <eugene.kabanov@status.im>
2022-05-14 11:08:13 +03:00
Etan Kissling
dedd7cb1d0
Guarantee exclusive use of HttpClientConnection. (#273)
When calling the HTTP `send` / `open` functions, `acquireConnection` is
called to obtain a connection in state `Ready`. In the next code block,
the connection state is advanced to `RequestHeadersSending`. However,
returning from chronos `async` procs yields control, similar to `await`.
This means that a connection may be added to the pool in `Ready` state,
and then a different `acquireConnection` call may obtain a second ref.
Introducing a new `Acquired` state ensures consistency in this case.

No tests added due to this being scheduler dependent; ran manual tests
by adding a `doAssert item.state != HttpClientConnectionState.Acquired`
between `if not(isNil(conn)):` and `return conn`. Eventually, the assert
got hit after several hours of repeated tests, confirming the edge case
to be solved by applying this fix.

Not sure if it is by design that returning from an `async` proc yields.
Even if it's not, this should solve current HTTP issues in nimbus-eth2.
2022-05-13 02:54:34 +03:00
cheatfate
ae2a87778f
Fix missing connection flag. 2022-04-10 19:35:04 +03:00
Emil
1f37dac810 Refactor preferredContentType proc & add tests
Fix of the following bug:
In case of multiple accept headers with same preference
`preferredContentType` used to select the first match within content types
provided by the application. For example, if user specifies accept headers
`application/octet-stream, application/json` and application provides
`application/json, application/octet-stream`, `application/octet-stream`
will be returned, and that is a bug.

Now the procedure returns the most suitable match according
to the application preferences.
2022-01-25 16:05:36 +02:00
Jacek Sieka
0fc82049ac enable --styleCheck:usages 2021-12-27 17:33:25 +02:00
Eugene Kabanov
36e5f6fc89
Attempt to fix connection state. (#248)
* Attempt to fix connection state.

* Add actual values dump on asserts.

* Total rework of connection's states.
2021-12-03 13:11:39 +02:00
Eugene Kabanov
1c17d4c094
Fix connections management in http client. (#246)
* Proper connection lifetime handling mechanism.
HttpClientResponse do not close connection anymore.

* Add unique HTTP client connection identifier.
2021-12-01 12:23:39 +02:00
Eugene Kabanov
f51158da65
Add NewConnectionAlways option to HttpClientFlags, which allows to ignore connection management in client. (#238)
Optimize server behavior when HeadersTimeout is set to Infinity.
2021-11-19 18:13:36 +02:00
Eugene Kabanov
0c7a0bd0a5
Handle accept() errors properly. (#234)
* Add more accept() call error handlers.
Fix issue when HTTP server silently stops accepting new connections.
Remove unneeded MacOS syscall to disable SIG_PIPE on socket, we already mask this signal in process.

* User `if` instead `case` because constants are actually variables.

* Fix mistypes.

* Do not use case for posix constants.

* Fix Linux compilation error.
2021-11-12 18:13:56 +02:00
cheatfate
2e57ddb455
Fix preferredContentType() function declaration to avoid confusion with preferredContentType(varargs). 2021-10-21 14:01:46 +03:00
Eugene Kabanov
59b91bf0ca
Add more helpers to handle Accept header. (#224)
* Add API working with strings instead of `HttpRequestRef` object.

* Fix comment.
2021-09-28 01:41:29 +03:00
Eugene Kabanov
bbbcb55493
Server-side events implementation. (#222)
Fix keep-alive issue.
Refactor some prepareXXX routines.
2021-09-14 20:32:58 +03:00
Eugene Kabanov
05c91418be
Export async-related imports to allow partial imports. (#218) 2021-08-26 14:22:29 +03:00
Eugene Kabanov
9d80c7068f
Server must not send Host header. (#215) 2021-08-15 04:29:48 +03:00
Eugene Kabanov
b14f66c295
No more Defect on stream close. (#213)
Fix async streams issue with replacing state.
Add `closing` states to HTTP's server connection, request and bodyrw.
Fix Http server cancellation leaks.
2021-08-06 13:13:55 +03:00
Eugene Kabanov
ef2430d08d
Add Accept header handling to httpserver.nim. (#211)
* Add `Accept` header handling to httpserver.nim.
Add simple test suite.
Bump version to 3.0.6.

* Fix compilation error.
2021-07-28 17:08:38 +03:00
Eugene Kabanov
15137f71c3
Basic authorization implementation for HTTP client. (#204)
* Basic authorization implementation for HTTP client.
Add tests for basic authorization.

* Bump chronos version to 3.0.5.
2021-06-29 02:38:08 +03:00
Eugene Kabanov
f7dd6b76c2
Safely close connection's streams. (#199) 2021-06-14 15:16:33 +03:00
Eugene Kabanov
e6fd38fd49
Add one more state Closing to help avoid race condition while in closeWait of request, response, connection. (#198) 2021-06-10 13:58:41 +03:00
Eugene Kabanov
7ccb170f7a
Enable comma as array delimiter and adding tests. (#191)
* Enable comma as array delimiter and adding tests.

* Bump version to 3.0.4.
2021-05-17 22:39:24 +03:00
cheatfate
67f0f1224f
Bump nimble version.
Make some useful fields public.
2021-05-10 10:40:56 +03:00