Using `tunnel`, a general TCP tunnel can be created for any protocol via
a HTTP proxy. This tunnel is typically used for establishing TLS
connections via a proxy but any protocol can be used.
When a tunnel is established, the connection is removed from the session
pool and given to the caller who is now responsible for closing it.
Because the tunnel connection itself can be encrypted, we return an
`AsyncStream` that the caller must close (rather than a
`StreamTransport`).
This PR also moves DNS name resolution to the connection provider - this
has the effect of delaying DNS lookup until the actual connect attempt
is made - earlier, DNS lookups would happen when creating the HTTP
address. The move allows the proxy to perform the name resolution when
relevant - the move also prepares the API for async name resolution.
Callers can still skip name resolution by providing an explicit list of
addresses to connect to.
The pipelining implementation was incomplete insofar as it only enabled
connection reuse without actually performing pipelining - keep the reuse
but deprecate pipelining.
With pipelining cleaned up, several other RFC 9112-related cleanups can
also be performed:
* clarify "perstistent connections" vs pipelining, where relevant
* in HTTP/1.1, don't send `Connection: keep-alive` (this is the default
in 1.1) - similarly, don't send `Connection: close` in other versions
* in older HTTP versions, disable persistent connections / keep-alive
entirely following RFC recommendations
* decode message body length according to RFC 9112
* prioritise `Transfer-Encoding` over `Content-Length`, per RFC 9112
* document lack of EOF monitoring when using persistent connections
* differentiate "no body" from "body length" tracking, ie a HEAD request
never has a body but might send a Content-Length for use as a length
discovery mechanism
* streamline request body sending code to reuse stream helpers
* fix sending and parsing of CONNECT authority form
* fix processing of `Connection` header - in particular, `Connection:
Keep-Alive` should not be used in HTTP/1.1 since persistent connections
are the default - instead, `Connection: close` should be used when the
connection is _not_ perisistent.
* exclude fragment from request-target
Using an absolute request URI and a connection provider that connects to
a different server than the http address, generic proxies can be
implemented
* add detail to http connection error
* fix transport/stream leak when http client construction fails
Add support for mTLS client certificate authentication
in `newTLSClientAsyncStream`. Both RSA and EC key types are supported.
* Add EC test
---------
Co-authored-by: Jacek Sieka <jacek@status.im>
* Fix original TokenBucket, consolidate the behaviors of chronos/TokenBucket and waku/TokenBucket (compensating) with no interface change
* TokenBucket extended with waku's strict mode replenish, that does not allow refill just after period boundary elapsed.
* Adjust for new chat-sdk needs, added setState and getAvailableCapacity with small refactoring
* Better comments
* rename for better self explain code, added more explanatory comments upon code review finding
* Document TokenBucket with detailed samples of Balanced mode replenishment algorithm, extended TokenBucket unit test
* Address review comments, renaming the replenish modes, removed old/new algo comparison from documentation, fix Discrete mode update calculation to properly calculate correct last update time by period distance calculation.
* protect from devide by zero, code style fix
* Make (re)setState close pending request before reseting state
* Polishing interface, move discrete mode initial start time setup into ctor and remove resetState to have a cleaner and consistent interface for both usage mode
* Addressing review observation, keep only singel new ctor and leave defaults to match former use.
* Removing leftovers and confirm with coding guideline
* refactor rate limiter implementation
* make sure that `tryConsume` respects queued `consume` requests
* refill tokens the same way regardless of replenish mode and source of
tokens (manual/time-based/cancellation)
* whem manually replenishing, compute budget cap after satisfying queued
requests
* don't run worker if fill duration is 0
* add some docs
* int vs int64
* Fix Discrete mode time window drift, added tests
* wait for sleeper/waiter
---------
Co-authored-by: Jacek Sieka <jacek@status.im>
* add missing calls to await
* add test run in v3 compatibility
* fix semantics for chronosHandleException so it does not override local raises/handleException annotations
* distinguish between explicit override and default setting; fix test
* re-enable wrongly disabled check
* make implementation simpler/clearer
* update docs
* reflow long line
* word swap
* Add automatic constructors for TCP and UDP transports.
* Add port number argument.
Add some documentation comments.
Fix tests.
* Make datagram test use request/response scheme.
* Add helper.
* Fix issue with non-zero port setups.
Add test.
* Fix tests to probe ports.
* Attempt to fix MacOS issue.
* Add Opt[IpAddress].
Make IPv4 mapping to IPv6 space automatic.
* Add tests.
* Add stream capabilities.
* Fix Linux issues.
* Make getTransportFlags() available for all OSes.
* Fix one more compilation issue.
* Workaround weird compiler bug.
* Fix forgotten typed version of constructor.
* Make single source for addresses calculation.
* Add one more check into tests.
* Fix flags not being set in transport constructor.
* Fix post-rebase issues with flags not being set.
* Address review comments.
We can check at compile-time that at least one parameter is passed
* clean up closure environment explicitly in some callbacks to release
memory earlier
* Fix `or` should not create future with OwnCancelSchedule flag set.
* Fix `CancelledError` missing from raises list when both futures has empty raises list.
* Fix macros tests.
* Add more specific accept() exceptions raised.
Add some refactoring to HTTP server code.
* Refactor acceptLoop.
* Print GC statistics in every failing test.
* Try to disable failing tests.
Because the callback types were used explicitly in some consumers of
chronos, the change of type introduces a backwards incompatibility
preventing a smooth transition to v4 for code that doesn't uses
`raises`.
This PR restores backwards compatibility at the expense of introducing a
new type with a potentially ugly name - that said, there is already
precedence for using numbered names to provide new error handling
strategy in chronos.
This avoids copies here and there throughout the pipeline - ie
`copyString` and friends can often be avoided when moving things into
and out of futures
Annoyingly, one has to sprinkle the codebase liberally with `sink` and
`move` for the pipeline to work well - sink stuff _generally_ works
better in orc/arc
Looking at nim 1.6/refc, sink + local variable + move generates the best
code:
msg directly:
```nim
T1_ = (*colonenv_).msg1; (*colonenv_).msg1 = copyStringRC1(msg);
```
local copy without move:
```nim
T60_ = (*colonenv_).localCopy1; (*colonenv_).localCopy1 =
copyStringRC1(msg);
```
local copy with move:
```nim
asgnRef((void**) (&(*colonenv_).localCopy1), msg);
```
Annoyingly, sink is also broken for refc+literals as it tries to
changes the refcount of the literal as part of the move (which shouldn't
be happening, but here we are), so we have to use a hack to find
literals and avoid moving them.
With these fixes, `transports`/`asyncsync` correctly propagate and document their raises information - generally, most transport functions (send etc) raise `TransportError` and `CancelledError` - `closeWait` is special in that it generally doesn't fail.
This PR introduces the syntax `Future[void].Raises([types])` to create the `InternalRaisesFuture` type with the correct encoding for the types - this allows it to be used in user code while retaining the possibility to change the internal representation down the line.
* introduce raising constraints on stream callbacks - these constraints now give a warning when called with a callback that can raise exceptions (raising callbacks would crash
* fix fail and its tests, which wasn't always given a good generic match
* work around nim bugs related to macro expansion of generic types
* make sure transports raise only `TransportError`-derived exceptions (and `CancelledError`)
Jacek Sieka