The `when defined(nimdoc)` stub branch omitted `contains`, which
threadsync.nim calls via `loop.contains(fd)`, so `nim doc` failed for
modules importing chronos/threadsync.
Using `tunnel`, a general TCP tunnel can be created for any protocol via
a HTTP proxy. This tunnel is typically used for establishing TLS
connections via a proxy but any protocol can be used.
When a tunnel is established, the connection is removed from the session
pool and given to the caller who is now responsible for closing it.
Because the tunnel connection itself can be encrypted, we return an
`AsyncStream` that the caller must close (rather than a
`StreamTransport`).
This PR also moves DNS name resolution to the connection provider - this
has the effect of delaying DNS lookup until the actual connect attempt
is made - earlier, DNS lookups would happen when creating the HTTP
address. The move allows the proxy to perform the name resolution when
relevant - the move also prepares the API for async name resolution.
Callers can still skip name resolution by providing an explicit list of
addresses to connect to.
The pipelining implementation was incomplete insofar as it only enabled
connection reuse without actually performing pipelining - keep the reuse
but deprecate pipelining.
With pipelining cleaned up, several other RFC 9112-related cleanups can
also be performed:
* clarify "perstistent connections" vs pipelining, where relevant
* in HTTP/1.1, don't send `Connection: keep-alive` (this is the default
in 1.1) - similarly, don't send `Connection: close` in other versions
* in older HTTP versions, disable persistent connections / keep-alive
entirely following RFC recommendations
* decode message body length according to RFC 9112
* prioritise `Transfer-Encoding` over `Content-Length`, per RFC 9112
* document lack of EOF monitoring when using persistent connections
* differentiate "no body" from "body length" tracking, ie a HEAD request
never has a body but might send a Content-Length for use as a length
discovery mechanism
* streamline request body sending code to reuse stream helpers
* fix sending and parsing of CONNECT authority form
* fix processing of `Connection` header - in particular, `Connection:
Keep-Alive` should not be used in HTTP/1.1 since persistent connections
are the default - instead, `Connection: close` should be used when the
connection is _not_ perisistent.
* exclude fragment from request-target
Using an absolute request URI and a connection provider that connects to
a different server than the http address, generic proxies can be
implemented
* add detail to http connection error
* fix transport/stream leak when http client construction fails
When fetching events with epoll/kqueue, we currently use a fixed size
array to fetch system events.
Taking an idea from
[libevent](https://github.com/libevent/libevent/blob/48296514d8fd9c0b3812b11d45ad80b0c002c14e/epoll.c#L568),
we can instead dynamically grow the number of events fetched as the
system comes under load thus reducing the number of poll calls, allowing
more work to be done per batch.
The change also has the effect that data that already arrived at the
time of the poll call gets processed before timers are fired - this
means that timeouts are less likely to be triggered unfairly due to
event queueing order and the time it takes to process each event.
* increase max events per loop to 4096 (same as libevent)
* reduce initial event list allocation to 32
* get rid of `ReadyKey.errorCode` which is unused
* streams: cleanup
* allow and more consistently handle zero-byte writes across
stream/transport
* remove redundant `read`/`consume` from stream vtable
* remove `toUnchecked` untyped template (not part of transport
interface!)
* make `AsyncStreamState` pure
* reset stream queue when done (releases memory a little earlier)
* fix cancel callback defect on some windows accept errors
* fix docs for `read(n)` (we'll read fewer than `n` bytes in case of
EOF)
* disable orc tests on 2.0
* verbose
* all orc tests >=2.2
The current approach of specializing streams for specific protocols
relies on each stream operating a buffer/queue and a read/write loop
that replenishes or drains these.
For some protocols such as TLS, a buffer is naturally required since
we're mutating the data along the way (in this case by encrypting it) -
for others, such as chunking or limiting, the data remains unchanged and
all the stream does is to either frame the data or provide limits and
safety nets - for such streams, the buffer approach carries significant
overhead - using a read as an example:
* a read loop starts reading from an underlying data source
* when data arrives, data is copied to a buffer
* when application wants to read, it checks the buffer and copies data
to user buffer
* if data is not available, it waits for the buffer to be replenished by
the read loop
In the above case, every byte must be copied twice and we have to wait
an additional loop iteration for each stream layer as the loop wakes the
actual reader.
Futher, each layer requires memory for the buffer - using a small buffer
means high latency due to the extra event loop round trips while a large
buffer is wasteful since most of the time, it's not actually being used
(ie while waiting for data from a socket).
The API of streams is fairly complex - operations such as `readLine` and
`readUntil` also require buffering since they must "look ahead" in the
data to find separators and at the same time not "overread" their
supporting source stream.
However, other operations do not require this extra buffering -
`readOnce` for example is already given a buffer to write to and all it
needs to do is continue using this buffer, similar as happens with [bulk
reads](https://github.com/status-im/nim-chronos/pull/632).
Similarly, when servicing a `write`, we do not actually need to copy
the data into a queue - all that's needed is that the data remains
available throughout the write operation, a guaranteee conveniently
provided by `async` already.
All in all, this means that for simple reads and writes, there is no
inherent need to do buffering - the buffering is merely a side effect of
the loop-based implementation.
By replacing the loop with a vtable, we can specialize the operations
that a stream based on the specific requirements of the stream protocol
being implemented instead of forcing them all to use a buffer.
This allows implementations that don't require a buffer to specialize
each operation according to its specific needs - a chunked reader for
example can pass the user buffer directly to the next layer, and if the
next layer is a socket, it means that the read is done directly into the
user buffer for the majority of data. Using knowledge about the limits,
it can also pre-allocate a target buffer of the correct size instead
of incrementally building it up.
A common structure of protocols is that they start with a header and
then perform bulk transfers - for these protocols, we can use the stream
buffer for the header read and perform the rest of the transfer directly
to the user buffer.
The effect is significant - here's the http benchmark (http is a heavy
user of streams):
Pre:
```
| Small/small | 0.063s | 1000 | 15817.438 | 0.031 MB | 0.016 MB | 0.483 MB/s | 0.256 MB/s |
| Medium/small | 0.845s | 1000 | 1182.766 | 1000.000 MB | 0.021 MB | 1182.766 MB/s | 0.025 MB/s |
| Small/Medium | 1.045s | 1000 | 956.632 | 0.031 MB | 1000.000 MB | 0.029 MB/s | 956.632 MB/s |
| Medium/Medium | 1.286s | 1000 | 777.588 | 1000.000 MB | 1000.000 MB | 777.588 MB/s | 777.588 MB/s |
```
Post:
```
| Small/small | 0.073s | 1000 | 13774.390 | 0.031 MB | 0.016 MB | 0.420 MB/s | 0.223 MB/s |
| Medium/small | 0.312s | 1000 | 3200.978 | 1000.000 MB | 0.021 MB | 3200.978 MB/s | 0.067 MB/s |
| Small/Medium | 0.537s | 1000 | 1862.990 | 0.031 MB | 1000.000 MB | 0.057 MB/s | 1862.990 MB/s |
| Medium/Medium | 0.880s | 1000 | 1136.555 | 1000.000 MB | 1000.000 MB | 1136.555 MB/s | 1136.555 MB/s |
```
* deprecate `udata` from streams - this is not being used anywhere and
just creates surface area for bugs
* deprecate `loop`-based streams
* move all streams to vtable-based approach, avoiding `if`-based
implementation selection
* keep readUntil/readLine implementation close to the buffer
* maintain atomicity of `write` calls with `AsyncLock` instead of a
queue
Once these foundational changes are done, there are a few more things
that can be improved:
* HTTP streams can be further simplified - in particular, the bounded
stream serves many masters and could be split into a "byte-limiting"
stream and a "readUntil"-limiting stream so that the implementation
of each is simpler, composing them instead.
* TLS streams use a trick to simplify the implementation: instead of
reading and writing concurrently, a read is initiated but all writes
are processed before that read is awaited - the two operations could
be made fully concurrent with some additional implementation
complexity
* TLS streams are documented not to perform `close_notify` since this
would require expanding the API to provide a notion of "full duplex
close", something that the split reader/writer streams don't do
well (in TLS 1.3, the protocol has been upgraded to support half-
closed streams which would fit the model better)
* Closing sequences could be simplified - in particular, the concept
of ownership should be introduced so that closing a "parent" stream
also closes the underlying source, when this is desireable.
Similarly, finishing a write stream should lead to `SHUT_WR` being
sent on the wire making the underlying TCP closing sequence more
predictable and efficient.
* Parts of the stream API can be simplified or removed - for example,
there's no need to maintain so many "states" since we no longer
need to track the activity of a "loop" that no longer exists.
* Potentially, buffered operations could be moved out from the API
entirely and/or offered only by certain stream types, dividing the
VTable into a "simple" unbuffered subset and the full buffered
set of operations.
Allowing `complete` to `sink` values avoids the compiler to move values
into the future which can avoid copies in a few cases:
Previous attempts have been reverted due to
https://github.com/nim-lang/Nim/issues/23354.
Incidentally, we can also remove the workaround for
[literals](https://github.com/nim-lang/Nim/issues/22175) since we gate
the usage of sink from 2.0.6.
Bulk reads allow data received from `StreamTransport` to be copied
directly to a user-supplied buffer, bypassing the transport buffer and
callback mechanism.
For large buffers, this increases throughput by almost 3x:
Pre:
```
3244.356 MB/s 209103.971 reads/s 16269.200 bytes/read
```
Post:
```
11181.966 MB/s 184897.441 reads/s 63414.298 bytes/read
```
Latency is also reduced significantly under load (due to fewer yields
per read) and the transport buffer can be smaller (or zero for
applications that do their own buffering).
Using `readExactly` as an example:
* `transp.buffer` is empty - `reader` future is set up, `resumeRead` is
called
* control yields to the chronos (even if the socket has data)
* data becomes available, selector wakes callback and `recv` is called
copying the data to `transp.buffer`, up to the size of the buffer
* `reader` future is completed, control yields to chronos
* `readExactly` resumes, copies `transp.buffer` to user buffer
* repeat
The new flow instead performs the following operations:
* `recv` is called in `readExactly` writing directly to the user buffer
until `EWOULDBOCK`
* `resumeRead` is called to wait for more data
* `recv` is called in the callback, writing directly to the user buffer
* `readExactly` resumes
* repeat
In the case that there is data already in the socket, `readExactly`
returns at once without 2 more loop iterations.
In the case that there is lots of data in the socket, fewer calls to
`recv` are needed since the user buffer already is as large as the
expected amount of data.
In the case there is no data, this process results in an extra `recv`
call (to discover `EWOULDBLOCK`) compared to the status quo.
Bulk reads are only used when the user buffer size exceeds transp.buffer
- this means:
* For "small message" protocols, we continue using `transp.buffer` as
before
* For "header+message" protocols the header and beginning of message
will be read with a single `recv` call (assuming it fits in
`transp.buffer`), then we switch to bulk reading mode.
Other misc changes:
* also treat EAGAIN as EWOULDBLOCK (on systems where they differ)
* add thread-to-thread throughput benchmark
* clarify windows TODO
Add support for mTLS client certificate authentication
in `newTLSClientAsyncStream`. Both RSA and EC key types are supported.
* Add EC test
---------
Co-authored-by: Jacek Sieka <jacek@status.im>
* `ReadClosed`/`WriteClosed` -> `Closed` (they're always set together)
* `atEof` doesn't need to check transport state
* remove redundant while looping in `readStreamLoop` (it's done by
`handleEintr`)
* move error handling to `resumeRead` / `resumeWrite` and make usre it
matches loop version
* clear reader future on read cancellation (no need to hold on to
memory)
* document closeFd
* Update mdbook-open-on-gh version.
2.4.1 would crash during build locally.
* Fix path to async_procs.md.
* Add the guide to update the book.
---------
Co-authored-by: Jacek Sieka <jacek@status.im>
Notable features include:
* Continuous and discrete replenishment modes for reate limiter
* `AsyncSemaphore` - similar to an `AsyncLock` but can be acquired
multiple times
* `cancelAndWait` that accepts multiple futures
In addition, there are several smaller efficiency improvements and
bugfixes.
Semantically, these two functions behave the same though race offers an
additional overload for heterogenous futures (and had a less expressive
return type - fixed here).
* fix `race` leak where callback would not be removed from unfinished
futures
* reduce memory allocations for varargs version
* return full future type in `race` when possible
In the future, we may want to deprecate either `one` or `race` since
they are redundant at this point.
* Fix original TokenBucket, consolidate the behaviors of chronos/TokenBucket and waku/TokenBucket (compensating) with no interface change
* TokenBucket extended with waku's strict mode replenish, that does not allow refill just after period boundary elapsed.
* Adjust for new chat-sdk needs, added setState and getAvailableCapacity with small refactoring
* Better comments
* rename for better self explain code, added more explanatory comments upon code review finding
* Document TokenBucket with detailed samples of Balanced mode replenishment algorithm, extended TokenBucket unit test
* Address review comments, renaming the replenish modes, removed old/new algo comparison from documentation, fix Discrete mode update calculation to properly calculate correct last update time by period distance calculation.
* protect from devide by zero, code style fix
* Make (re)setState close pending request before reseting state
* Polishing interface, move discrete mode initial start time setup into ctor and remove resetState to have a cleaner and consistent interface for both usage mode
* Addressing review observation, keep only singel new ctor and leave defaults to match former use.
* Removing leftovers and confirm with coding guideline
* refactor rate limiter implementation
* make sure that `tryConsume` respects queued `consume` requests
* refill tokens the same way regardless of replenish mode and source of
tokens (manual/time-based/cancellation)
* whem manually replenishing, compute budget cap after satisfying queued
requests
* don't run worker if fill duration is 0
* add some docs
* int vs int64
* Fix Discrete mode time window drift, added tests
* wait for sleeper/waiter
---------
Co-authored-by: Jacek Sieka <jacek@status.im>
In case there are many waiters, this helps preserve O(1) performance -
also clear out some cancelled waiters more aggressively, in case they
appear in order.
Jacek Sieka