nim-dagger/codex/streams/asyncstreamwrapper.nim

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

130 lines
3.6 KiB
Nim
Raw Normal View History

## Nim-LibP2P
## Copyright (c) 2019 Status Research & Development GmbH
## Licensed under either of
## * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
## * MIT license ([LICENSE-MIT](LICENSE-MIT))
## at your option.
## This file may not be copied, modified, or distributed except according to
## those terms.
import pkg/upraises
push:
{.upraises: [].}
import pkg/chronos
import pkg/libp2p
feat: create logging proxy (#663) * implement a logging proxy The logging proxy: - prevents the need to import chronicles (as well as export except toJson), - prevents the need to override `writeValue` or use or import nim-json-seralization elsewhere in the codebase, allowing for sole use of utils/json for de/serialization, - and handles json formatting correctly in chronicles json sinks * Rename logging -> logutils to avoid ambiguity with common names * clean up * add setProperty for JsonRecord, remove nim-json-serialization conflict * Allow specifying textlines and json format separately Not specifying a LogFormat will apply the formatting to both textlines and json sinks. Specifying a LogFormat will apply the formatting to only that sink. * remove unneeded usages of std/json We only need to import utils/json instead of std/json * move serialization from rest/json to utils/json so it can be shared * fix NoColors ambiguity Was causing unit tests to fail on Windows. * Remove nre usage to fix Windows error Windows was erroring with `could not load: pcre64.dll`. Instead of fixing that error, remove the pcre usage :) * Add logutils module doc * Shorten logutils.formatIt for `NBytes` Both json and textlines formatIt were not needed, and could be combined into one formatIt * remove debug integration test config debug output and logformat of json for integration test logs * Use ## module doc to support docgen * bump nim-poseidon2 to export fromBytes Before the changes in this branch, fromBytes was likely being resolved by nim-stew, or other dependency. With the changes in this branch, that dependency was removed and fromBytes could no longer be resolved. By exporting fromBytes from nim-poseidon, the correct resolution is now happening. * fixes to get compiling after rebasing master * Add support for Result types being logged using formatIt
2024-01-23 18:35:03 +11:00
import ../logutils
logScope:
topics = "libp2p asyncstreamwrapper"
const AsyncStreamWrapperName* = "AsyncStreamWrapper"
type AsyncStreamWrapper* = ref object of LPStream
reader*: AsyncStreamReader
writer*: AsyncStreamWriter
method initStream*(self: AsyncStreamWrapper) =
if self.objName.len == 0:
self.objName = AsyncStreamWrapperName
procCall LPStream(self).initStream()
proc new*(
C: type AsyncStreamWrapper,
reader: AsyncStreamReader = nil,
writer: AsyncStreamWriter = nil,
): AsyncStreamWrapper =
## Create new instance of an asynchronous stream wrapper
feat: create logging proxy (#663) * implement a logging proxy The logging proxy: - prevents the need to import chronicles (as well as export except toJson), - prevents the need to override `writeValue` or use or import nim-json-seralization elsewhere in the codebase, allowing for sole use of utils/json for de/serialization, - and handles json formatting correctly in chronicles json sinks * Rename logging -> logutils to avoid ambiguity with common names * clean up * add setProperty for JsonRecord, remove nim-json-serialization conflict * Allow specifying textlines and json format separately Not specifying a LogFormat will apply the formatting to both textlines and json sinks. Specifying a LogFormat will apply the formatting to only that sink. * remove unneeded usages of std/json We only need to import utils/json instead of std/json * move serialization from rest/json to utils/json so it can be shared * fix NoColors ambiguity Was causing unit tests to fail on Windows. * Remove nre usage to fix Windows error Windows was erroring with `could not load: pcre64.dll`. Instead of fixing that error, remove the pcre usage :) * Add logutils module doc * Shorten logutils.formatIt for `NBytes` Both json and textlines formatIt were not needed, and could be combined into one formatIt * remove debug integration test config debug output and logformat of json for integration test logs * Use ## module doc to support docgen * bump nim-poseidon2 to export fromBytes Before the changes in this branch, fromBytes was likely being resolved by nim-stew, or other dependency. With the changes in this branch, that dependency was removed and fromBytes could no longer be resolved. By exporting fromBytes from nim-poseidon, the correct resolution is now happening. * fixes to get compiling after rebasing master * Add support for Result types being logged using formatIt
2024-01-23 18:35:03 +11:00
##
let stream = C(reader: reader, writer: writer)
stream.initStream()
return stream
template withExceptions(body: untyped) =
try:
body
except CancelledError as exc:
raise exc
except AsyncStreamIncompleteError:
# for all intents and purposes this is an EOF
raise newLPStreamIncompleteError()
except AsyncStreamLimitError:
raise newLPStreamLimitError()
except AsyncStreamUseClosedError:
raise newLPStreamEOFError()
except AsyncStreamError as exc:
raise newException(LPStreamError, exc.msg)
method readOnce*(
self: AsyncStreamWrapper, pbytes: pointer, nbytes: int
Chore/update nim version (#1052) * Move to version 2.0.6 * Update nim-confutils submodule to latest version * Update dependencies * Update Nim version to 2.0.12 * Add gcsafe pragma * Add missing import * Update specific conf for Nim 2.x * Fix method signatures * Revert erasure coding attempt to fix bug * More gcsafe pragma * Duplicate code from libp2p because it is not exported anymore * Fix camelcase function names * Use alreadySeen because need is not a bool anymore * newLPStreamReadError does not exist anymore so use another error * Replace ValidIpAddress by IpAddress * Add gcsafe pragma * Restore maintenance parameter deleted by mistake when removing esasure coding fix attempt code * Update method signatures * Copy LPStreamReadError code from libp2p which was removed * Fix camel case * Fix enums in tests * Fix camel case * Extract node components to a variable to make Nim 2 happy * Update the tests using ValidIpAddress to IpAddress * Fix cast for value which is already an option * Set nim version to 2.0.x for CI * Set nim version to 2.0.x for CI * Move to miniupnp version 2.2.4 to avoid symlink error * Set core.symlinks to false for Windows for miniupnp >= 2.2.5 support * Update to Nim 2.0.14 * Update CI nim versions to 2.0.14 * Try with GCC 14 * Replace apt-fast by apt-get * Update ubuntu runner to latest * Use Ubuntu 20.04 for coverage * Disable CI cache for coverage * Add coverage property description * Remove commented test * Check the node value of seen instead of using alreadySeen * Fix the merge. The taskpool work was reverted. * Update nim-ethers submodule * Remove deprecated ValidIpAddress. Fix missing case and imports. * Fix a weird issue where nim-confutils cannot find NatAny * Fix tests and remove useless static keyword
2025-01-10 15:12:37 +01:00
): Future[int] {.async: (raises: [CancelledError, LPStreamError]).} =
trace "Reading bytes from reader", bytes = nbytes
if isNil(self.reader):
error "Async stream wrapper reader nil"
raiseAssert("Async stream wrapper reader nil")
if self.atEof:
raise newLPStreamEOFError()
withExceptions:
return await self.reader.readOnce(pbytes, nbytes)
proc completeWrite(
self: AsyncStreamWrapper, fut: Future[void], msgLen: int
): Future[void] {.async.} =
withExceptions:
await fut
method write*(self: AsyncStreamWrapper, msg: seq[byte]): Future[void] =
# Avoid a copy of msg being kept in the closure created by `{.async.}` as this
# drives up memory usage
trace "Writing bytes to writer", bytes = msg.len
if isNil(self.writer):
error "Async stream wrapper writer nil"
raiseAssert("Async stream wrapper writer nil")
if self.closed:
let fut = newFuture[void]("asyncstreamwrapper.write.closed")
fut.fail(newLPStreamClosedError())
return fut
self.completeWrite(self.writer.write(msg, msg.len), msg.len)
method closed*(self: AsyncStreamWrapper): bool =
var
readerClosed = true
writerClosed = true
if not isNil(self.reader):
readerClosed = self.reader.closed
if not isNil(self.writer):
writerClosed = self.writer.closed
return readerClosed and writerClosed
method atEof*(self: AsyncStreamWrapper): bool =
self.reader.atEof()
Chore/update nim version (#1052) * Move to version 2.0.6 * Update nim-confutils submodule to latest version * Update dependencies * Update Nim version to 2.0.12 * Add gcsafe pragma * Add missing import * Update specific conf for Nim 2.x * Fix method signatures * Revert erasure coding attempt to fix bug * More gcsafe pragma * Duplicate code from libp2p because it is not exported anymore * Fix camelcase function names * Use alreadySeen because need is not a bool anymore * newLPStreamReadError does not exist anymore so use another error * Replace ValidIpAddress by IpAddress * Add gcsafe pragma * Restore maintenance parameter deleted by mistake when removing esasure coding fix attempt code * Update method signatures * Copy LPStreamReadError code from libp2p which was removed * Fix camel case * Fix enums in tests * Fix camel case * Extract node components to a variable to make Nim 2 happy * Update the tests using ValidIpAddress to IpAddress * Fix cast for value which is already an option * Set nim version to 2.0.x for CI * Set nim version to 2.0.x for CI * Move to miniupnp version 2.2.4 to avoid symlink error * Set core.symlinks to false for Windows for miniupnp >= 2.2.5 support * Update to Nim 2.0.14 * Update CI nim versions to 2.0.14 * Try with GCC 14 * Replace apt-fast by apt-get * Update ubuntu runner to latest * Use Ubuntu 20.04 for coverage * Disable CI cache for coverage * Add coverage property description * Remove commented test * Check the node value of seen instead of using alreadySeen * Fix the merge. The taskpool work was reverted. * Update nim-ethers submodule * Remove deprecated ValidIpAddress. Fix missing case and imports. * Fix a weird issue where nim-confutils cannot find NatAny * Fix tests and remove useless static keyword
2025-01-10 15:12:37 +01:00
method closeImpl*(self: AsyncStreamWrapper) {.async: (raises: []).} =
try:
trace "Shutting down async chronos stream"
if not self.closed():
if not isNil(self.reader) and not self.reader.closed():
await self.reader.closeWait()
if not isNil(self.writer) and not self.writer.closed():
await self.writer.closeWait()
trace "Shutdown async chronos stream"
except CancelledError as exc:
Chore/update nim version (#1052) * Move to version 2.0.6 * Update nim-confutils submodule to latest version * Update dependencies * Update Nim version to 2.0.12 * Add gcsafe pragma * Add missing import * Update specific conf for Nim 2.x * Fix method signatures * Revert erasure coding attempt to fix bug * More gcsafe pragma * Duplicate code from libp2p because it is not exported anymore * Fix camelcase function names * Use alreadySeen because need is not a bool anymore * newLPStreamReadError does not exist anymore so use another error * Replace ValidIpAddress by IpAddress * Add gcsafe pragma * Restore maintenance parameter deleted by mistake when removing esasure coding fix attempt code * Update method signatures * Copy LPStreamReadError code from libp2p which was removed * Fix camel case * Fix enums in tests * Fix camel case * Extract node components to a variable to make Nim 2 happy * Update the tests using ValidIpAddress to IpAddress * Fix cast for value which is already an option * Set nim version to 2.0.x for CI * Set nim version to 2.0.x for CI * Move to miniupnp version 2.2.4 to avoid symlink error * Set core.symlinks to false for Windows for miniupnp >= 2.2.5 support * Update to Nim 2.0.14 * Update CI nim versions to 2.0.14 * Try with GCC 14 * Replace apt-fast by apt-get * Update ubuntu runner to latest * Use Ubuntu 20.04 for coverage * Disable CI cache for coverage * Add coverage property description * Remove commented test * Check the node value of seen instead of using alreadySeen * Fix the merge. The taskpool work was reverted. * Update nim-ethers submodule * Remove deprecated ValidIpAddress. Fix missing case and imports. * Fix a weird issue where nim-confutils cannot find NatAny * Fix tests and remove useless static keyword
2025-01-10 15:12:37 +01:00
error "Error received cancelled error when closing chronos stream", msg = exc.msg
except CatchableError as exc:
trace "Error closing async chronos stream", msg = exc.msg
await procCall LPStream(self).closeImpl()