965 Commits

Author SHA1 Message Date
Eric
a652c8dedb
fix makefile: DEBUG enables DebugHardhat 2025-11-04 20:59:56 +11:00
Eric
c4459f2322
Remove new implementation of eventuallySafe by overidding the default params of eventually
With this fix in, there is no need to use the asynctest update that sets longer defaults for eventually, so downgrade asynctest
2025-11-04 20:59:56 +11:00
Eric
866d754de6
safeEventually > eventuallySafe 2025-11-04 20:59:48 +11:00
Eric
93832a6ef1
fix: forcefully kill windows hardhat processes after termination
On windows, termination of hardhat processes would not actually kill the process, and then closing the process' streams would then hang the calling nim process. To get around this, the process is now killed externally using a script, winkillhardhat.sh. This script first queries open processes by inspecting the command line value of all "node.exe" processes, searching for "vendor/codex-contracts-eth" and for the port parameter it was started with. After querying, the process is killed using the `Stop-Process` powershell command (passing the pid of the windows process).
2025-11-04 20:18:44 +11:00
Arnaud
1663bd224f
Reduce the number of requests for asserts in tests 2025-11-04 20:18:44 +11:00
Eric
98b42f4319
formatting 2025-11-04 20:16:05 +11:00
Eric
498c0c42cb
attempt to kill windows hardhat process
Windows hangs when attempting to hardhat's process streams, so try to kill the process externally.
TODO: This doesn't actually kill the process, as the pid given by chronos is an msys2 pid, and the command is used to kill a windows process. `ps -ef` in msys2 also doesn't show hardhat running as a process, so the only way to kill the process is to kill it with the windows pid. So we need to figure out a way to get a windows pid from the msys2 pid.
2025-11-04 20:16:05 +11:00
Eric
98f3ff76e0
add new line to stdout/stderr logs when logging lines 2025-11-04 20:16:05 +11:00
Eric
05fae04db6
fix compilation error 2025-11-04 20:16:04 +11:00
Eric
c93694ac36
write to stdout/stderr as the stream is read, rearrange node stop logging 2025-11-04 20:16:04 +11:00
Eric
58b70a5ee3
remove unneeded exception from start raises 2025-11-04 20:16:04 +11:00
Eric
2659913c2d
Add "starting test" log to multinodes logging output for context 2025-11-04 20:16:04 +11:00
Eric
dc3ea9b16d
unittest2 > unittest1 2025-11-04 20:16:04 +11:00
Eric
4e3673edd2
remove noCancel from node startup in multinodes 2025-11-04 20:16:04 +11:00
Eric
389ce830d9
Remove noCancel from captureProcessOutput call 2025-11-04 20:16:03 +11:00
Eric
c1b36e7f21
Continuously read from stdout/stderr to prevent buffers from filling 2025-11-04 20:16:03 +11:00
Eric
e86f596133
reinstate logging after it also failed on windows and ci 2025-11-04 20:16:03 +11:00
Eric
2d8783f49e
disable all logging to see if windows passes 2025-11-04 20:16:03 +11:00
Eric
c22cc10d48
Move test assertions at end of start so that test.status can be set after timeout 2025-11-04 20:16:03 +11:00
Eric
f3997dd95e
fix rebase on top of async http client change 2025-11-04 20:16:03 +11:00
Eric
7acb05d5b1
Remove debug from twonodesconfig 2025-11-04 20:16:03 +11:00
Eric
828fcf6edb
Add wait to stdout/stderr stream future 2025-11-04 20:16:02 +11:00
Eric
8eb5013b5a
fix windows issue where reading stdout/stderr hangs forever 2025-11-04 20:16:02 +11:00
Eric
6d1b924c41
ensure runTests doesn't raise CancelledError
TrackedFutures now only accepts Futures that do not raise any exceptions
2025-11-04 20:16:02 +11:00
Eric
338ee181d3
Remove DebugCodexNodes
- Codex nodes will log to file by default, so this parameter could be removed.
- Add NoCodexLogFilters define, that unsets chronicles' topic filters when logging codex nodes chronicles logs to file
2025-11-04 20:16:02 +11:00
Eric
6057d5ac45
do not re-raise CancelledError as convertError not used in async procs 2025-11-04 20:16:02 +11:00
Eric
2695b46dc4
re-raise CancelledError 2025-11-04 20:16:02 +11:00
Eric
b9dd464a93
readd prevention of hanging when closing process streams on windows 2025-11-04 20:16:02 +11:00
Eric
1dc20de0c2
Add debugging and assertions after test run
Mostly to debug in windows
2025-11-04 20:16:01 +11:00
Eric
5ededbb247
CI yml fix for gcc-14 2025-11-04 20:16:01 +11:00
Eric
d410455b33
Update NBS action.yml in CI 2025-11-04 20:16:01 +11:00
Eric
0bdba3fb99
formatting 2025-11-04 20:16:01 +11:00
Eric
da6097155c
formatting 2025-11-04 20:16:01 +11:00
Eric
919f46b2e1
Fix process hanging after restart 2025-11-04 20:16:01 +11:00
Eric
dff1ca961f
refactor: debug and logging
- Move test manager values to config object
- Increase codex port separation (between api and disc ports) in an attempt to prevent overlap across tests (ie Test1: api=8000(tcp), disc=9000(udp), and Test2: api=9000(tcp), disc=10000(udp))
- print stderr when exitcode == 1 if there's < 3 lines of stdout
- Logging:
  - Always write test manager harness chronicles logs to file, ie testmanager.chronicles.log in the root of the `integration/logs/<run name>` dir
  - Always write individual test stdout to file, ie `<test file name>.stdout.log` in the root of the `integration/logs/<run name>/<test file name>` dir
  - On error, print stderr to screen and write stderr to file. Or on failure, if stdout is sufficiently short, write stderr to screen and file in `integration/logs/<run name>/<test file name>/<test file name>.stderr.log`
  - When debugging, ie DebugCodexNodes == true
- Removes DebugTestHarness from controlling anything other than printing chronicles output from the testmanager to the terminal
- Now, if DebugCodexNodes is set to true:
  - Codex node (chronicles) output for multinodesuite tests is logged to file, eg `integration/logs/<run name>/<test file name>/<test name>/<role>_<idx>.log`
  - Codex chronicles output is logged to stdout, which also written to file (see above)
2025-11-04 20:16:01 +11:00
Eric
a1ea23007f
enable more logging to file
- test harness chronicles logs to log file
- test process stdout and stderr to log file for each integration test file

Note: does not work when DEBUG=1 is not present in make command
2025-11-04 20:16:00 +11:00
Eric
e83a692b03
Ensure started tests are torn down properly on timeout 2025-11-04 20:15:16 +11:00
Eric
7d29afbae8
filter out log topics that are making too much noise
- set disabled topics during codex compilation due to lack of support for disabling chronicles topics at runtime in codex
- ensure that TRACE logs are set for nodes running in multinodesuite when DebugCodexNodes is set
- add TRACE loglevel to testcli
2025-11-04 20:15:16 +11:00
Slava
14d4dfae38
ci: check runner for parallel integration tests and use value from vars 2025-11-04 20:15:16 +11:00
Eric
ff62751cce
Colorise stdout from test runs
Adds terminal colour to [OK], [FAILED], and [Suite]

# Conflicts:
#	tests/integration/testmanager.nim
2025-11-04 20:12:44 +11:00
Eric
53a3a2ae18
Refactor process launch
Refactor to keep a process handle so that the processes can be stopped on timeout.
2025-11-04 20:12:44 +11:00
Eric
8a8030c298
run tests async from start so that stop can cancel all tracked futures 2025-11-04 20:12:44 +11:00
Eric
d968ef9a39
Add proper async exception tracking to multinodesuite 2025-11-04 20:12:44 +11:00
Eric
6bdad7db7f
bump chronos to 4.0.4
potentially fixes an issue on macos where process exit codes were reported -1
2025-11-04 20:12:43 +11:00
Eric
821393d025
formatting 2025-11-04 20:12:43 +11:00
Eric
ab8e5418b5
CI: enable artefact upload for parallel tests 2025-11-04 20:12:43 +11:00
Eric
e46f67bfb7
run npm install for parallel tests in ci 2025-11-04 20:12:43 +11:00
Eric
0f2467a820
Allow debug runs from the github UI 2025-11-04 20:12:43 +11:00
Eric
afe31d7319
clean up 2025-11-04 20:12:42 +11:00
Eric
7c2254a249
increase port gap by 1000 to avoid clashes in windows 2025-11-04 20:12:11 +11:00