147 Commits

Author SHA1 Message Date
Eric
54fb0d7172
increase port gap by 1000 to avoid clashes in windows 2025-02-19 15:43:30 +11:00
Eric
6c513260ad
lock codex port selection in multinodesuites
Codex api/disc ports were clashing in windows because `lastUsedCodexApiPort` and `lastUsedCodeexDiscPort` were not locked when read and written, while client and provider nodes were started concurrently (async).
2025-02-19 15:43:29 +11:00
Eric
e39497bf37
add test name to datadir path
avoids reusing datadirs between tests
2025-02-19 15:43:29 +11:00
Eric
4119dd6a53
update Make flag to simply debug
DEBUG -> enables DebugCodexNodes, DebugTestHarness, and ShowContinuousStatusUpdates
DEBUG_HARDHAT -> enables DebugHardhat
PARALLEL -> enables EnableParallelTests

Additionally, when DEBUG is enabled, all integration tests debug configs are enabled for Codex nodes, the Codex node output is printed with the test output (not interleaved), and the Codex node output is logged to file in `tests/integrations/logs/<starttime>__IntegrationTests/<integration_test_name>/<suite_name>/<testname>/<role>_<idx>.log`.
When DEBUG_HARDHAT is enabled, all hardhat output is printed with the test output (not interleaved), and the output is also written to a log file in `tests/integrations/logs/<starttime>__IntegrationTests/<integration_test_name>/hardhat.log
2025-02-19 15:43:29 +11:00
Eric
22b4847179
cancel processes soon
Windows really does not like to terminate processes, so if we try to cancel the process, we will hang in Windows.
2025-02-19 15:43:29 +11:00
Eric
7628aa529c
fix: hardhot not being stopped when test times out 2025-02-19 15:43:28 +11:00
Eric
a1569c5a16
fix: use the last used port instead of the globally defined port 2025-02-19 15:43:28 +11:00
Eric
8468fc4c74
Increase hardhat, codex api, and codex discovery ports for each subsequent test
Previously, each test would start with a port specified by a global const and overwritable using `define` cli flag. This seems to be causing some port clashes that `nextFreePort` is not picking up. Instead of relying on `nextFreePort`, do not reuse port numbers for any of the tests.
2025-02-19 15:43:27 +11:00
Eric
3ec996699d
enable debug logs for testpurchasing since the failure in ci 2025-02-19 15:43:27 +11:00
Eric
b0ac3fd180
fix waiting for hardhat node process exit on windows 2025-02-19 15:43:27 +11:00
Eric
5fbd3d167a
handle cancellations for race and allFutures 2025-02-19 15:43:26 +11:00
Eric
0c53371cc0
fix hardhat running on windows
Solution was two-fold:
1. Run the hardhat.cmd script instead of hardhat
2. Use the absolute path to the script executable instead of relative to the working directory
2025-02-19 15:43:26 +11:00
Eric
1f5fca88ca
Bubble errors from hardhat start
- bubble errors from hardhatprocess.start (instead of just logging)
- push raises: [] in all nodeprocess procs/methods to avoid leaking `Exception` exception types in method overrides
2025-02-19 15:43:26 +11:00
Eric
2bbf903062
nil check hardhat process before stopping 2025-02-19 15:43:25 +11:00
Eric
943b552e5f
fix exit status 2025-02-19 15:43:25 +11:00
Eric
84d04815b9
indicate with exit code 1 if the tests failed 2025-02-19 15:43:24 +11:00
Eric
ba2ce9eef2
fix: pass show continuous updates into test manager instance 2025-02-19 15:43:24 +11:00
Eric
b0fd67ca53
undo changes done for CI debugging 2025-02-19 15:43:22 +11:00
Eric
dd1a761a5e
Back off port inc in the hopes that this is causing issues with CI 2025-02-19 15:43:21 +11:00
Eric
9ebc95fe89
formatting
trying to make nph happy
2025-02-19 15:43:20 +11:00
Eric
a99949228b
formatting 2025-02-19 15:43:20 +11:00
Eric
8e8cbc511f
clean up 2025-02-19 15:43:20 +11:00
Eric
7132983e1d
use tracked futures for showContinuousStatusUpdates 2025-02-19 15:43:19 +11:00
Eric
b290ee758e
formatting 2025-02-19 15:43:19 +11:00
Eric
583792f52c
fix rebase, formatting 2025-02-19 15:41:47 +11:00
Eric
849fd00a2c
fix rebase 2025-02-19 15:41:47 +11:00
Eric
ca3aeb28cf
Use absolute path for binary when checking for existence 2025-02-19 15:41:47 +11:00
Eric
567a8444cb
fail early with message when hardhat binary not found
# Conflicts:
#	tests/integration/hardhatprocess.nim
2025-02-19 15:41:47 +11:00
Eric
cc8f00e707
remove unneeded sleep 2025-02-19 15:41:46 +11:00
Eric
9b0eca0815
make onoutputline callback gcsafe 2025-02-19 15:41:46 +11:00
Eric
aea6c99a69
increase port distance per test run 2025-02-19 15:41:46 +11:00
Eric
7f058deece
add constructor param for continuous status update in terminal
# Conflicts:
#	tests/testIntegration.nim
2025-02-19 15:41:45 +11:00
Eric
749f65b134
fix nodeprocess not exiting 2025-02-19 15:41:45 +11:00
Eric
8501220fda
ensure timeouts work
Work around the AsyncProcess timeout not working correctly
2025-02-19 15:41:45 +11:00
Eric
92513e083e
continually show test status updates
allows user to know which tests are still running
2025-02-19 15:41:45 +11:00
Eric
2ef044dfcb
cleanup
# Conflicts:
#	tests/testIntegration.nim
2025-02-19 15:41:44 +11:00
Eric
a46bf719ef
lock hardhat start to avoid intermittent start failures 2025-02-19 15:41:44 +11:00
Eric
ff8ac244bd
clean up 2025-02-19 15:41:44 +11:00
Eric
beb270de3d
print hardhat testmanager errors and hardhat stdout when there is an error starting hardhat 2025-02-19 15:41:43 +11:00
Eric
f4712684ba
hang manager off IntegrationTest, cleanup
- add a TestManager property to IntegrationTest, so manager does not need to be passed into all functions
- cleanup:
  - remove unneeded stopHardhat function
  - add hardhat instance to manager outside of startHardhat
2025-02-19 15:41:43 +11:00
Eric
f55831a2c8
reinstate removed types 2025-02-19 15:41:43 +11:00
Eric
0a8487618a
Add additional logging info print outs
# Conflicts:
#	tests/integration/testcli.nim
2025-02-19 15:41:42 +11:00
Eric
34f445993d
expect process exit code
prevents showing error in the logs when an expected process exit code is encountered

# Conflicts:
#	tests/integration/testcli.nim
2025-02-19 15:41:42 +11:00
Eric
6a18238205
stop hardhat and show logs
# Conflicts:
#	tests/integration/hardhatprocess.nim
2025-02-19 15:41:42 +11:00
Eric
e4c035651f
default test name to the filename of the integration test
# Conflicts:
#	tests/testIntegration.nim
2025-02-19 15:41:42 +11:00
Eric
cf9134861e
print start/finish stderr/stdout markers
# Conflicts:
#	tests/testIntegration.nim
2025-02-19 15:41:41 +11:00
Eric
1400848f86
clean up 2025-02-19 15:41:41 +11:00
Eric
ce4b1d439c
stdout formatting for test results
# Conflicts:
#	tests/testIntegration.nim
2025-02-19 15:41:41 +11:00
Eric
7252a32579
prevent errors during node shutdown 2025-02-19 15:41:40 +11:00
Eric
40142fa684
test summary formatting 2025-02-19 15:41:40 +11:00