36 Commits

Author SHA1 Message Date
Eric
40c243a080
Colorise stdout from test runs
Adds terminal colour to [OK], [FAILED], and [Suite]

# Conflicts:
#	tests/integration/testmanager.nim
2025-06-05 15:58:04 +10:00
Eric
72d2931183
Refactor process launch
Refactor to keep a process handle so that the processes can be stopped on timeout.
2025-06-05 15:58:04 +10:00
Eric
bc5fdd49be
run tests async from start so that stop can cancel all tracked futures 2025-06-05 15:58:03 +10:00
Eric
cddb407ddc
clean up 2025-06-05 15:58:02 +10:00
Eric
c59fa9f1e5
increase port gap by 1000 to avoid clashes in windows 2025-06-05 15:58:02 +10:00
Eric
25ce97e81e
update Make flag to simply debug
DEBUG -> enables DebugCodexNodes, DebugTestHarness, and ShowContinuousStatusUpdates
DEBUG_HARDHAT -> enables DebugHardhat
PARALLEL -> enables EnableParallelTests

Additionally, when DEBUG is enabled, all integration tests debug configs are enabled for Codex nodes, the Codex node output is printed with the test output (not interleaved), and the Codex node output is logged to file in `tests/integrations/logs/<starttime>__IntegrationTests/<integration_test_name>/<suite_name>/<testname>/<role>_<idx>.log`.
When DEBUG_HARDHAT is enabled, all hardhat output is printed with the test output (not interleaved), and the output is also written to a log file in `tests/integrations/logs/<starttime>__IntegrationTests/<integration_test_name>/hardhat.log
2025-06-05 15:58:02 +10:00
Eric
29df100e93
cancel processes soon
Windows really does not like to terminate processes, so if we try to cancel the process, we will hang in Windows.
2025-06-05 15:58:01 +10:00
Eric
92ca6de95b
fix: hardhot not being stopped when test times out 2025-06-05 15:58:01 +10:00
Eric
3554c772cc
handle cancellations for race and allFutures 2025-06-05 15:58:00 +10:00
Eric
52479ef0e8
nil check hardhat process before stopping 2025-06-05 15:58:00 +10:00
Eric
d260e6350c
fix exit status 2025-06-05 15:57:59 +10:00
Eric
0dcf8ded69
indicate with exit code 1 if the tests failed 2025-06-05 15:57:59 +10:00
Eric
7d3013f3b3
fix: pass show continuous updates into test manager instance 2025-06-05 15:57:59 +10:00
Eric
e902df5365
undo changes done for CI debugging 2025-06-05 15:57:58 +10:00
Eric
d3245c3aa8
Back off port inc in the hopes that this is causing issues with CI 2025-06-05 15:57:57 +10:00
Eric
e4ebfaafac
use tracked futures for showContinuousStatusUpdates 2025-06-05 15:57:56 +10:00
Eric
1218c493be
formatting 2025-06-05 15:57:56 +10:00
Eric
756af9b2ab
remove unneeded sleep 2025-06-05 15:57:54 +10:00
Eric
982af54648
increase port distance per test run 2025-06-05 15:57:54 +10:00
Eric
35964f6e73
add constructor param for continuous status update in terminal
# Conflicts:
#	tests/testIntegration.nim
2025-06-05 15:57:54 +10:00
Eric
ff1eb8b05e
ensure timeouts work
Work around the AsyncProcess timeout not working correctly
2025-06-05 15:57:54 +10:00
Eric
78260feda3
continually show test status updates
allows user to know which tests are still running
2025-06-05 15:57:54 +10:00
Eric
855ba637e3
cleanup
# Conflicts:
#	tests/testIntegration.nim
2025-06-05 15:57:53 +10:00
Eric
c8d69b646c
lock hardhat start to avoid intermittent start failures 2025-06-05 15:57:53 +10:00
Eric
44faac5a97
clean up 2025-06-05 15:57:53 +10:00
Eric
5db892adf3
print hardhat testmanager errors and hardhat stdout when there is an error starting hardhat 2025-06-05 15:57:53 +10:00
Eric
0d186c69a1
hang manager off IntegrationTest, cleanup
- add a TestManager property to IntegrationTest, so manager does not need to be passed into all functions
- cleanup:
  - remove unneeded stopHardhat function
  - add hardhat instance to manager outside of startHardhat
2025-06-05 15:57:53 +10:00
Eric
878a384794
reinstate removed types 2025-06-05 15:57:52 +10:00
Eric
f957c58c58
Add additional logging info print outs
# Conflicts:
#	tests/integration/testcli.nim
2025-06-05 15:57:52 +10:00
Eric
f9e26bde31
stop hardhat and show logs
# Conflicts:
#	tests/integration/hardhatprocess.nim
2025-06-05 15:57:52 +10:00
Eric
b841f04602
default test name to the filename of the integration test
# Conflicts:
#	tests/testIntegration.nim
2025-06-05 15:57:52 +10:00
Eric
8cce661d76
print start/finish stderr/stdout markers
# Conflicts:
#	tests/testIntegration.nim
2025-06-05 15:57:52 +10:00
Eric
bbea4371da
clean up 2025-06-05 15:57:51 +10:00
Eric
0e9439b201
stdout formatting for test results
# Conflicts:
#	tests/testIntegration.nim
2025-06-05 15:57:51 +10:00
Eric
38f752667d
test summary formatting 2025-06-05 15:57:51 +10:00
Eric
092de71d9c
parallel tests running
- need to test with longer tests to ensure the parallelisation is truly happening
- is the +10 hardhat port needed?
- try with more integration tests

# Conflicts:
#	tests/integration/hardhatprocess.nim
#	tests/integration/multinodes.nim
#	tests/integration/testcli.nim
#	tests/testIntegration.nim
2025-06-05 15:57:51 +10:00