In situations like timeouts, windows will hang when attempting to close the test process streams. In this case, we have to force kill the test process externally. This is the same process as force killing hardhat nodes after they are already terminated, but windows refuses hangs when closing their process streams. This commit creates a forceKillProcess utility that allows a process to be killed by its process name and matching commandline criteria, like TestId (for test process) or --port (for hardhat)
Hardhat output is logged to file in hardhat.log for each test, and printing to screen is not necessarily needed as it is already logged to file and can create clutter in the output, so stop writing logging output to memory and writing to screen.
On windows, termination of hardhat processes would not actually kill the process, and then closing the process' streams would then hang the calling nim process. To get around this, the process is now killed externally using a script, winkillhardhat.sh. This script first queries open processes by inspecting the command line value of all "node.exe" processes, searching for "vendor/codex-contracts-eth" and for the port parameter it was started with. After querying, the process is killed using the `Stop-Process` powershell command (passing the pid of the windows process).
Windows hangs when attempting to hardhat's process streams, so try to kill the process externally.
TODO: This doesn't actually kill the process, as the pid given by chronos is an msys2 pid, and the command is used to kill a windows process. `ps -ef` in msys2 also doesn't show hardhat running as a process, so the only way to kill the process is to kill it with the windows pid. So we need to figure out a way to get a windows pid from the msys2 pid.
- Codex nodes will log to file by default, so this parameter could be removed.
- Add NoCodexLogFilters define, that unsets chronicles' topic filters when logging codex nodes chronicles logs to file
- Move test manager values to config object
- Increase codex port separation (between api and disc ports) in an attempt to prevent overlap across tests (ie Test1: api=8000(tcp), disc=9000(udp), and Test2: api=9000(tcp), disc=10000(udp))
- print stderr when exitcode == 1 if there's < 3 lines of stdout
- Logging:
- Always write test manager harness chronicles logs to file, ie testmanager.chronicles.log in the root of the `integration/logs/<run name>` dir
- Always write individual test stdout to file, ie `<test file name>.stdout.log` in the root of the `integration/logs/<run name>/<test file name>` dir
- On error, print stderr to screen and write stderr to file. Or on failure, if stdout is sufficiently short, write stderr to screen and file in `integration/logs/<run name>/<test file name>/<test file name>.stderr.log`
- When debugging, ie DebugCodexNodes == true
- Removes DebugTestHarness from controlling anything other than printing chronicles output from the testmanager to the terminal
- Now, if DebugCodexNodes is set to true:
- Codex node (chronicles) output for multinodesuite tests is logged to file, eg `integration/logs/<run name>/<test file name>/<test name>/<role>_<idx>.log`
- Codex chronicles output is logged to stdout, which also written to file (see above)
- test harness chronicles logs to log file
- test process stdout and stderr to log file for each integration test file
Note: does not work when DEBUG=1 is not present in make command
- set disabled topics during codex compilation due to lack of support for disabling chronicles topics at runtime in codex
- ensure that TRACE logs are set for nodes running in multinodesuite when DebugCodexNodes is set
- add TRACE loglevel to testcli
