nim-unittest2/nim.cfg

8 lines
239 B
INI
Raw Permalink Normal View History

Revamped output handling, cleanup and removal of thread mode (#31) This PR introduces a facelift for `unittest2` bringing its UX into the 21st century! By default, a compact dot-based format is used to avoid spamming the console on success - successes are normal and only minimal information is needed in the vast majority of runs: ``` [ 20/25] HTTP client testing suite ................s..... (14.2s) [ 21/25] Asynchronous process management test suite ....................F. (14.5s) =========================== /home/arnetheduck/status/nimbus-eth2/vendor/nim-chronos/build/testall 'Asynchronous process management test suite::File descriptors leaks test' --------------------------- /home/arnetheduck/status/nimbus-eth2/vendor/nim-chronos/tests/testproc.nim(465, 27): Check failed: getCurrentFD() == markFD getCurrentFD() was 3 markFD was 5 [FAILED ] ( 0.00s) File descriptors leaks test ``` For each line, we see a suite followed by single-character status markers - mostly successes but also a skipped and a failed test. Failures are printed verbosely after the suite is done so they don't get lost in the success spam along with a simple copy-pasteable line showing how to reproduce the failure. The verbose mode has also received an upgrade: ``` [ 20/22] terminateAndWaitForExit() timeout test [OK ] ( 2.00s) terminateAndWaitForExit() timeout test [ 21/22] File descriptors leaks test =========================== /home/arnetheduck/status/nimbus-eth2/vendor/nim-chronos/tests/testall 'Asynchronous process management test suite::File descriptors leaks test' --------------------------- /home/arnetheduck/status/nimbus-eth2/vendor/nim-chronos/tests/testproc.nim(465, 27): Check failed: getCurrentFD() == markFD getCurrentFD() was 3 markFD was 5 [FAILED ] ( 0.00s) File descriptors leaks test [ 22/22] Leaks test ``` Here, we can see a "test started" marker so that it becomes clear which test is currently running and so that any output the test itself prints has the proper name attached to it. At the end, there's a summary as well that reiterates the tests that failed so they can be found in the success spam that the verbose mode generates: ``` [Summary ] 22 tests run: 21 OK, 1 FAILED, 0 SKIPPED [FAILED ] ( 0.00s) File descriptors leaks test ``` As seen above, the new mode knows how many tests will run: this is thanks to a new two-phase mode of test running: "collect" and "run"! The "collection" phase is responsible for collecting test metadata which is later used to run tests in smarter ways, for example in isolated processes. Unfortunately, it is not fully backwards-compatible because the global setup might expose order-dependency problems in test suites that previously would not be visible - for this, we have acompile-time setting for a compatibilty mode that runs things roughly in the same order as before but disables some of the fancy functionality. The changes also come with a bag of mixed stuff: * the parallel mode is removed entirely - it was broken beyond fixing and needs to be rewritten from zero - deadlocks, GC violations and everything in between rendered it practically unusable and a source of CI failures above all * an experimental process isolation mode where tests are run in a separate process - this is allows many features such as timeouts, output verification etc but it also breaks for similar reasons as above: "global" code gets executed out of order and repeatedly. * random UX fixes and cleanups of things like env var reading and command line options
2023-09-01 10:23:01 +00:00
nimcache = "build/nimcache/$projectName"
# Avoid some rare stack corruption while using exceptions with a SEH-enabled
# toolchain: https://github.com/status-im/nimbus-eth2/issues/3121
@if windows and not vcc:
--define:nimRawSetjmp
@end