status-im
/
cs-codex-dist-tests
mirror of
2
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
cs-codex-dist-tests/Tests/CodexContinuousTests
History
benbierens 3a61fc89c6
Adds WaitForCleanup test attribute to allow tests to wait for resources to be cleaned up 		2024-06-06 15:09:52 +02:00
..
Tests Restores continuous tests 2024-04-01 08:29:55 +02:00
reports Merge pull request #72 from GoodDaisy/master 2024-03-20 14:45:09 +01:00
CodexContinuousTests.csproj Restores continuous test runner 2023-09-20 13:33:58 +02:00
Configuration.cs Target duration now supports timespan string format. 2023-11-17 14:51:32 +01:00
ContinuousTest.cs Cleanup 2023-10-04 09:36:59 +02:00
ContinuousTestRunner.cs Adds WaitForCleanup test attribute to allow tests to wait for resources to be cleaned up 2024-06-06 15:09:52 +02:00
ElasticSearchLogDownloader.cs add ability to stop single containers 2024-04-13 17:12:14 +03:00
EntryPointFactory.cs Bumps k8s operation timeout for continuous test runner. 2023-10-04 09:26:11 +02:00
NodeRunner.cs Adds WaitForCleanup test attribute to allow tests to wait for resources to be cleaned up 2024-06-06 15:09:52 +02:00
Program.cs Merge pull request #72 from GoodDaisy/master 2024-03-20 14:45:09 +01:00
README.md Updates READMEs 2023-09-29 10:19:59 +02:00
SingleTestRun.cs Adds WaitForCleanup test attribute to allow tests to wait for resources to be cleaned up 2024-06-06 15:09:52 +02:00
StartupChecker.cs add ability to stop single containers 2024-04-13 17:12:14 +03:00
TestFactory.cs Moves projects into folders 2023-09-20 10:51:47 +02:00
TestHandle.cs Moves projects into folders 2023-09-20 10:51:47 +02:00
TestLoop.cs Remove run ID and introduce deploy ID (#93) 2024-02-22 10:41:07 -03:00
TestMomentAttribute.cs Moves projects into folders 2023-09-20 10:51:47 +02:00
deploy-and-run.sh Locks up deploy-and-run script after tests are started. 2023-11-01 10:27:31 +01:00
run.sh Automate Continuous Tests using GitHub Actions (#69) 2023-11-06 15:09:51 +02:00

README.md

Codex Continuous Tests

This CLI tool runs tests in an endless loop, using a network of Codex nodes in a kubernetes cluster. Run dotnet run -- --help to view all CLI options.

Choosing tests

By default, all tests in the CodexContinuousTests/Tests folder will be used. If you want to limit your test run to a subset of tests, please delete or disable the other test code files. TODO: We'd like a CLI option for selecting tests. Similar to dotnet test --filter, maybe?

Where do I get a codex-deployment.json

See THIS

Output

The test runner will produce a folder with all the test logs. They are sorted by timestamp and reflect the names of the tests. When a test fails, the log file for that specific test will be postfixed with _FAILED.

Pass and fail conditions

While individual tests can pass or fail for a number of times and/or over a length of time as configurable with the CLI argument, the test run entirely is not considered passed or failed until either of the following conditions are met:

  1. Failed: The number of test failures has reached the specifid number, or the test runner was manually cancelled.
  2. Passed: The failed condition was not reached within the time specified by the target-duration option.

Transient nodes

The continuous tests runner is designed to use a network of Codex nodes deployed in a kubernetes cluster. The runner will not influence or manage the lifecycle of the nodes in this deployment. However, some test cases require transient nodes. A transient node is a node started and managed by the test runner on behalf of a specific test. The runner facilitates the tests to start and stop transient nodes, as well as bootstrap those nodes against (permanent) nodes from the deployed network. The test runner makes sure that the transient nodes use the same docker image as the permanent nodes, to avoid version conflicts. However, the image used for transient nodes can be overwritten by setting the CODEXDOCKERIMAGE environment variable. The use of a local Codex repository for building override images is not supported for transient nodes.