Updates READMEs
This commit is contained in:
benbierens
parent
8d2ce8ae7a
commit
dcdcd110a1
|
|
@ -10,15 +10,19 @@ Nethereum: v4.14.0
|
|||
|
||||
## Tests/CodexTests and Tests/CodexLongTests
|
||||
These are test assemblies that use NUnit3 to perform tests against transient Codex nodes.
|
||||
Read more [HERE](/Tests/CodexTests/README.md)
|
||||
|
||||
## Tests/ContinousTests
|
||||
A console application that runs tests in an endless loop against a persistent deployment of Codex nodes.
|
||||
Read more [HERE](/Tests/CodexContinuousTests/README.md)
|
||||
|
||||
## Tools/CodexNetDeployer
|
||||
A console application that can deploy Codex nodes.
|
||||
Read more [HERE](/Tools/CodexNetDeployer/README.MD)
|
||||
|
||||
## Test logs
|
||||
Because tests potentially take a long time to run, logging is in place to help you investigate failures afterwards. Should a test fail, all Codex terminal output (as well as metrics if they have been enabled) will be downloaded and stored along with a detailed, step-by-step log of the test. If something's gone wrong and you're here to discover the details, head for the logs.
|
||||
## Framework architecture
|
||||
The framework is designed to be extended by project-specific plugins. These plugins contribute functionality and abstractions to the framework. Users of the framework use these to perform tasks such as testing and deploying.
|
||||
|
||||
|
||||
## How to contribute a plugin
|
||||
If you want to add support for your project to the testing framework, follow the steps [HERE](/CONTRIBUTINGPLUGINS.MD)
|
||||
|
|
|
|||
|
|
@ -27,9 +27,6 @@ namespace ContinuousTests
|
|||
[Uniform("target-duration", "td", "TARGETDURATION", false, "If greater than zero, runner will run for this many seconds before stopping.")]
|
||||
public int TargetDurationSeconds { get; set; } = 0;
|
||||
|
||||
[Uniform("dl-logs", "dl", "DLLOGS", false, "If true, runner will periodically download and save/append container logs to the log path.")]
|
||||
public bool DownloadContainerLogs { get; set; } = false;
|
||||
|
||||
public CodexDeployment CodexDeployment { get; set; } = null!;
|
||||
}
|
||||
|
||||
|
|
@ -55,10 +52,7 @@ namespace ContinuousTests
|
|||
{
|
||||
var nl = Environment.NewLine;
|
||||
Console.WriteLine("ContinuousTests will run a set of tests against a codex deployment given a codex-deployment.json file." + nl +
|
||||
"The tests will run in an endless loop unless otherwise specified, using the test-specific timing values." + nl);
|
||||
|
||||
Console.WriteLine("ContinuousTests assumes you are running this tool from *inside* the Kubernetes cluster. " +
|
||||
"If you are not running this from a container inside the cluster, add the argument '--external'." + nl);
|
||||
"The tests will run in an endless loop unless otherwise specified." + nl);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -37,6 +37,9 @@ namespace ContinuousTests
|
|||
public void RunNode(ICodexNode bootstrapNode, Action<ICodexSetup> setup, Action<ICodexNode> operation)
|
||||
{
|
||||
var entryPoint = CreateEntryPoint();
|
||||
// We have to be sure that the transient node we start is using the same image as whatever's already in the deployed network.
|
||||
// Therefore, we use the image of the bootstrap node.
|
||||
CodexContainerRecipe.DockerImageOverride = bootstrapNode.Container.Recipe.Image;
|
||||
|
||||
try
|
||||
{
|
||||
|
|
|
|||
|
|
@ -0,0 +1,20 @@
|
|||
# Codex Continuous Tests
|
||||
This CLI tool runs tests in an endless loop, using a network of Codex nodes in a kubernetes cluster. Run `dotnet run -- --help` to view all CLI options.
|
||||
|
||||
## Choosing tests
|
||||
By default, all tests in the `CodexContinuousTests/Tests` folder will be used. If you want to limit your test run to a subset of tests, please delete or disable the other test code files. TODO: We'd like a CLI option for selecting tests. Similar to `dotnet test --filter`, maybe?
|
||||
|
||||
## Where do I get a `codex-deployment.json`
|
||||
See [THIS](../../Tools/CodexNetDeployer/README.MD)
|
||||
|
||||
## Output
|
||||
The test runner will produce a folder with all the test logs. They are sorted by timestamp and reflect the names of the tests. When a test fails, the log file for that specific test will be postfixed with `_FAILED`.
|
||||
|
||||
### Pass and fail conditions
|
||||
While individual tests can pass or fail for a number of times and/or over a length of time as configurable with the CLI argument, the test run entirely is not considered passed or failed until either of the following conditions are met:
|
||||
1. Failed: The number of test failures has reached the specifid number, or the test runner was manually cancelled.
|
||||
1. Passed: The failed condition was not reached within the time specified by the target-duration option.
|
||||
|
||||
## Transient nodes
|
||||
The continuous tests runner is designed to use a network of Codex nodes deployed in a kubernetes cluster. The runner will not influence or manage the lifecycle of the nodes in this deployment. However, some test cases require transient nodes.
|
||||
A transient node is a node started and managed by the test runner on behalf of a specific test. The runner facilitates the tests to start and stop transient nodes, as well as bootstrap those nodes against (permanent) nodes from the deployed network. The test runner makes sure that the transient nodes use the same docker image as the permanent nodes, to avoid version conflicts. However, the image used for transient nodes can be overwritten by setting the `CODEXDOCKERIMAGE` environment variable. The use of a local Codex repository for building override images is not supported for transient nodes.
|
||||
|
|
@ -3,5 +3,4 @@ dotnet run \
|
|||
--codex-deployment=codex-deployment.json \
|
||||
--keep=1 \
|
||||
--stop=10 \
|
||||
--dl-logs=1 \
|
||||
--target-duration=172800 # 48 hours
|
||||
|
|
|
|||
|
|
@ -0,0 +1,26 @@
|
|||
# Codex Tests
|
||||
This is an NUnit test assembly that can be used with the standard dotnet test runner. For all its CLI options, run `dotnet test --help`.
|
||||
|
||||
## Example tests
|
||||
Running all the tests in the assembly can take a while. In order to check basic viability of your setup as well as the Codex image you're using, consider running only the example tests using the filter option: `dotnet test --filter=Example`.
|
||||
|
||||
## Output
|
||||
The test runner will produce a folder named `CodexTestLogs` with all the test logs. They are sorted by timestamp and reflect the names of the test fixtures and individual tests. When a test fails, the log file for that specific test will be postfixed with `_FAILED`. The same applies to the fixture log file. The `STATUS` files contain the test results in JSON, for easy machine reading.
|
||||
|
||||
## Overrides
|
||||
The following environment variables allow you to override specific aspects of the behaviour of the tests.
|
||||
|
||||
| Variable | Description |
|
||||
|------------------|-------------------------------------------------------------------------------------------------------------|
|
||||
| RUNID | A pod-label 'runid' is added to each pod created during the tests. Use this to set the value of that label. |
|
||||
| TESTID | Similar to RUNID, except the label is 'testid'. |
|
||||
| CODEXDOCKERIMAGE | If set, this will be used instead of the default Codex docker image. |
|
||||
|
||||
## Using a local Codex repository
|
||||
If you have a clone of the Codex git repository, and you want to run the tests using your local modifications, the following environment variable options are for you. Please note that any changes made in Codex's 'vendor' directory will be discarded during the build process.
|
||||
|
||||
| Variable | Description |
|
||||
|----------------|---------------------------------------------------------------------------------------------------------------------|
|
||||
| CODEXREPOPATH | Path to the Codex repository. |
|
||||
| DOCKERUSERNAME | Username of your Dockerhub account. |
|
||||
| DOCKERPASSWORD | Password OR access-token for your Dockerhub account. You can omit this variable to use your system-default account. |
|
||||
|
|
@ -45,7 +45,7 @@ public class Program
|
|||
private static void PrintHelp()
|
||||
{
|
||||
var nl = Environment.NewLine;
|
||||
Console.WriteLine("CodexNetDeployer allows you to easily deploy multiple Codex nodes in a Kubernetes cluster. " +
|
||||
Console.WriteLine("CodexNetDeployer allows you to deploy multiple Codex nodes in a Kubernetes cluster. " +
|
||||
"The deployer will set up the required supporting services, deploy the Codex on-chain contracts, start and bootstrap the Codex instances. " +
|
||||
"All Kubernetes objects will be created in the namespace provided, allowing you to easily find, modify, and delete them afterwards." + nl);
|
||||
}
|
||||
|
|
|
|||
Loading…
Reference in New Issue