Setup and contributing instructions

CONTRIBUTINGTESTS.MD

@@ -0,0 +1,24 @@
+# Distributed System Tests for Nim-Codex
+
+## Contributing tests
+Do you want to write some tests for Codex using this distributed test setup? Great! Here's what you do.
+
+1. Create a branch. Name it something descriptive, but start it with `tests/` please. [Example: `tests/data-redundancy`.]
+1. Checkout your branch, and decide if your tests will be 'short' tests (minutes to hours), or 'long' tests (hours to days), or both! Create a folder for your tests in the matching folders (`/Tests`, `/LongTests`) and don't worry: You can always move your tests later if you like. [Example, short: `/Tests/DataRedundancy/`, long: `/LongTests/DataRedundancy/`]
+1. Create one or more code files in your new folder, and write some tests! Here are some tips to help you get started. You can always take a look at the example tests found in [`/Tests/BasicTests/ExampleTests.cs`](/Tests/BasicTests/ExampleTests.cs)
+   1. Set up a standard NUnit test fixture.
+   1. Inherrit from `DistTest` or `AutoBootstrapDistTest`.
+   1. When using `DistTest`:
+      1. You must start your own Codex bootstrap node. You can use `SetupCodexBootstrapNode(...)` for this.
+      1. When you start other Codex nodes with `SetupCodexNodes(...)` you can pass the bootstrap node by adding the `.WithBootstrapNode(...)` option.
+   1. When using `AutoBootstrapDistTest`:
+      1. The test-infra creates the bootstrap node for you, and automatically passes it to each Codex node you create in your tests. Handy for keeping your tests clean and to-the-point.
+      1. When using the auto-bootstrap, you have no control over the bootstrap node from your tests. You can't (for example) shut it down during the course of the test. If you need this level of control for your scenario, use the `DistTest` instead.
+   1. You can generate files of random test data by calling `GenerateTestFile(...)`.
+   1. If your test needs a long time to run, add the `[UseLongTimeouts]` function attribute. This will greatly increase maximum time-out values for operations like for example uploading and downloading files.
+   1. You can enable access to the Codex node metrics by adding the option `.EnableMetrics()`. Enabling metrics will make the test-infra download and save all Codex metrics in case of a test failure. (The metrics are stored as CSV, in the same location as the test log file.)
+   1. You can enable access to the blockchain marketplace by adding the option `.EnableMarketplace(...)`.
+   1. Enabling metrics and/or enabling the marketplace takes extra resources from the test-infra and increases the time needed during Codex node setup. Please don't enable these features unless your tests need them.
+   1. Tip: Codex nodes can be named. Use the option `WithName(...)` and make reading your test logs a little nicer!
+   1. Tip: Commit often.
+1. Once you're happy with your tests, please create a pull-request and ask (another) Codex core contributor to review your changes.

LOCALSETUP.MD

@@ -0,0 +1,18 @@
+# Distributed System Tests for Nim-Codex
+
+## Local setup
+These steps will help you set up everything you need to run and debug the tests on your local system.
+
+### Installing the requirements.
+1. Install dotnet v6.0 or newer. (If you install a newer version, consider updating the .csproj files by replacing all mention of `net6.0` with your version.)
+1. Set up a nice C# IDE or plugin for your current IDE.
+1. Install docker desktop.
+1. In the docker-desktop settings, enable kubernetes. (This might take a few minutes.)
+
+### Configure to taste.
+The tests should run as-is. You can change the configuration. The items below explain the what and how.
+1. Open `DistTestCore/Configuration.cs`.
+1. `k8sNamespace` defines the Kubernetes namespace the tests will use. All Kubernetes resources used during the test will be created in it. At the beginning of a test run and at the end of each test, the namespace and all resources in it will be deletes.
+1. `kubeConfigFile`. If you are using the Kubernetes cluster created in docker desktop, this field should be set to null. If you wish to use a different cluster, set this field to the path (absolute or relative) of your KubeConfig file.
+1. `LogConfig(path, debugEnabled)`. Path defines the path (absolute or relative) where the tests logs will be saved. The debug flag allows you to enable additional logging. This is mostly useful when something's wrong with the test infra.
+1. `FileManagerFolder` defines the path (absolute or relative) where the test infra will generate and store test data files. This folder will be deleted at the end of every test run.

README.md

@@ -1,18 +1,28 @@
 # Distributed System Tests for Nim-Codex
 
+Using a common dotnet unit-test framework and a few other libraries, this project allows you to write tests that use multiple Codex node instances in various configurations to test the distributed system in a controlled, reproducable environment.
+
 Nim-Codex: https://github.com/status-im/nim-codex
+Dotnet: v6.0
+Kubernetes: v1.25.4
+Dotnet-kubernetes SDK: v10.1.4 https://github.com/kubernetes-client/csharp
+Nethereum: v4.14.0
 
-Tests are built on dotnet v6.0 and Kubernetes v1.25.4, using dotnet-kubernetes SDK: https://github.com/kubernetes-client/csharp
+## Tests
+Tests are devided into two assemblies: `/Tests` and `/LongTests`.
+`/Tests` is to be used for tests that take several minutes to hours to execute.
+`/LongTests` is to be used for tests that take hours to days to execute.
 
-## Requirement
+TODO: All tests will eventually be running as part of a dedicated CI pipeline and kubernetes cluster. Currently, we're developing these tests and the infra-code to support it by running the whole thing locally.
 
-At this moment, the tests require a local kubernetes cluster to be installed.
+## Test logs
+Because tests potentially take a long time to run, logging is in place to help you investigate failures afterwards. Should a test fail, all Codex terminal output (as well as metrics if they have been enabled) will be downloaded and stored along with a details, step-by-step log of the test. If something's gone wrong and you're here to discover the details, head for the logs.
 
-## Run
+## How to contribute tests
+An important goal of the test infra is to provide an simple, accessible way for developers to write their tests. If you want to contribute tests for Codex, please follow the steps [HERE](/CONTRIBUTINGTESTS.MD).
 
-Short tests: These tests may take minutes to an hour.
-`dotnet test Tests`
-
-Long tests: These may takes hours to days.
-`dotnet test LongTests`
+## Run the tests on your machine
+Creating tests is much easier when you can debug them on your local system. This is possible, but requires some set-up. If you want to be able to run the tests on your local system, follow the steps [HERE](/LOCALSETUP.MD). Please note that tests which require explicit node locations cannot be executed locally. (Well, you could comment out the location statements and then it would probably work. But that might impact the validity/usefulness of the test.)
 
+## Missing functionality
+Surely the test-infra doesn't do everything we'll need it to do. If you're running into a limitation and would like to request a new feature for the test-infra, please create an issue.

Tests/DurabilityTests/DurabilityTests.cs

@@ -11,12 +11,12 @@ namespace Tests.DurabilityTests
         [Test]
         public void BootstrapNodeDisappearsTest()
         {
-            var bootstrapNode = SetupCodexNode();
+            var bootstrapNode = SetupCodexBootstrapNode();
             var group = SetupCodexNodes(2, s => s.WithBootstrapNode(bootstrapNode));
             var primary = group[0];
             var secondary = group[1];
 
-            // There is 1 minute of time for the nodes to connect to each other.
+            // There is 1 minute of time f or the nodes to connect to each other.
             // (Should be easy, they're in the same pod.)
             Time.Sleep(TimeSpan.FromMinutes(6));
             bootstrapNode.BringOffline();
@@ -31,7 +31,7 @@ namespace Tests.DurabilityTests
         [Test]
         public void DataRetentionTest()
         {
-            var bootstrapNode = SetupCodexNode(s => s.WithLogLevel(CodexLogLevel.Trace));
+            var bootstrapNode = SetupCodexBootstrapNode(s => s.WithLogLevel(CodexLogLevel.Trace));
 
             var startGroup = SetupCodexNodes(2, s => s.WithLogLevel(CodexLogLevel.Trace).WithBootstrapNode(bootstrapNode));
             var finishGroup = SetupCodexNodes(10, s => s.WithLogLevel(CodexLogLevel.Trace).WithBootstrapNode(bootstrapNode));