Commit Graph

387 Commits

Author SHA1 Message Date
benbierens b78f527c39
Adds test-type to STATUS log jsons. 2023-11-10 15:28:53 +01:00
benbierens 0be6df46bc
Merge branch 'deployment-json-rework' into feature/continuous-test-status-logs 2023-11-10 09:49:35 +01:00
benbierens c5cc15daa0
Updates endpoint to match latest Codex 2023-11-10 08:20:08 +01:00
benbierens 0301c3b076
Adds status log entry for each test run in continuous tests. 2023-11-09 11:35:45 +01:00
benbierens b8ce4c49d6
Merge branch 'master' into deployment-json-rework
# Conflicts:
#	Tests/CodexContinuousTests/run.sh
#	Tools/CodexNetDeployer/deploy-continuous-testnet.sh
2023-11-08 09:29:22 +01:00
benbierens 90070de028
Enable debug logging from single environment variable or static field. 2023-11-08 09:24:39 +01:00
benbierens 3f8699a59b
Removes dependency on pod information for elastic search download 2023-11-07 14:33:45 +01:00
benbierens 2b463d060d
Restores debug setting 2023-11-07 14:00:08 +01:00
benbierens ea66b5b408
debugging address selection in cluster 2023-11-07 12:02:17 +01:00
benbierens 655f4895d0
Fixes serialization issue for running containers for discord bot and metrics container 2023-11-07 11:17:20 +01:00
benbierens dc9f3ab090
removes dependency on static pod name and address info 2023-11-06 14:33:47 +01:00
Veaceslav Doina ead1ff7ec6
Automate Continuous Tests using GitHub Actions (#69) 2023-11-06 15:09:51 +02:00
Veaceslav Doina 807e8b7b5f
Fix TwoClientTest metric name
https://github.com/codex-storage/nim-codex/pull/603
2023-11-06 15:06:47 +02:00
benbierens 5241144e99
Locks up deploy-and-run script after tests are started. 2023-11-01 10:27:31 +01:00
benbierens 49300273e0
Cleanup continuous tests nicely even if no tests were selected for running. 2023-10-31 15:33:00 +01:00
benbierens e87f255f48
Creates dockerimage that will deploy-and-run from environment args 2023-10-31 11:01:10 +01:00
benbierens 85ad0b414f
Cleans up some gethnode arguments. 2023-10-30 13:30:14 +01:00
GoodDaisy 65473d8c15 fix Continous typo 2023-10-26 20:27:44 +08:00
benbierens 14e8222dfe
Adds codex debug-info to deployment json. Needed for bootstrapping later on. 2023-10-25 09:14:35 +02:00
benbierens dc0edce251
Moves continuous-tests default log path to persistent volume 2023-10-25 08:56:48 +02:00
benbierens 0e087c6fee
Logs continuous test config at startup. 2023-10-23 14:17:52 +02:00
benbierens 7a6d7d787b
Splits up metrics-scraper from metrics-endpoints configuration in deployer. 2023-10-23 10:57:59 +02:00
benbierens 4adce837ec
Logs total run duration in overview log. 2023-10-23 10:32:11 +02:00
benbierens e11a7d1600
Gives deployments a name. 2023-10-23 10:19:52 +02:00
benbierens 45fbd699a9
Disables calls to custom API endpoints. 2023-10-23 09:36:31 +02:00
benbierens bf18fa03a2
adds sleep to the starting of the test screens 2023-10-22 11:29:16 +02:00
benbierens 45050c34e4
Implements GetAddress method for runningContainers. 2023-10-19 11:18:59 +02:00
benbierens 6b1102efa7
Adds name argument to deploy-and-run script 2023-10-17 14:32:05 +02:00
benbierens d0cafb83a1
Fixes compile error in single test runner 2023-10-16 13:10:45 +02:00
benbierens 8f37b4cf38
Sets long timeouts for debug/repostore call. 2023-10-10 18:08:21 +02:00
benbierens 1a277ef1b5
Logging repostore content on twoclient test failure 2023-10-10 17:54:19 +02:00
benbierens b81d574a4b
adds block exchange tests. Updates namespaces 2023-10-09 16:59:52 +02:00
benbierens b0610393af
rigged for twoclient test with full container log download 2023-10-08 19:11:31 +02:00
benbierens e4b7e461e9
Adds pod name to downloaded logs 2023-10-08 18:58:15 +02:00
benbierens 0dcf3b0b16
Adds pod and deployment name 2023-10-08 18:56:00 +02:00
benbierens f3555f2703
Adds node id to downloaded container logs 2023-10-08 08:31:48 +02:00
benbierens ab9257ce56
moving the deployment json 2023-10-07 10:29:09 +02:00
benbierens 1a48c40ad7
Example of setting codex log topics for libp2p and discv5 2023-10-07 09:48:12 +02:00
benbierens 092128d77b
with working loops this time 2023-10-07 08:04:11 +02:00
benbierens c653afa554
trying with script 2023-10-07 07:53:10 +02:00
benbierens fab855c4bb
adds log and data paths 2023-10-07 07:49:55 +02:00
benbierens 75e263be31
Try with script 2023-10-07 07:47:54 +02:00
benbierens 9cb8c57615
Adds test report for September 2023-10-05 08:31:44 +02:00
benbierens 5db8be3252
Merge branch 'feature/log-mon'
# Conflicts:
#	Tests/CodexContinuousTests/ContinuousTestRunner.cs
2023-10-04 09:48:21 +02:00
benbierens ccc4732752
Cleanup 2023-10-04 09:36:59 +02:00
benbierens 562f886e30
Bumps k8s operation timeout for continuous test runner. 2023-10-04 09:26:11 +02:00
benbierens 3f2c789dd5
Fixes issue where boolean arguments can't be set to false. 2023-10-04 09:19:45 +02:00
benbierens 917d715497
Fixes startup of log reconstruction 2023-10-03 16:11:14 +02:00
benbierens 696fb10386
debugging the downloader in the cluster 2023-10-03 15:36:44 +02:00
benbierens a69bea47af
Debugging downloader in cluster 2023-10-03 15:18:58 +02:00
benbierens adbd481c7c
Sets elasticsearch downloader for in-cluster running. 2023-10-03 14:32:31 +02:00
benbierens 1bca2bb928
Updates parser for new counter format. 2023-10-03 13:34:39 +02:00
benbierens 6fe9d38eb3
Adds filter option for selecting tests. Adds cleanup option for deleting deployment namespace. 2023-10-03 09:53:02 +02:00
benbierens 6a96bd7639
Implements elastic search log downloader. Requires log-counter codex update! 2023-10-02 14:42:36 +02:00
benbierens c3ec64f25e
very very WIP for how to download complete log from elastic search 2023-10-02 12:01:49 +02:00
benbierens 3dcbb78204
Prevents multiple continuous tests from running interleaved. 2023-10-02 11:18:27 +02:00
benbierens 73c49b42c6
Fixes incorrect log message when test run gets cancelled. 2023-10-02 09:24:01 +02:00
benbierens da855b8d0d
Proper log monitoring shutdown 2023-10-01 10:52:05 +02:00
benbierens b23c66c86e
Working timerange limited log download 2023-10-01 09:57:32 +02:00
benbierens ac7fa4905d
Adds dockertag description 2023-09-29 10:40:48 +02:00
benbierens dcdcd110a1
Updates READMEs 2023-09-29 10:19:59 +02:00
benbierens 3e1b361b7b
Fixes cleanup of online codexnodes in Codex test fixture 2023-09-28 15:04:21 +02:00
benbierens 993f2ff813
Fixes issue where autobootstrap node doesn't clear old bootstrap node handles. 2023-09-28 14:23:48 +02:00
benbierens 260294fb35
Fixes marketplace interference check tests. 2023-09-28 14:00:13 +02:00
benbierens b0f72cf453
Automated local docker build is working. 2023-09-28 12:22:35 +02:00
benbierens 453a1a35a2
Adds status log to continuous tests 2023-09-28 09:39:15 +02:00
benbierens dabf836838
Makes default run id static. 2023-09-28 08:53:01 +02:00
benbierens 8caa7ab4fa
Adds option to limit continuous test run to set duration. 2023-09-27 11:33:54 +02:00
benbierens 6a2bd11dd5
Adds codex environment variables to startup log of continuous tests. 2023-09-27 09:09:42 +02:00
benbierens d830ccb48e
Sets timestamp as default runID when environment variable is not set. 2023-09-27 08:49:36 +02:00
benbierens 42b0ab67a0
Logging bytes per millisecond for twoclient upload and download 2023-09-26 15:17:35 +02:00
benbierens 0a2bca9f52
Restores pod labels in deployer. Adds metrics assert for twoclient continuous test. 2023-09-26 14:32:28 +02:00
benbierens 10697f1047
Updates container location support 2023-09-25 08:47:19 +02:00
benbierens d2854bb905
Cleanup 2023-09-22 07:43:46 +02:00
benbierens 418daf1e3f
Updates the docs 2023-09-21 14:39:41 +02:00
benbierens 01b2ff2181
Merge branch 'master' into plugin-refactor
# Conflicts:
#	DistTestCore/CodexStarter.cs
#	DistTestCore/GethStarter.cs
#	ProjectPlugins/CodexPlugin/CodexContainerRecipe.cs
#	ProjectPlugins/CodexPlugin/CodexSetup.cs
#	ProjectPlugins/CodexPlugin/CodexStartupConfig.cs
2023-09-21 11:07:27 +02:00
benbierens dbf0ed714c
Restores continuous test runner. 2023-09-21 10:33:09 +02:00
benbierens 5b2557b3f4
Set up loading of plugins. 2023-09-20 13:56:01 +02:00
benbierens 8cde69a483
Restores continuous test runner 2023-09-20 13:33:58 +02:00
benbierens 75369d68f7
Restores CodexNetDeployer 2023-09-20 12:02:32 +02:00
benbierens bf4c8d8026
Restores project references. 2023-09-20 10:59:52 +02:00
benbierens 09670e00e9
Moves projects into folders 2023-09-20 10:51:47 +02:00
benbierens 5fa4e0ff9f
Sets serialization gate between each deploy and wrap to ensure application lifecycle invariance. 2023-09-20 09:16:57 +02:00
benbierens cedaf84740
Wires up marketplace access 2023-09-20 08:45:55 +02:00
ThatBen 6cbf363bb1 restores balance asserts. 2023-09-19 16:22:07 +02:00
benbierens 6cf86af3b5
Successful testtoken mint and balance 2023-09-19 13:39:24 +02:00
benbierens 58b1c1e03c
Can send eth 2023-09-19 11:51:59 +02:00
benbierens a20fc6864b
working out marketplace details 2023-09-19 10:28:02 +02:00
ThatBen 3d4370b154 deploys codex contract 2023-09-15 16:27:08 +02:00
ThatBen 4cc93eba73 Can start Geth 2023-09-15 15:52:02 +02:00
ThatBen 7ef3f615e1 Moves log downloading to core. 2023-09-14 15:26:46 +02:00
benbierens ec5aebb47b
Restores full-connectivity tests. 2023-09-13 14:24:43 +02:00
benbierens 84dd514517
Restores connectivity test helpers 2023-09-13 12:24:46 +02:00
benbierens d1895bab02
Implements bringOffline for codex nodes 2023-09-13 12:09:44 +02:00
benbierens 53bb9968ff
Metrics test passed 2023-09-13 11:59:21 +02:00
benbierens 1ca3ddc67e
Implements metrics plugin 2023-09-13 11:25:08 +02:00
benbierens 7e7414d491
Removes core-interface associate methods. 2023-09-13 08:55:04 +02:00
benbierens dd6b99c670
Brings back all the tests. 2023-09-12 15:43:30 +02:00
ThatBen 48dda1735c Good progress 2023-09-11 16:57:57 +02:00
benbierens 83d184177a
Moving all codex details to CodexPlugin 2023-09-11 11:59:33 +02:00
benbierens 6915e90861
Extracts file manager utils to separate assembly 2023-09-11 10:43:27 +02:00
benbierens d074f64c42
a 2023-09-08 14:03:47 +02:00
benbierens 9d2dbbbcab
Fixes missing usings 2023-09-08 10:21:40 +02:00
benbierens df6da29a69
Moves ByteSize to Utils assembly. 2023-09-08 09:39:56 +02:00
benbierens 23f5318dfc
More uploads 2023-09-01 14:40:48 +02:00
benbierens 3df4a97bba
debugging local 2023-09-01 10:39:41 +02:00
benbierens 8f61c46ff0
Adds simple test to ensure routing table is correctly filled. 2023-08-29 16:08:40 +02:00
benbierens 249b13d503
Adds tests to ensure marketplace and metrics don't interfere with peer discovery 2023-08-28 11:16:58 +02:00
benbierens b276592daf
Fixes download stream error in peerDownloadHelper 2023-08-24 15:56:56 +02:00
benbierens 7da90fc53b
cleanup 2023-08-24 15:45:21 +02:00
benbierens 7eda26d177
Cleanup of full-connectivity helper. 2023-08-24 10:59:11 +02:00
benbierens 47aedc2ac8
cleanup 2023-08-22 15:51:39 +02:00
benbierens 7e5121a642
Fixes test for cloud environment 2023-08-22 09:06:44 +02:00
benbierens ce3850b0b0
I accidentally filled up my harddrive with random numbers last night. 2023-08-21 07:58:58 +02:00
benbierens 1682dfc08f
Adds holdmybeer test to continuous test runner 2023-08-18 10:31:20 +02:00
benbierens c8cb04d859
Allows downloading only log-tails. 2023-08-16 16:13:29 +02:00
benbierens 30364abf21
Correct checking of upload and download log statements 2023-08-16 11:39:24 +02:00
benbierens 6486dba289
Asserting correct filesize and block numbers in the codex logs 2023-08-16 10:52:44 +02:00
benbierens 57c46004b1
cleanup 2023-08-15 15:00:47 +02:00
benbierens 3abb1770c2
Fixes crash log downloads 2023-08-15 12:55:30 +02:00
benbierens 08ee09af1e
wires crashWatcher into CodexStarter 2023-08-15 11:01:18 +02:00
benbierens 72fa368357
working example of crash log recovery 2023-08-15 10:03:01 +02:00
benbierens 86bae93e98
Wires up block-maintenance number of blocks param 2023-08-14 16:37:31 +02:00
benbierens c0b7d3a747
Hooks up block maintenance interval config param. 2023-08-14 15:51:03 +02:00
benbierens 6e5e30afd4
rigged up a pod crash detector maybe 2023-08-14 15:10:36 +02:00
benbierens 9f0600d6c1
Automatic quota line. Holding test. 2023-08-14 10:35:42 +02:00
benbierens 68d19353aa
moves to 80MB and long timeouts 2023-08-08 14:42:59 +02:00
benbierens eaf5db5e91
Run test against increasing file sizes 2023-08-08 10:45:16 +02:00
benbierens 598dc766fa
use extra log codex image 2023-08-08 09:44:38 +02:00
benbierens ab42fa8004
removes just-wait tests 2023-08-07 18:31:13 +02:00
benbierens 5ad2aa4c6a
Merge branch 'master' into spike/local-continuous-debug 2023-08-07 10:58:27 +02:00
benbierens 8a6c5e95ee
Makes sure codex nodes are always started in their own pods. 2023-08-07 10:44:48 +02:00
benbierens 3884a6a7f7
a 2023-08-02 15:11:27 +02:00
benbierens 8d229f1720
cleans up waiting for storage contract state helper methods. 2023-07-31 09:56:37 +02:00
benbierens 9a7946c863
Basic tests pass 2023-07-31 09:16:52 +02:00
benbierens ac33d3c7be
Moves some of the fully-connected download tests to the long-tests 2023-07-17 13:47:41 +02:00
benbierens 4815194db0
Removes variable image tests. 2023-07-17 08:56:18 +02:00
benbierens b295314bde
fixes blockTTL envvar 2023-07-14 10:45:26 +02:00
benbierens f23926636c
Setting up way to run different codex images 2023-07-14 10:18:37 +02:00
benbierens 1f8fc52cb1
Adds three-client test to check discovery relaying 2023-07-13 09:20:24 +02:00
benbierens 63068aae1d
Debugging marketplace test 2023-06-27 15:28:00 +02:00
benbierens 56dc1c632d
Variable size download tests 2023-06-06 16:10:30 +02:00
benbierens e33a6776e5
Cleanup 2023-06-06 14:37:02 +02:00
benbierens 10a0ac3311
Splits up download tests from connectivity tests and improves testfile logging 2023-06-04 08:59:51 +02:00
benbierens ad71cff465
Automatically map location enum to available k8s nodes. 2023-06-02 10:04:07 +02:00
benbierens 9747846787
Increase retry timeout 2023-06-01 12:27:54 +02:00
benbierens 61d2185bb6
Attempt internal connection using pod IP and exposed port. 2023-06-01 10:14:32 +02:00
benbierens b92c1b970d
Sets up support for internal and external container addresses 2023-06-01 09:35:18 +02:00
benbierens 3d1d82f582
pushes codex loglevel to configuration 2023-05-31 13:34:12 +02:00
benbierens f2145a747f
Clearly distinguishes between peer-connections that can't be made and problems with the node-to-test link. 2023-05-31 13:15:41 +02:00
benbierens 8c85cd22bb
Adds clear logging for peer helper invocations. 2023-05-29 09:25:03 +02:00
benbierens 1016f568b8
Cleanup of peer connection helpers. Adds peer download helpers. 2023-05-29 09:13:38 +02:00
benbierens ab7a334987
Adds assert that nodes have correct addresses in their table of peers. 2023-05-29 08:35:46 +02:00
benbierens 025b03e678
Retain for a while in staged variable nodes peer discovery tests 2023-05-18 10:42:04 +02:00
benbierens 830d74a5ae
Faster peer-pair checking 2023-05-12 10:48:12 +02:00
benbierens 90a5aafa1c
Better peer discovery result logging. Staged discovery test. 2023-05-12 09:11:05 +02:00
benbierens f7e7849460
Improves retrying in peer test helper 2023-05-11 13:59:53 +02:00
benbierens 38d5b172f4
successful peer discovery tests 2023-05-11 12:44:53 +02:00
benbierens e9679e18c0
logs all checks before failing 2023-05-10 10:47:10 +02:00
benbierens d58cb38c79
Cleanup duplicated retry logic. Adds layered and chain tests 2023-05-10 09:55:36 +02:00
benbierens 2a55328d11
Debugs AutoBootstrapDistTest 2023-05-10 09:09:31 +02:00
benbierens 1d224cf2d3
Sets up working peer-discovery tests 2023-05-10 08:53:57 +02:00
benbierens 9c6f00dbce
Disable net-isolation tests 2023-05-05 08:47:20 +02:00
benbierens 3c9cb9b7da
Turns off test cases for two-client tests 2023-05-05 08:35:13 +02:00
benbierens 3aba6d5082
Merge branch 'feature/parallel-tests' 2023-05-05 08:33:24 +02:00
benbierens e96c8a2a13
Enables checking for engine peers in peer tests 2023-05-05 07:58:48 +02:00
benbierens d0b4e31167
Turns off parallel tests for now. 2023-05-04 15:22:11 +02:00
benbierens 3db2217c32
oops forgot to update calls to log 2023-05-04 15:21:36 +02:00
benbierens eae138f7fd
Adds policy and test for network isolation. This will not work on docker-desktop clusters. 2023-05-04 14:55:39 +02:00
benbierens 533bf32577
Better logging in case codex node fails to respond to debug/info request. 2023-05-04 11:34:43 +02:00
benbierens 2ed6993b58
wip debugging anomalous test failures 2023-05-04 08:25:48 +02:00
benbierens 79a40904e4
setup 2023-05-03 14:18:37 +02:00
benbierens 01c8238311
Applies faster geth image 2023-05-03 10:21:15 +02:00
Eric Mastro c16055d7e4
bump to .net 7 2023-05-02 15:17:43 +10:00
benbierens 085c4431a2
Setup and contributing instructions 2023-05-01 16:26:26 +02:00
benbierens 5c00205f62
Cleanup and support for automatic bootstrap tests 2023-05-01 11:14:42 +02:00
benbierens 2d5554fadf
wip test, but passes. 2023-04-30 11:10:38 +02:00
benbierens db4364d336
Much nicer logging by means of container name-override and long-id log replacements 2023-04-30 10:56:19 +02:00
benbierens 4dd02a96e9
Isolating issue with peer tests 2023-04-30 10:08:32 +02:00
benbierens 1da481daa1
Restores original test numbers 2023-04-28 07:22:17 +02:00
benbierens 96c2ade7ff
Allows for the use of UseLongTimeouts in combination with test-case attribute 2023-04-27 15:55:33 +02:00
benbierens d236fe6cea
Merge parallel-upload-download tests 2023-04-26 15:32:27 +02:00
benbierens e2586ddd26
Merge branch 'master' into feature/marketplace-contracts 2023-04-26 15:31:26 +02:00
benbierens 2ba5a1491a
Updates to latest image 2023-04-26 15:31:01 +02:00
Corbo12 ec311f4802 MDF change upload tests file name 2023-04-26 15:00:09 +02:00
Corbo12 aed3edaf6a MDF split tests into different files 2023-04-26 14:59:26 +02:00
benbierens 1c9e59a6ea
Bonus: sets up peer-tests 2023-04-26 14:40:54 +02:00
benbierens ef546a435b
Supports multiple codex nodes from a single geth companion node. 2023-04-26 11:12:33 +02:00
Corbo12 5d5b08e084 ADD multithreading 2023-04-25 13:43:51 +02:00
benbierens 62f4eed221
Removes BringOnline method 2023-04-25 12:52:11 +02:00
benbierens 9db35be2ec
Allows for nethereum interactions with companion nodes. Verifies that marketplace contract is available before proceeding with codex-node setup. 2023-04-24 14:09:23 +02:00
Corbo12 b70e8033f3 WIP merge with main 2023-04-20 15:58:47 +02:00
Corbo12 4492b4f0d2 Merge remote-tracking branch 'upstream/master' 2023-04-20 14:16:53 +02:00
Corbo12 71b061cac7 MDF threaded functions not running correctly 2023-04-20 14:16:04 +02:00
Corbo12 a7e024002f ADD upload tests 2023-04-20 12:43:25 +02:00
benbierens cdd51f6dc7
Sets up some durability tests. 2023-04-19 14:57:00 +02:00
Corbo12 18e59f3490 ADD Parallel download Tests 2023-04-19 12:34:46 +02:00
benbierens 5b132bcb54
Fixes connection from codex node to its companion 2023-04-19 11:23:11 +02:00
benbierens e9d84a5cf7
Better logging for json errors and node actions. 2023-04-19 10:42:08 +02:00
benbierens e4e7afd580
Better logging 2023-04-19 09:57:37 +02:00