Commit Graph

46 Commits

Author SHA1 Message Date
Alberto 21454c806c Fixed wrong function refactor 2023-05-08 12:38:19 +02:00
Alberto 2cc0857aa7 Added summary to plot 2023-05-08 12:38:19 +02:00
Alberto e8a65ae5c9 Added missing await in WLS 2023-04-20 12:42:25 +02:00
Alberto Soutullo 1f6f5397d3
Merge branch 'master' into Alberto/analysis_multinode 2023-04-20 10:36:51 +02:00
0xFugue c069923bd0
Data collection: docker stats and host-proc (#106)
* sysfs, docker stats, docker ps

* .gitignore

* updated file names

* cpu, mem, i/o

* .gitignore

* .gitignore

* added scheduler

* pruned mem/blk measurement

* cpu, mem, blk, net stats

* single node containers one

* renaming,bug fix,  signal handling

* code clean up; added all the necessary xlations to the logs

* cpustat, pid2docker

* remove temp. files

* tear_down, permission exception

* updating .gitignore

* param validation, sudo blk reads

* added output dir

* comments

* .gitignore & rename

* .gitignore & rename

* de-escalate privileges

* proc monitor

* container id to host end of  virtual interface

* complete code rewrite with minimal sudo section

* set default iface for non-docker wakus

* added signal handlers and clean shutdown

* added 3 different per-process network stats

* +VmData, +VmStk, handling of veth pairs, optimal cpu stats

* +VmHWM, net1=net3, net2

* +VmHWM, net1=net3, net2

* pre merge

* pre merge

* .gitignore

* adjusted paths & stats only nim/go waku

* monitor.sh

* adding docker wait, nim-waku:nwaku-trace2

* TTY bug

* copy config/metric dirs to logs

* cleanup.sh

* procfs monitor updates

* run.sh cleanup

* run.sh  metrics_infra cadvisor

* cleanup fix

* empty output dir

* host-proc

* added cadvisor conditionals

* enable WLS wait

* signal WLS once the measuring infra is up

* spin_up wls_cid

* remove print/exit

* renaming

* graceful signal exit, reduce logging freq

* stat only running waku instances

* comments, remove prints

* de-duplicate docker calls

* deduplicate docker calls in procfs.py

* multi nodes

* adding Jordi's signal'ng code and timing

* minimise the setup time

* split and move sudo ahead

* run.sh/procfs/docker stats named pipe synchornisation

* jack-up the max# open filehandles

* 100k file handles

* log timings, set resource limits

* complete rewrite of pid2did

* added start time + waku versions

* requirements.txt

* assert, stat collection duration

* main.sh -> dstats.sh
2023-04-20 00:16:02 +05:30
Alberto 8da7777a5b Fixed conflicts and disabling monitoring 2023-04-18 18:08:29 +02:00
Daimakaimura f0599313c0 Updated WLS to non blocking IO and added wait for signal 2023-04-18 16:56:55 +02:00
Daimakaimura a17c6d3179 Added aiohttp and asyncio dependencies 2023-04-18 16:56:55 +02:00
Alberto Soutullo d8e5067c93 Hotfix tests because changed separator in last PR. 2023-04-18 16:56:55 +02:00
Alberto Soutullo 139fedc2d6 Hotfix tests because changed separator in last PR. 2023-03-21 20:21:47 +01:00
Alberto Soutullo 8de3cd6968 Changed port name to match with kubernetes changes in wls 2023-03-16 15:51:12 +01:00
Alberto Soutullo caaf3f0741 Fixed test 2023-03-13 16:41:40 +01:00
Alberto Soutullo 09ee68e44c Changed wls payload to base64 2023-03-13 16:41:02 +01:00
Alberto Soutullo Rendo 5f45ae4a06 Deleted old wls 2023-03-10 19:24:20 +01:00
Alberto Soutullo 23c612153d Added test for uniform mode in wls and fixed bug 2023-03-10 12:41:24 +01:00
Alberto Soutullo fe268b84b3 Fixed imports in wls main 2023-03-10 12:41:24 +01:00
Alberto Soutullo ca73887361 Fixed imports in wls tests 2023-03-10 12:41:23 +01:00
Alberto Soutullo fcd814d8e8 Fixed pythonpath in wls dockerfile 2023-03-10 12:41:22 +01:00
Alberto Soutullo 1ae60e8395 Removed unnecessary init py in wls-module 2023-03-10 12:41:22 +01:00
Alberto Soutullo f4fb0bdd45 Removed unnecessary catches 2023-03-10 12:41:21 +01:00
Alberto Soutullo a774ecec84 Added tests for wls.py 2023-03-10 12:41:14 +01:00
Alberto Soutullo 1345575333 Finished test payloads 2023-03-10 12:41:14 +01:00
Alberto Soutullo c4ee242471 Remove comment in payload and added error 2023-03-10 12:41:14 +01:00
Alberto Soutullo da58f57994 Make wls actually take arguments 2023-03-10 12:41:13 +01:00
Alberto Soutullo 7ddbbc2e53 Added tests for payloads 2023-03-10 12:41:12 +01:00
Alberto Soutullo d640d977ce Added tests for waku messaging 2023-03-10 12:41:12 +01:00
Alberto Soutullo 8db84f0324 Added exit code to sys.exit 2023-03-10 12:41:11 +01:00
Alberto Soutullo 18f7f585ab Cleaned imports 2023-03-10 12:41:11 +01:00
Alberto Soutullo 557e149a2e Added tests for Moved node builder star files to types folder.py 2023-03-10 12:41:10 +01:00
Alberto Soutullo ca4bb88931 Refactored payloads.py to add tests 2023-03-10 12:41:10 +01:00
Alberto Soutullo 112ed64d40 Deleted unnecessary check 2023-03-10 12:41:09 +01:00
Alberto Soutullo 523020c588 Fixed logger and wls.py call before starting with tests 2023-03-10 12:41:09 +01:00
Alberto Soutullo 4edd198793 More refactor and PEP-8 2023-03-10 12:41:08 +01:00
Alberto Soutullo 1236ae679b Refactor and restructured wls 2023-03-10 12:41:08 +01:00
Alberto Soutullo db6996ded1 First refactor after rebase with Analysis 2023-03-10 12:41:07 +01:00
Alberto Soutullo 013a799dac WIP: Starting clean code to refactor as it already works for several nodes per container. 2023-03-10 12:41:06 +01:00
Alberto Soutullo 3fdc0f1602 Changed wls to work with kurtosis information 2023-03-10 12:41:06 +01:00
Alberto Soutullo a0789087eb WIP trying to pass new network information to wls 2023-03-10 12:41:05 +01:00
Gusto Bacvinka 7cda520f51 Add multiple nomos nodes per container support 2023-03-10 12:41:04 +01:00
Gusto Bacvinka 5c81b48b4d Nomos wsl statistics 2023-03-10 12:41:03 +01:00
Gusto Bacvinka 600c9e9693 Interconnect waku and nomos nodes via the same method 2023-03-10 12:41:02 +01:00
Gusto Bacvinka 2df81532ec Simplify dockerfiles and mark files as executable 2023-03-10 12:41:00 +01:00
Gusto Bacvinka 34d903e796 Extend nomos wsl and add visualizations 2023-03-10 12:41:00 +01:00
Gusto Bacvinka cb81c8676b Gennet and WSL changes for nomos node 2023-03-10 12:40:58 +01:00
Alberto Soutullo 777f67b1b4 Last wsl to wls changes 2023-03-10 12:40:54 +01:00
Alberto Soutullo 13ebdea853 Renamed propperly WSL to WLS to avoid confussions 2023-03-10 12:40:54 +01:00