vacp2p
/
wakurtosis
mirror of https://github.com/vacp2p/wakurtosis.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
611 Commits 51 Branches 0 Tags 1.6 MiB
Commit Graph

48 Commits

Author SHA1 Message Date
Alberto Soutullo b6566e395b Added range to prometheus dumping data. 2023-07-24 11:01:30 +02:00
Alberto Soutullo 447029e473 Modified wls to accept prometheus arguments, to dump data once inyection is finished 2023-07-06 14:22:05 +02:00
Alberto Soutullo e627141586 Added prometheus lib in wls 2023-07-06 14:21:29 +02:00
Alberto Soutullo a6d36d04da
Analysis multinode refactor (#113) 
* Savd rpc and metrics by default in config

* More flags true by default

* Added disk usage and refactored plot

* Set metrics back to "good ones"

* Added summary to plot

* Added analysis to build

* Fixed wrong function refactor

* Now just one mount is needed

* Cleaned main

* Prepared prometheus script to handle list of given metrics

* Deleted old plotting function

* Converted data to megabytes with flag, and fixed bug in continue when fetching metrics

* Added continue back to fix bug

* Now prometheus can accept a variable number of metrics to gather

* Modified plotting function to accept a variable number of metrics

* wip in main to work with variable number of metrics

* Plotting is now read in config.json

* Cleaned main

* Cleaned plotting file

* Created function to put custom metrics inside metrics dict

* Fixed error when having multiple metrics in same plot

* Updated config to have plotting information as example

* Updated dockerfile

* Updated README.md

* Renamed variable and function

* Deleted toml folder from arg parser

* Deleted print in plotting.py

* Deleted toml path from vars file

* Deleted unnused functions in prometheus.py

* updated main to not use tomls folder

* Added more tests in test_analysis.py

* refactored arg_parser.py test

* Added more tests in log_parser

* Created prometheus tests

* Finalized prometheus tests

* Fixed bug in prometheus, if data does not exists (yet, like could be container_fs_writes_bytes_total), we add a zero.

* Added xticks in plots

* Typo

* Updated readme
 2023-05-09 21:55:49 +02:00
Alberto e8a65ae5c9 Added missing await in WLS 2023-04-20 12:42:25 +02:00
Alberto Soutullo 1f6f5397d3
Merge branch 'master' into Alberto/analysis_multinode 2023-04-20 10:36:51 +02:00
0xFugue c069923bd0
Data collection: docker stats and host-proc (#106) 
* sysfs, docker stats, docker ps

* .gitignore

* updated file names

* cpu, mem, i/o

* .gitignore

* .gitignore

* added scheduler

* pruned mem/blk measurement

* cpu, mem, blk, net stats

* single node containers one

* renaming,bug fix,  signal handling

* code clean up; added all the necessary xlations to the logs

* cpustat, pid2docker

* remove temp. files

* tear_down, permission exception

* updating .gitignore

* param validation, sudo blk reads

* added output dir

* comments

* .gitignore & rename

* .gitignore & rename

* de-escalate privileges

* proc monitor

* container id to host end of  virtual interface

* complete code rewrite with minimal sudo section

* set default iface for non-docker wakus

* added signal handlers and clean shutdown

* added 3 different per-process network stats

* +VmData, +VmStk, handling of veth pairs, optimal cpu stats

* +VmHWM, net1=net3, net2

* +VmHWM, net1=net3, net2

* pre merge

* pre merge

* .gitignore

* adjusted paths & stats only nim/go waku

* monitor.sh

* adding docker wait, nim-waku:nwaku-trace2

* TTY bug

* copy config/metric dirs to logs

* cleanup.sh

* procfs monitor updates

* run.sh cleanup

* run.sh  metrics_infra cadvisor

* cleanup fix

* empty output dir

* host-proc

* added cadvisor conditionals

* enable WLS wait

* signal WLS once the measuring infra is up

* spin_up wls_cid

* remove print/exit

* renaming

* graceful signal exit, reduce logging freq

* stat only running waku instances

* comments, remove prints

* de-duplicate docker calls

* deduplicate docker calls in procfs.py

* multi nodes

* adding Jordi's signal'ng code and timing

* minimise the setup time

* split and move sudo ahead

* run.sh/procfs/docker stats named pipe synchornisation

* jack-up the max# open filehandles

* 100k file handles

* log timings, set resource limits

* complete rewrite of pid2did

* added start time + waku versions

* requirements.txt

* assert, stat collection duration

* main.sh -> dstats.sh
 2023-04-20 00:16:02 +05:30
Alberto 8da7777a5b Fixed conflicts and disabling monitoring 2023-04-18 18:08:29 +02:00
Daimakaimura f0599313c0 Updated WLS to non blocking IO and added wait for signal 2023-04-18 16:56:55 +02:00
Daimakaimura a17c6d3179 Added aiohttp and asyncio dependencies 2023-04-18 16:56:55 +02:00
Alberto Soutullo d8e5067c93 Hotfix tests because changed separator in last PR. 2023-04-18 16:56:55 +02:00
Alberto Soutullo 139fedc2d6 Hotfix tests because changed separator in last PR. 2023-03-21 20:21:47 +01:00
Alberto Soutullo 8de3cd6968 Changed port name to match with kubernetes changes in wls 2023-03-16 15:51:12 +01:00
Alberto Soutullo caaf3f0741 Fixed test 2023-03-13 16:41:40 +01:00
Alberto Soutullo 09ee68e44c Changed wls payload to base64 2023-03-13 16:41:02 +01:00
Alberto Soutullo Rendo 5f45ae4a06 Deleted old wls 2023-03-10 19:24:20 +01:00
Alberto Soutullo 23c612153d Added test for uniform mode in wls and fixed bug 2023-03-10 12:41:24 +01:00
Alberto Soutullo fe268b84b3 Fixed imports in wls main 2023-03-10 12:41:24 +01:00
Alberto Soutullo ca73887361 Fixed imports in wls tests 2023-03-10 12:41:23 +01:00
Alberto Soutullo fcd814d8e8 Fixed pythonpath in wls dockerfile 2023-03-10 12:41:22 +01:00
Alberto Soutullo 1ae60e8395 Removed unnecessary init py in wls-module 2023-03-10 12:41:22 +01:00
Alberto Soutullo f4fb0bdd45 Removed unnecessary catches 2023-03-10 12:41:21 +01:00
Alberto Soutullo a774ecec84 Added tests for wls.py 2023-03-10 12:41:14 +01:00
Alberto Soutullo 1345575333 Finished test payloads 2023-03-10 12:41:14 +01:00
Alberto Soutullo c4ee242471 Remove comment in payload and added error 2023-03-10 12:41:14 +01:00
Alberto Soutullo da58f57994 Make wls actually take arguments 2023-03-10 12:41:13 +01:00
Alberto Soutullo 7ddbbc2e53 Added tests for payloads 2023-03-10 12:41:12 +01:00
Alberto Soutullo d640d977ce Added tests for waku messaging 2023-03-10 12:41:12 +01:00
Alberto Soutullo 8db84f0324 Added exit code to sys.exit 2023-03-10 12:41:11 +01:00
Alberto Soutullo 18f7f585ab Cleaned imports 2023-03-10 12:41:11 +01:00
Alberto Soutullo 557e149a2e Added tests for Moved node builder star files to types folder.py 2023-03-10 12:41:10 +01:00
Alberto Soutullo ca4bb88931 Refactored payloads.py to add tests 2023-03-10 12:41:10 +01:00
Alberto Soutullo 112ed64d40 Deleted unnecessary check 2023-03-10 12:41:09 +01:00
Alberto Soutullo 523020c588 Fixed logger and wls.py call before starting with tests 2023-03-10 12:41:09 +01:00
Alberto Soutullo 4edd198793 More refactor and PEP-8 2023-03-10 12:41:08 +01:00
Alberto Soutullo 1236ae679b Refactor and restructured wls 2023-03-10 12:41:08 +01:00
Alberto Soutullo db6996ded1 First refactor after rebase with Analysis 2023-03-10 12:41:07 +01:00
Alberto Soutullo 013a799dac WIP: Starting clean code to refactor as it already works for several nodes per container. 2023-03-10 12:41:06 +01:00
Alberto Soutullo 3fdc0f1602 Changed wls to work with kurtosis information 2023-03-10 12:41:06 +01:00
Alberto Soutullo a0789087eb WIP trying to pass new network information to wls 2023-03-10 12:41:05 +01:00
Gusto Bacvinka 7cda520f51 Add multiple nomos nodes per container support 2023-03-10 12:41:04 +01:00
Gusto Bacvinka 5c81b48b4d Nomos wsl statistics 2023-03-10 12:41:03 +01:00
Gusto Bacvinka 600c9e9693 Interconnect waku and nomos nodes via the same method 2023-03-10 12:41:02 +01:00
Gusto Bacvinka 2df81532ec Simplify dockerfiles and mark files as executable 2023-03-10 12:41:00 +01:00
Gusto Bacvinka 34d903e796 Extend nomos wsl and add visualizations 2023-03-10 12:41:00 +01:00
Gusto Bacvinka cb81c8676b Gennet and WSL changes for nomos node 2023-03-10 12:40:58 +01:00
Alberto Soutullo 777f67b1b4 Last wsl to wls changes 2023-03-10 12:40:54 +01:00
Alberto Soutullo 13ebdea853 Renamed propperly WSL to WLS to avoid confussions 2023-03-10 12:40:54 +01:00