Commit Graph

668 Commits

Author SHA1 Message Date
Alberto Soutullo Rendo a166eaed07 Added option to do simulation without sending messages 2023-09-05 11:53:16 +02:00
0xFugue 1e18d72de4
removes containers that "contain" the names to be excluded (#136) 2023-08-17 23:55:10 +05:30
0xFugue 451c0f0510
MiB to MegaBytes renaming and the "divide" option (#132)
* MiB to Mega Bytes renaming

* remove fifo explicitly

* modify the fifo name

* added the per-container "divide"; configurable
2023-08-17 21:34:29 +05:30
0xFugue 495fe8cd84
Gennet: added regular graph (#135)
* make regular graphs the default

* added regular graph

* network and degree distribution plots

* log network type

* handle n*d odd

* set fanout to 6

* title/ylable

* added --num-nodes --fanout to title

* remove prints in draw

* avg degree, float format

* config.json
2023-08-17 20:51:47 +05:30
Alberto Soutullo ceab94f247
Nomos testing in master (#141)
* Changed gennet to generate simple nomos topology

* Added nomos node to build.sh

* Updated nomos yaml trait

* Added nomos variables to system_variables.star

* Prepared nomos_builder.star

* Modify gennet to work with nomos topology and traits

* Gennet currently doesn't support mixed topologies between waku and nomos nodes.

* Restructured config.json to add separated testing and simulation functionalities

* Added assertions.star file

* Modified wakurtosis to set up integration tests

* Updated kurtosis version in README.md
2023-07-30 13:42:18 +02:00
Alberto Soutullo be18041437
Merge pull request #129 from vacp2p/Alberto/discv5
Discv5
2023-07-30 13:24:30 +02:00
Alberto Soutullo 48268813e0
Merge pull request #140 from vacp2p/Alberto/dump_prometheus
Added range to prometheus dumping data.
2023-07-24 11:05:05 +02:00
Alberto Soutullo b6566e395b Added range to prometheus dumping data. 2023-07-24 11:01:30 +02:00
Alberto Soutullo ae4cb0383c Added run_waku_node script 2023-07-17 18:05:53 +02:00
Alberto Soutullo 306fd03067 Downgrading kurtosis version because of docker subnetwork limitation 2023-07-17 18:05:22 +02:00
Alberto Soutullo Rendo 6f7e960362 Solved conflicts with prometheus dump 2023-07-15 12:52:17 +02:00
Alberto Soutullo Rendo 993013a45f Modified arp tables to make discv5 work in bigger simulations 2023-07-12 11:45:42 +02:00
Alberto Soutullo 7b6fa0ce28 Dumping prometheus information 2023-07-06 14:22:41 +02:00
Alberto Soutullo 956b9ba324 Modifications to Starlark so prometheus dumping works in wls 2023-07-06 14:22:26 +02:00
Alberto Soutullo 447029e473 Modified wls to accept prometheus arguments, to dump data once inyection is finished 2023-07-06 14:22:05 +02:00
Alberto Soutullo e627141586 Added prometheus lib in wls 2023-07-06 14:21:29 +02:00
Alberto Soutullo 43948698bf
Merge pull request #130 from logos-co/Alberto/cadvisor_bandwidth
Alberto/cadvisor bandwidth into master
2023-06-15 14:10:32 +02:00
Alberto 92439bf18b Disabled by_simulation plotting configuration 2023-06-13 11:27:15 +02:00
Alberto ef56eab3d7 Fixed bug in build.sh 2023-06-13 11:25:44 +02:00
Alberto Soutullo Rendo dfe6588e43 commented wrong lines 2023-06-12 18:00:50 +02:00
Alberto 26e9fccf27 IP cadvisor calculation also moved 2023-06-12 17:53:27 +02:00
Alberto 23bc0a2e2c added execution permissions to bash scripts 2023-06-12 17:10:11 +02:00
Alberto 81e7d9546c Fixed bug in build.sh 2023-06-12 17:02:39 +02:00
Alberto Soutullo Rendo 618190ca0f Moved out enclave preparation independently of the infra 2023-06-12 16:48:53 +02:00
Alberto Soutullo 48a8ee2ba3 Updated documentation 2023-06-09 18:27:00 +02:00
Alberto Soutullo 5c27ba6c94 Moved bash scripts to folder 2023-06-09 18:01:56 +02:00
Alberto Soutullo d862dc7e1b Deleted unnecesary functions 2023-06-09 17:56:35 +02:00
Alberto Soutullo 5da15275f8 Updated discv5.toml with default empty enr 2023-06-09 17:56:19 +02:00
Alberto Soutullo 3f09c5fc95 Resolved conflicts with main 2023-06-09 17:38:49 +02:00
Alberto Soutullo a8eadf31a9 Fixed tests 2023-06-09 16:08:19 +02:00
Alberto Soutullo d014ba2713 Cleaned files after cleaning git cache 2023-06-09 13:19:27 +02:00
Alberto Soutullo 3c206431a1 Deleted comment with older kurtosis version 2023-06-09 12:55:15 +02:00
Alberto Soutullo Rendo 9d9f424e0a updated gitignore 2023-06-09 12:39:12 +02:00
Alberto Soutullo Rendo 88d3554ec6 Deleted pycache tracked folders 2023-06-09 12:36:51 +02:00
Alberto Soutullo ec151f910f Added discv5 as default in gennet 2023-06-09 12:24:11 +02:00
Alberto Soutullo 12c4b44016 Added option to config.json to skip topology interconnection 2023-06-09 12:23:54 +02:00
Alberto Soutullo 912a4afcb2 Changed kurtosis to properly use discv5 2023-06-09 12:23:28 +02:00
Alberto Soutullo cb92208213 Reverted get toml from gennet to host again 2023-06-09 12:22:33 +02:00
Alberto Soutullo f626dd8aba Cleaned bootstrap node and ENR in cleanup.sh 2023-06-09 12:22:10 +02:00
Alberto Soutullo 0b9fa13612 Deleted bootstrap node from gennet.py 2023-06-09 12:20:04 +02:00
Alberto Soutullo 2b8365e48c Updated system_variables.star 2023-06-09 12:11:20 +02:00
Alberto Soutullo e58ceee4c4 Added scripts to run bootstrap node in host, and waku node in kurtosis 2023-06-09 12:10:57 +02:00
Alberto Soutullo d5378aa77f Now bootstrap node is run correctly in run.sh, and ENR is modified in traits to be inyected in gennet 2023-06-09 12:09:59 +02:00
Alberto Soutullo 6026ff7822 Added note in node.toml about port shift 2023-06-09 12:08:51 +02:00
Alberto Soutullo 0a00b909de Updated default values for rpc.toml 2023-06-09 12:08:04 +02:00
Alberto Soutullo ff9234a5f7 Updated default values for metrics.toml 2023-06-09 12:07:43 +02:00
Alberto Soutullo 8cf906bf27 Updated default values for discv5 2023-06-09 12:07:08 +02:00
Alberto Soutullo caf691ecbe Deleted unnecesary file 2023-06-09 12:06:27 +02:00
0xFugue 996b7323f4
dstats/host-proc analysis (#125)
* cli scaffolding for processings metrics

* pandas description: dstats

* added dstats columns

* added procfs columns

* prune procfs/dstats columns

* sanitize, convert dstats

* re-write dstats and size conversion

* faster Human2ByteConveter

* dstats violin plots

* dstats, all plots, cdf

* all plots, pdf

* added prefix option

* option for cdf/pdf plots

* add jordis propagation delay calc - partial

* dstats takes only one arg now

* refactor jordis module

* added file check to dstats

* make analysis classes singleton

* ProcFS read/processing

* refactoring; added Plot baseclass

* added TODO

* Plot, violin_plots_helper()

* added ContainerID

* procfs plots - v0

* add rpc/metrics and update config.json

* added settling time plots

* setting labels

* added InOctets/OutOctets

* procfs plots - v1

* MiBs

* TODO

* change perms for prometheus and grafana configs

* conditional import tomli/tomllib

* BLKR/BLKW in MiB

* grafana dashboard

* run.sh refactoring

* dstats is now a standalone module

* added dstats dir

* bug fix

* fast pid2node_name

* kurtosis version check

* dstats bug fix

* bug fix

* dstats analysis - v2

* container2node_name via kurtosis inspect

* added dstats to cleanup

* dstats - container2name - v3

* dstats - v3

* host-proc - v2

* delete old host-proc launcher

* host-proc - v2: inline cpu stats

* host proc v2 : analysis : cpu stats

* add kinspect to host-proc

* host-proc-helper.sh

* host-proc v4

* plots 4 total and avg metrics

* clusters: kmeans

* per container clusters

* panelling is now class-wide

* per container clusters

* take care of incomplete samples

* extract the maximal, complete sample set

* grouping is now done by Key + removed HWM

* host-proc bug fix

* uniformise run.sh conditionals

* bug fix run.sh

* --full-uuids

* remove kurtosis inspect altogether

* cleanup run.sh

* kurtosis run cleanup

* added container size

* host-proc-helper.py

* run.sh

* incorporated config.json

* run.sh

* analysis.py

* config.json .plotting.host-proc

* conditional histograms - v0

* host-proc-helper.py omit fields

* conditional histograms

* compare plots done

* config.json dstats/host-proc

* config.json dstats/host-proc

* dstats/host-proc plotting run.sh

* run.sh

* cmd_helper

* analysis requirements.txt

* monitoring/host-proc/analysis.py -> analysis-module/src/hproc.py

* run.sh

* make top_plot strict

* remove/modify prints

* cleanup the old dockers

* set default to compare plots

* hproc.py

* src/main.py

* Dockerfile main.py

* plot_compare_panel

* set common --out-prefix

* build.sh

* work around ifindex issues

* run.sh: synchronous sudo and supressed analysis output

* run.sh

* venv for host-proc

* cleanup container-proc outputs

* add run summary to plot titles

* delete monitoring/host-proc/analysis.py

* remove log.info()

* host-proc: profile only wakunodes with config

* col rename, output plot dir, medians

* CPU % fixed

* export all plots to wakurtosis_log

* setting defaults to config.json

* config.json: num_nodes >= fanout

* run.sh plot dirs

* MemUse to VmPeak/VmSize is now configurable

* install python3-venv

* K -> KiB; i -> fo in summary
2023-06-08 14:11:36 +05:30
Alberto Soutullo 955581c9b6 Changing structure, boostrap node is initiated in run.sh. ENR will be sent to gennet 2023-06-06 11:33:02 +02:00