Commit Graph

668 Commits

Author SHA1 Message Date
Alberto 1c4b1a2f8e Now prometheus can accept a variable number of metrics to gather 2023-05-08 12:38:19 +02:00
Alberto 12c80086ad Added continue back to fix bug 2023-05-08 12:38:19 +02:00
Alberto 88f4913663 Converted data to megabytes with flag, and fixed bug in continue when fetching metrics 2023-05-08 12:38:19 +02:00
Alberto a8d0bacfea Deleted old plotting function 2023-05-08 12:38:19 +02:00
Alberto e2a20bfa1d Prepared prometheus script to handle list of given metrics 2023-05-08 12:38:19 +02:00
Alberto 9d702fc920 Cleaned main 2023-05-08 12:38:19 +02:00
Alberto ce85ec3ced Now just one mount is needed 2023-05-08 12:38:19 +02:00
Alberto 21454c806c Fixed wrong function refactor 2023-05-08 12:38:19 +02:00
Alberto 091b5a3511 Added analysis to build 2023-05-08 12:38:19 +02:00
Alberto 2cc0857aa7 Added summary to plot 2023-05-08 12:38:19 +02:00
Alberto f0aba7d585 Set metrics back to "good ones" 2023-05-08 12:38:19 +02:00
Alberto 92988ed7f3 Added disk usage and refactored plot 2023-05-08 12:38:19 +02:00
Alberto eb0a9a0202 More flags true by default 2023-05-08 12:38:19 +02:00
Alberto 8406463388 Savd rpc and metrics by default in config 2023-05-08 12:38:19 +02:00
Daimakaimura 885cc9ac65 Added analysis build 2023-05-05 11:42:30 +01:00
Daimakaimura 3305fb3682 Updated analysis module README 2023-05-05 11:32:31 +01:00
Daimakaimura fe94b683ce Added container-proc logic 2023-05-05 11:25:09 +01:00
Daimakaimura c01d8adc3e Throws warnings if latencies are empty 2023-05-05 11:24:50 +01:00
Daimakaimura 7426297531 Added config logger in the analysis module 2023-05-05 11:23:48 +01:00
Daimakaimura b68e66ee65 Minor 2023-05-04 15:26:50 +01:00
Daimakaimura f4f75a1135 Changed container-proc metrics path to inside wakurtosis-logs 2023-05-04 15:26:40 +01:00
Daimakaimura 2f119b1aad Updated Nim-Waku image 2023-05-04 15:06:25 +01:00
Daimakaimura 447e56d0b3 Updated the monitoring paths and the probe version 2023-05-04 14:53:50 +01:00
Daimakaimura 24a204fcd5 Minor 2023-05-04 14:53:33 +01:00
Daimakaimura c5dc1dce6c Created container-proc folder and moved everything related 2023-05-04 14:43:44 +01:00
Daimakaimura e3f31cafa1 Emable container-proc monitoring and cleanup 2023-05-04 14:43:14 +01:00
Alberto Soutullo d7fe55f8b5
Merge pull request #107 from logos-co/Alberto/analysis_multinode
Analysis multinode into master
2023-04-27 21:23:54 +02:00
Alberto b09e496d19 Updated readme with right command and flags 2023-04-27 11:53:49 +02:00
Alberto 1a4369bd78 Changed prometheus address so it works with both ubuntu and windows subnetworks 2023-04-27 11:53:31 +02:00
Alberto 8a8e8f0252 Signaling wls when also using cadvisor 2023-04-27 11:51:14 +02:00
Alberto 0a9739d161 Fixed bug in glob when grabbing containers names 2023-04-20 13:39:47 +02:00
Alberto e8a65ae5c9 Added missing await in WLS 2023-04-20 12:42:25 +02:00
Alberto 51295525d8 Added missing line when merging 2023-04-20 12:42:05 +02:00
Alberto Soutullo 1f6f5397d3
Merge branch 'master' into Alberto/analysis_multinode 2023-04-20 10:36:51 +02:00
0xFugue c069923bd0
Data collection: docker stats and host-proc (#106)
* sysfs, docker stats, docker ps

* .gitignore

* updated file names

* cpu, mem, i/o

* .gitignore

* .gitignore

* added scheduler

* pruned mem/blk measurement

* cpu, mem, blk, net stats

* single node containers one

* renaming,bug fix,  signal handling

* code clean up; added all the necessary xlations to the logs

* cpustat, pid2docker

* remove temp. files

* tear_down, permission exception

* updating .gitignore

* param validation, sudo blk reads

* added output dir

* comments

* .gitignore & rename

* .gitignore & rename

* de-escalate privileges

* proc monitor

* container id to host end of  virtual interface

* complete code rewrite with minimal sudo section

* set default iface for non-docker wakus

* added signal handlers and clean shutdown

* added 3 different per-process network stats

* +VmData, +VmStk, handling of veth pairs, optimal cpu stats

* +VmHWM, net1=net3, net2

* +VmHWM, net1=net3, net2

* pre merge

* pre merge

* .gitignore

* adjusted paths & stats only nim/go waku

* monitor.sh

* adding docker wait, nim-waku:nwaku-trace2

* TTY bug

* copy config/metric dirs to logs

* cleanup.sh

* procfs monitor updates

* run.sh cleanup

* run.sh  metrics_infra cadvisor

* cleanup fix

* empty output dir

* host-proc

* added cadvisor conditionals

* enable WLS wait

* signal WLS once the measuring infra is up

* spin_up wls_cid

* remove print/exit

* renaming

* graceful signal exit, reduce logging freq

* stat only running waku instances

* comments, remove prints

* de-duplicate docker calls

* deduplicate docker calls in procfs.py

* multi nodes

* adding Jordi's signal'ng code and timing

* minimise the setup time

* split and move sudo ahead

* run.sh/procfs/docker stats named pipe synchornisation

* jack-up the max# open filehandles

* 100k file handles

* log timings, set resource limits

* complete rewrite of pid2did

* added start time + waku versions

* requirements.txt

* assert, stat collection duration

* main.sh -> dstats.sh
2023-04-20 00:16:02 +05:30
Alberto 71478bf7c0 Quick readme update 2023-04-18 18:24:18 +02:00
Alberto 96b43246dd Deleted old analysis 2023-04-18 18:12:00 +02:00
Alberto 8da7777a5b Fixed conflicts and disabling monitoring 2023-04-18 18:08:29 +02:00
Alberto ca33cf9a01 commit before rebasing 2023-04-18 16:57:42 +02:00
Daimakaimura bcb0073a95 Runs the analyis at the end of the simulation 2023-04-18 16:57:42 +02:00
Daimakaimura ff1d77d74c Added process level stats and simulation summary output 2023-04-18 16:57:41 +02:00
Daimakaimura c7b92caee4 Stop and delete the enclave at the end 2023-04-18 16:56:55 +02:00
Daimakaimura f0599313c0 Updated WLS to non blocking IO and added wait for signal 2023-04-18 16:56:55 +02:00
Daimakaimura 3d030ebdcf Fixed WLS pattern 2023-04-18 16:56:55 +02:00
Daimakaimura 99d59db21f Fixed WLS monitoring pattern 2023-04-18 16:56:55 +02:00
Daimakaimura a17c6d3179 Added aiohttp and asyncio dependencies 2023-04-18 16:56:55 +02:00
Daimakaimura a536dc6e5d Added "monitoring" section into config.json 2023-04-18 16:56:55 +02:00
Daimakaimura 9d797b639c Added clean_repo, p_run and p_analasys scripts 2023-04-18 16:56:55 +02:00
Daimakaimura 6fc90e523a Added new monitor.py and probe.sh 2023-04-18 16:56:55 +02:00
0xFugue cf24c7f44e removed tomli, traits into dockerfile, build/run context update (#104)
* removed tomli, traits into dockerfile, build/run context update

* issue 98

* build_docker.sh, --traits-dir

* minor edits
2023-04-18 16:56:55 +02:00