Commit Graph

119 Commits

Author SHA1 Message Date
Benjamin Arntzen 7f95301509 Various fixes 2024-09-22 21:35:24 -04:00
Benjamin Arntzen f8dbcfb3b3 Remove breaking flag from gowaku 2024-09-22 20:07:21 -04:00
Benjamin Arntzen 5d0b6ce5f3 Fix typo in waku binary name 2024-09-16 07:12:20 -04:00
Benjamin Arntzen 4df7a77d0f Deployment 2024-09-16 07:11:20 -04:00
Benjamin Arntzen 7b9aa7206e Switch gowaku binary path 2024-09-16 12:09:08 +01:00
Benjamin Arntzen 6d7605b0ea Switch gowaku-preview binary path 2024-09-16 12:08:29 +01:00
Benjamin Arntzen 6984ae546f new script, new bootstrap 2024-09-16 07:07:10 -04:00
Benjamin Arntzen 36e9e6d324 Publisher changes 2024-09-16 00:49:52 -04:00
Benjamin Arntzen e8620faeed Finished manifest work for heterogenous deploy, working on deployment scripts now 2024-09-16 00:47:59 -04:00
Benjamin Arntzen 22931183a4 Reorganise docker images 2024-09-16 05:11:06 +01:00
Benjamin Arntzen d928e10e6e Working towards two heterogenous sims 2024-09-15 23:49:04 -04:00
Benjamin Arntzen a8b1e6411c Fix perf reporting 2024-09-02 13:10:50 +01:00
Benjamin Arntzen 7b0a24f616 Switch to normal mount instead of CSI mount 2024-08-16 00:34:50 +09:00
Benjamin Arntzen 7897513a32 Add privileges to perf containr 2024-08-13 14:31:36 +01:00
Benjamin Arntzen b6674c01cb Return readiness probe 2024-08-12 16:35:56 +01:00
Benjamin Arntzen ff5e3ee9ad Final working perf example 2024-08-12 16:31:06 +01:00
Benjamin Arntzen b5ca85c651 Add perf to image 2024-08-12 15:47:39 +01:00
Benjamin Arntzen d5d42ec919 Check in 2024-08-12 15:43:09 +01:00
Benjamin Arntzen cd549c8814 Add chmod +x 2024-06-17 09:59:14 +01:00
Alberto Soutullo 6afe499ca9
Update waku tracer to version 0.28 (#34) 2024-06-12 22:02:27 +02:00
Alberto Soutullo f20e56be30
Ignore columns option for plotting (#32)
* Renamed functions and split responsibilities. Added argument to ignore columns

* PR suggestions: Remove constructor
2024-06-11 14:48:18 +02:00
Alberto Soutullo 6de8efb1ad
Alberto/single axs fix (#33)
* Fix bux where there is a single axs in a figure

* Force legend to top right corner
2024-06-11 11:59:56 +02:00
Alberto Soutullo 6786960c92
Nim-libp2p deploy (#31)
* Add yamls for nim-libp2p deployment

* Delete old deployment file

* Rename folder
2024-05-29 11:01:58 +02:00
Alberto Soutullo 7c0a786536
Publisher changes (#30)
* Add:
- Nº of message
- Handle exceptions
- Correctly sent KBytes
- Send X nº of messages

* Change default value for delay
2024-05-27 19:52:45 +01:00
Alberto Soutullo 07ef73dd58
Update yamls (#29) 2024-05-24 13:59:15 +02:00
Alberto Soutullo 249f35a970
Add new parameter for selecting how many datapoints we will add to the plots (#28) 2024-05-24 11:56:38 +02:00
Alberto Soutullo a6a21a4045
Extend query to multiple timestamps (#27)
* Added midstrap for ordering

* Changed scrapper for query a batch of timestamps

* Changed debug level to better tracking

* Updated scrape to have as example
2024-05-23 16:22:25 +02:00
Alberto Soutullo 2a32cb468a
Improved logging and error checking (#26) 2024-05-23 15:50:24 +02:00
Alberto Soutullo 86445bab0c
Alberto/graph visualization into main (#25)
* WIP: Create plot using qt5 in jupyter. Select from first timestamp to given one, and message hash.

* WIP: First approach efficiently replot subgraph saving positions

* WIP: Maintain ax limits between plots for easier visualization

* WIP: Plot elapsed time on edges with msg hash

* WIP: Correctly calculate elapsed time based on msg hash.

* WIP: Change multiindex order. Now dropdown only shows timestamps related to msg hash

* WIP: Moved functions to external files. Figure crashes with weird behavior.

* WIP: Fix Python crashing with jupyter backend
2024-05-22 13:02:24 +02:00
Alberto Soutullo c5ce9dbf43
Msg tracker - parsing (#24)
* WIP: Parsing and creating df from log files done

* First tracking version structure

* Add PR suggestions
2024-05-13 16:42:24 +02:00
Alberto Soutullo 8fe63202b8
Download pod logs (#21)
* Added method to download pod logs

* Fixed typo in ok_value

* Added kubeconfig in yaml file

* Added missing comment
2024-04-15 16:01:00 +02:00
Alberto Soutullo 1be49b98e1
Update tests scrapper and utils (#20)
* Updated tests in scrapper class

* Added utils tests

* Deleted old files

* Added PR suggestions.
2024-04-09 10:02:41 +02:00
Alberto Soutullo a50b182126
Update plotter and data classes tests (#19)
* Delete unnecessary method for concat dfs.

* Added tests for data_handler

* Deleted todo as it is not needed

* Added tests for data file handler

* Moved add file as mean to parent class

* Fixed new classes for more than one folder usecase

* Adjusted test data handler to be more understandable

* Fixed tests in data file handler

* Moved test to correspondant classes

* Renamed variables

* Minor format

* Added tests for data request handler

* Added tests for plotter

* Removed unnecessary comments

* Changed setUp for setUpClass

* Changed test data files to resources folder
2024-04-03 18:12:32 +02:00
Alberto Soutullo 2ca551ce44
Yaml improvements - plotter (#18)
* Add new fields to yaml

* Using new yaml parameters in plotter class

* Update src/plotting/plotter.py

string format

Co-authored-by: Álex Cabeza Romero <alex93cabeza@gmail.com>

---------

Co-authored-by: Álex Cabeza Romero <alex93cabeza@gmail.com>
2024-03-27 18:04:34 +01:00
Alberto Soutullo 657562b198
Scrape yaml improvements (#17)
* Changed YAML to possible variance

* Remove out folder from Scrapper parameters

* Corrected YAML indentation

* Use correct yaml format to create_query

* Use correct yaml format to when using dump data from scrapper

* Removed file utils and correctly move them to path

* Final yaml version

* Removed unnecessary list from yaml structure

* Use Path.parent, and remove wrongly added methods

* Removed other wrongly added function hehe
2024-03-27 16:41:17 +01:00
Alberto Soutullo 52cc8b4396
Alberto/data manager (#16)
* WIP: Trying portforward

* Created Kubernetes class to manage api inside scraper class.

* Modify scrapper class to use kubernetes class

* Extracted get query data to utils and deleted unused function

* Cleaned main.py

* Fix bug with pod name

* Typing and cleaning

* Extract order function to list utils file

* Extract path function to path utils file

* Created data handler class

* Changed scrapper to use data handler class

* Cleaned imports

* Alex PR suggestions

* Removed unnecessary if

* Extract order function to list utils file

* Extract path function to path utils file

* Created data handler class

* Changed scrapper to use data handler class

* Cleaned imports

* Add function to file_utils.py

* Created structure of data handler classes

* Created Parent class for data handler

* Modified data file handler

* Created data request handler

* Changed file utils to use Path instead of str

* Use correct class in scrapper

* Abstracted plotter to use data handler

* Added some typing left

* Modify scrapper class to use kubernetes class

* Extract order function to list utils file

* Extract path function to path utils file

* Created data handler class

* Changed scrapper to use data handler class

* Cleaned imports

* Changed scrapper to use data handler class

* Cleaned imports

* Add function to file_utils.py

* Created structure of data handler classes

* Created Parent class for data handler

* Modified data file handler

* Created data request handler

* Changed file utils to use Path instead of str

* Use correct class in scrapper

* Abstracted plotter to use data handler

* Added some typing left

* Remove redundant method for check if a Path exists

* Changed Ok content return from message to object.

* Changed get_dataframe method for property, and moved it to parent class

* Removed unnecessary method to get the file name of a Path

* Changed list comprehension to use Path instead of os
2024-03-27 15:16:55 +01:00
Alberto Soutullo 774d544b06
Add Thanos portforward (#15)
* WIP: Trying portforward

* Created Kubernetes class to manage api inside scraper class.

* Modify scrapper class to use kubernetes class

* Extracted get query data to utils and deleted unused function

* Cleaned main.py

* Fix bug with pod name

* Typing and cleaning

* Alex PR suggestions

* Removed unnecessary if
2024-03-21 13:08:27 +01:00
Alberto Soutullo 2b60bf6555
Merge pull request #13 from vacp2p/plotter
Plotter into master
2024-03-14 15:21:15 +01:00
Alberto Soutullo 704c38eaf2
Merge remote-tracking branch 'origin/plotter' into plotter
# Conflicts:
#	scrape.yaml
#	src/utils/file_utils.py
2024-03-14 15:19:57 +01:00
Alberto Soutullo ff22e16d3f
Added readme for plotter. 2024-03-14 15:17:55 +01:00
Alberto Soutullo 103d3b724d
WIP: test for plotter 2024-03-14 15:17:54 +01:00
Alberto Soutullo f14c66bf3c
Refactored plotter 2024-03-14 15:17:53 +01:00
Alberto Soutullo 9afc94b28a
Fixed logger to avoid logging external libraries 2024-03-14 15:17:52 +01:00
Alberto Soutullo 42fc400c8d
Fixed scrapper test 2024-03-14 15:17:51 +01:00
Alberto Soutullo 6c0f119498
Changed Unix Timestamp for Time when creating csv 2024-03-14 15:17:51 +01:00
Alberto Soutullo 0fc2fb967e
Improved plotter.py 2024-03-14 15:17:49 +01:00
Alberto Soutullo f1a4f65459
Cleaned plotter and modified yaml file. Also added np to requirements 2024-03-14 15:17:19 +01:00
Alberto Soutullo 5c6e928516
First Logger wip version 2024-03-14 15:17:16 +01:00
Alberto Soutullo 569f144415
WIP: Added versions of plotters 2024-03-14 15:14:23 +01:00
Alberto Soutullo 0a7d508366
Merge pull request #12 from vacp2p/metrics_scrapper
Improve scrape interval
2024-03-14 15:09:26 +01:00