* Fix get_files_from_folder_path when using '*'
* Add _get_number_nodes function
* Add option to concurrently query for logs
* Comment sleep in fetch data
* Change logging
* Add function to dump csv
* Merge dfs and dump
* Change path file name to path_u
* Change pattern in extension function
* Return error when single queries go through catch
* Enhance result returning
* Add extension when reading files to not have problems with already existing ones
* Add intermediate folder for message summary
* Fix folder path
* Parallelize dump log information
* Change read logs parallelization
* Split functions for reading logs concurrently
* Add result error checking
* Enhance typing
* Change query to get nº nodes
* Improve writing csv
* Sort dataframes
* Better handle result in get number of nodes
* Improve result error response
* Add missing pandas as pd import
* Change log file name to peer_id
* Logger changes
* Add file name info to logging
* Address PR comment.
* Improved logging
* Changed to work with sender_id logs
* Update requirements.txt
* Adjusted logger to be used from other work directories
* Modified waku tracer to work with multiple patterns
* Created victoria reader
* Clean victoria_reader.py file
* Refactored victoria and waku reader to work with multiple queries
* Add method for sent pattern
* Portforwarding not currently needed
* Fix wrong arguments in patterns
* Add message reliability functions
* Add pod information
* Improved waku traced and victoria reader
* Add waku log analyzer class
* Add example script
* Use Path in File reader
* Use Path in file_utils
* Use Path in path.py
* Refactor single query info
* Add path and result to waku tracer
* Complete waku message log analyzer functionality
* Add time between requests as Victoria metrics DNS is refusing connections if too much are done in short time
* Finish example analysis file
* Deleted unused code from notebook
* PR review comments
* Add comment to analyze_message_timestamps
* Added midstrap for ordering
* Changed scrapper for query a batch of timestamps
* Changed debug level to better tracking
* Updated scrape to have as example
* WIP: Create plot using qt5 in jupyter. Select from first timestamp to given one, and message hash.
* WIP: First approach efficiently replot subgraph saving positions
* WIP: Maintain ax limits between plots for easier visualization
* WIP: Plot elapsed time on edges with msg hash
* WIP: Correctly calculate elapsed time based on msg hash.
* WIP: Change multiindex order. Now dropdown only shows timestamps related to msg hash
* WIP: Moved functions to external files. Figure crashes with weird behavior.
* WIP: Fix Python crashing with jupyter backend
* Delete unnecessary method for concat dfs.
* Added tests for data_handler
* Deleted todo as it is not needed
* Added tests for data file handler
* Moved add file as mean to parent class
* Fixed new classes for more than one folder usecase
* Adjusted test data handler to be more understandable
* Fixed tests in data file handler
* Moved test to correspondant classes
* Renamed variables
* Minor format
* Added tests for data request handler
* Added tests for plotter
* Removed unnecessary comments
* Changed setUp for setUpClass
* Changed test data files to resources folder
* Add new fields to yaml
* Using new yaml parameters in plotter class
* Update src/plotting/plotter.py
string format
Co-authored-by: Álex Cabeza Romero <alex93cabeza@gmail.com>
---------
Co-authored-by: Álex Cabeza Romero <alex93cabeza@gmail.com>
* Changed YAML to possible variance
* Remove out folder from Scrapper parameters
* Corrected YAML indentation
* Use correct yaml format to create_query
* Use correct yaml format to when using dump data from scrapper
* Removed file utils and correctly move them to path
* Final yaml version
* Removed unnecessary list from yaml structure
* Use Path.parent, and remove wrongly added methods
* Removed other wrongly added function hehe
* WIP: Trying portforward
* Created Kubernetes class to manage api inside scraper class.
* Modify scrapper class to use kubernetes class
* Extracted get query data to utils and deleted unused function
* Cleaned main.py
* Fix bug with pod name
* Typing and cleaning
* Extract order function to list utils file
* Extract path function to path utils file
* Created data handler class
* Changed scrapper to use data handler class
* Cleaned imports
* Alex PR suggestions
* Removed unnecessary if
* Extract order function to list utils file
* Extract path function to path utils file
* Created data handler class
* Changed scrapper to use data handler class
* Cleaned imports
* Add function to file_utils.py
* Created structure of data handler classes
* Created Parent class for data handler
* Modified data file handler
* Created data request handler
* Changed file utils to use Path instead of str
* Use correct class in scrapper
* Abstracted plotter to use data handler
* Added some typing left
* Modify scrapper class to use kubernetes class
* Extract order function to list utils file
* Extract path function to path utils file
* Created data handler class
* Changed scrapper to use data handler class
* Cleaned imports
* Changed scrapper to use data handler class
* Cleaned imports
* Add function to file_utils.py
* Created structure of data handler classes
* Created Parent class for data handler
* Modified data file handler
* Created data request handler
* Changed file utils to use Path instead of str
* Use correct class in scrapper
* Abstracted plotter to use data handler
* Added some typing left
* Remove redundant method for check if a Path exists
* Changed Ok content return from message to object.
* Changed get_dataframe method for property, and moved it to parent class
* Removed unnecessary method to get the file name of a Path
* Changed list comprehension to use Path instead of os
* WIP: Trying portforward
* Created Kubernetes class to manage api inside scraper class.
* Modify scrapper class to use kubernetes class
* Extracted get query data to utils and deleted unused function
* Cleaned main.py
* Fix bug with pod name
* Typing and cleaning
* Alex PR suggestions
* Removed unnecessary if
Alberto Soutullo
Benjamin Arntzen
Benjamin Arntzen