logos-storage/das-research
1
0
Fork 0
mirror of https://github.com/logos-storage/das-research.git synced 2026-01-03 13:43:11 +00:00
Code Issues Packages Projects Releases Wiki Activity
193 Commits 82 Branches 0 Tags 2.9 MiB
Commit Graph

135 Commits

Author SHA1 Message Date
Csaba Kiraly
0f4e59ce7f
use SparseBlock to store segments in nodes 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 23:36:30 +02:00
Csaba Kiraly
8122290f37
add SparseBlock with selected rows and columns only 
SparseBlock stores only selected rows and columns.
Initial implementation without checks and asserts.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 23:35:43 +02:00
Csaba Kiraly
c01b0c9e39
keep track of active send queues 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 23:28:49 +02:00
Csaba Kiraly
1cc99a1255
add send/receive abstaction 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 23:28:49 +02:00
Csaba Kiraly
5c1095cee7
use shuffled round robin between output queues 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

# Conflicts:
#	DAS/validator.py
 2023-03-27 23:28:49 +02:00
Csaba Kiraly
8b0650361c
optimize debug printing 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 23:28:49 +02:00
Csaba Kiraly
75a9b484e9
fixup: close plot to release memory 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 23:28:12 +02:00
Csaba Kiraly
04ad03f175
fixup: avoid warning on mean if empty 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 23:27:49 +02:00
Csaba Kiraly
dc51727b32
plot progress per run 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 23:16:05 +02:00
Csaba Kiraly
eb4f451303
save progress and traffic statistics to XML 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 23:15:44 +02:00
Csaba Kiraly
23af30e381
add generalized metrics collection 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 22:29:22 +02:00
Csaba Kiraly
7e4074938a
add duplicate statistics 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 22:29:22 +02:00
Csaba Kiraly
6616cc799f
move traffic stats calculation to observer 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 22:29:20 +02:00
Csaba Kiraly
119777787e
add progress meters to observer 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 22:28:30 +02:00
Csaba Kiraly
037c4cd67a
count number of validators having all rows/columns 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 22:28:30 +02:00
Csaba Kiraly
3795948564
fix requirements.txt 2023-03-27 11:37:23 +02:00
Csaba Kiraly
894009b414
Validator node: add nodeClass property 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 11:37:22 +02:00
Csaba Kiraly
ff93161b8d
set vpn in validator 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 11:37:22 +02:00
Csaba Kiraly
122f6a8348
remove resetShape 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 11:37:22 +02:00
Csaba Kiraly
6e4b37a3d2
adding log level TRACE 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 11:37:22 +02:00
Csaba Kiraly
af124c0755
simplify code 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 11:37:22 +02:00
Csaba Kiraly
952d191ccd
remove unused golden data 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 11:37:22 +02:00
Leonardo Bautista-Gomez
680817b97b Remove global randomness parameter for validator 2023-03-23 20:10:27 +01:00
Leonardo Bautista-Gomez
3642083f22 Fixing the global random uniform distribution of topics across validators 2023-03-21 15:16:19 +01:00
Leonardo Bautista-Gomez
125ba2ad4a Fix visualizer to loop over all parameters 2023-03-21 10:41:52 +01:00
Csaba Kiraly
64fcbff0ef
warn if no nodes for a row/column 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-20 19:36:42 +01:00
Csaba Kiraly
353efec141
fixup: use vpn1 and vpn2 correctly 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-16 20:38:58 +01:00
Csaba Kiraly
c366c05616
handle overlap for multiple validators per node correctly 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-16 20:38:57 +01:00
Csaba Kiraly
9d9612fd34
rename numberValidators to numberNodes 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-16 20:38:53 +01:00
Csaba Kiraly
065086f88c
configure using validatorsPerNode1/2 instead of chi1/2 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-16 20:00:52 +01:00
Csaba Kiraly
45fe454204
WIP: extend Vis with new parameters 
Current code assumes 6 parameters in some places.
Still needs some work

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-16 19:57:46 +01:00
Csaba Kiraly
ef4e32ed53
introduce node classes 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-16 19:57:43 +01:00
Csaba Kiraly
dc7a4d3c03
generate row/column interest locally in validator 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-16 19:54:16 +01:00
Csaba Kiraly
4a5d410f6a
fixup: add bwUplink to save file name 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-16 19:53:02 +01:00
Csaba Kiraly
99e051e6fc
adding uplinkBw configuration 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-16 19:50:48 +01:00
Leo
b39d75125e
Merge branch 'develop' into fix-deterministic 2023-03-15 13:37:02 +01:00
Leonardo Bautista-Gomez
0a92ef7071 Adding and logging random seed in the shape for each simulation 2023-03-15 13:18:02 +01:00
Leonardo Bautista-Gomez
833ae76097 add shape repr 2023-03-15 12:37:23 +01:00
leobago
ba94cc8da1 Update requirements 2023-03-14 08:26:37 +01:00
Csaba Kiraly
16b670e916
fix issues with logging in parallel execution 
For fixing logging issues see
https://stackoverflow.com/questions/58026381/logging-nested-functions-using-joblib-parallel-and-delayed-calls
and https://github.com/joblib/joblib/issues/1017

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-07 13:11:17 +01:00
Csaba Kiraly
0a5afd97de
remove old config files 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-07 12:10:36 +01:00
Leo
37ff89bd82
Merge pull request #17 from status-im/bandwidth 
Bandwidth limited diffusion
 2023-03-03 11:48:21 +01:00
Leonardo Bautista-Gomez
b4348b0005 Cosmetic changes for documentation 2023-03-03 11:47:27 +01:00
Csaba Kiraly
35d1790429
Merge remote-tracking branch 'origin/vis' into develop 
# Conflicts:
#	.gitignore
#	DAS/simulator.py
#	study.py
 2023-03-02 01:32:25 +01:00
Csaba Kiraly
daee84b9ea
add more function docustrings 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-01 23:59:35 +01:00
Csaba Kiraly
66a9d66dc6
moving helper functions to tools.py 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-01 22:34:16 +01:00
Csaba Kiraly
68fdaf3572
add method descriptions 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-01 22:21:31 +01:00
Csaba Kiraly
e611b5143c
refactor dumbRandomScheduler 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-01 11:11:39 +01:00
Csaba Kiraly
3095e440c6
factorize segmentShuffleScheduler code 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-01 11:11:39 +01:00
Csaba Kiraly
2bf85c41a2
factorize send code 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-01 09:54:22 +01:00