logos-storage/das-research
1
0
Fork 0
mirror of https://github.com/logos-storage/das-research.git synced 2026-01-21 22:43:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
192 Commits 82 Branches 0 Tags 2.9 MiB
Commit Graph

134 Commits

Author SHA1 Message Date
Csaba Kiraly
8122290f37
add SparseBlock with selected rows and columns only 
SparseBlock stores only selected rows and columns.
Initial implementation without checks and asserts.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 23:35:43 +02:00
Csaba Kiraly
c01b0c9e39
keep track of active send queues 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 23:28:49 +02:00
Csaba Kiraly
1cc99a1255
add send/receive abstaction 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 23:28:49 +02:00
Csaba Kiraly
5c1095cee7
use shuffled round robin between output queues 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

# Conflicts:
#	DAS/validator.py
 2023-03-27 23:28:49 +02:00
Csaba Kiraly
8b0650361c
optimize debug printing 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 23:28:49 +02:00
Csaba Kiraly
75a9b484e9
fixup: close plot to release memory 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 23:28:12 +02:00
Csaba Kiraly
04ad03f175
fixup: avoid warning on mean if empty 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 23:27:49 +02:00
Csaba Kiraly
dc51727b32
plot progress per run 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 23:16:05 +02:00
Csaba Kiraly
eb4f451303
save progress and traffic statistics to XML 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 23:15:44 +02:00
Csaba Kiraly
23af30e381
add generalized metrics collection 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 22:29:22 +02:00
Csaba Kiraly
7e4074938a
add duplicate statistics 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 22:29:22 +02:00
Csaba Kiraly
6616cc799f
move traffic stats calculation to observer 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 22:29:20 +02:00
Csaba Kiraly
119777787e
add progress meters to observer 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 22:28:30 +02:00
Csaba Kiraly
037c4cd67a
count number of validators having all rows/columns 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 22:28:30 +02:00
Csaba Kiraly
3795948564
fix requirements.txt 2023-03-27 11:37:23 +02:00
Csaba Kiraly
894009b414
Validator node: add nodeClass property 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 11:37:22 +02:00
Csaba Kiraly
ff93161b8d
set vpn in validator 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 11:37:22 +02:00
Csaba Kiraly
122f6a8348
remove resetShape 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 11:37:22 +02:00
Csaba Kiraly
6e4b37a3d2
adding log level TRACE 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 11:37:22 +02:00
Csaba Kiraly
af124c0755
simplify code 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 11:37:22 +02:00
Csaba Kiraly
952d191ccd
remove unused golden data 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-27 11:37:22 +02:00
Leonardo Bautista-Gomez
680817b97b Remove global randomness parameter for validator 2023-03-23 20:10:27 +01:00
Leonardo Bautista-Gomez
3642083f22 Fixing the global random uniform distribution of topics across validators 2023-03-21 15:16:19 +01:00
Leonardo Bautista-Gomez
125ba2ad4a Fix visualizer to loop over all parameters 2023-03-21 10:41:52 +01:00
Csaba Kiraly
64fcbff0ef
warn if no nodes for a row/column 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-20 19:36:42 +01:00
Csaba Kiraly
353efec141
fixup: use vpn1 and vpn2 correctly 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-16 20:38:58 +01:00
Csaba Kiraly
c366c05616
handle overlap for multiple validators per node correctly 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-16 20:38:57 +01:00
Csaba Kiraly
9d9612fd34
rename numberValidators to numberNodes 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-16 20:38:53 +01:00
Csaba Kiraly
065086f88c
configure using validatorsPerNode1/2 instead of chi1/2 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-16 20:00:52 +01:00
Csaba Kiraly
45fe454204
WIP: extend Vis with new parameters 
Current code assumes 6 parameters in some places.
Still needs some work

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-16 19:57:46 +01:00
Csaba Kiraly
ef4e32ed53
introduce node classes 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-16 19:57:43 +01:00
Csaba Kiraly
dc7a4d3c03
generate row/column interest locally in validator 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-16 19:54:16 +01:00
Csaba Kiraly
4a5d410f6a
fixup: add bwUplink to save file name 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-16 19:53:02 +01:00
Csaba Kiraly
99e051e6fc
adding uplinkBw configuration 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-16 19:50:48 +01:00
Leo
b39d75125e
Merge branch 'develop' into fix-deterministic 2023-03-15 13:37:02 +01:00
Leonardo Bautista-Gomez
0a92ef7071 Adding and logging random seed in the shape for each simulation 2023-03-15 13:18:02 +01:00
Leonardo Bautista-Gomez
833ae76097 add shape repr 2023-03-15 12:37:23 +01:00
leobago
ba94cc8da1 Update requirements 2023-03-14 08:26:37 +01:00
Csaba Kiraly
16b670e916
fix issues with logging in parallel execution 
For fixing logging issues see
https://stackoverflow.com/questions/58026381/logging-nested-functions-using-joblib-parallel-and-delayed-calls
and https://github.com/joblib/joblib/issues/1017

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-07 13:11:17 +01:00
Csaba Kiraly
0a5afd97de
remove old config files 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-07 12:10:36 +01:00
Leo
37ff89bd82
Merge pull request #17 from status-im/bandwidth 
Bandwidth limited diffusion
 2023-03-03 11:48:21 +01:00
Leonardo Bautista-Gomez
b4348b0005 Cosmetic changes for documentation 2023-03-03 11:47:27 +01:00
Csaba Kiraly
35d1790429
Merge remote-tracking branch 'origin/vis' into develop 
# Conflicts:
#	.gitignore
#	DAS/simulator.py
#	study.py
 2023-03-02 01:32:25 +01:00
Csaba Kiraly
daee84b9ea
add more function docustrings 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-01 23:59:35 +01:00
Csaba Kiraly
66a9d66dc6
moving helper functions to tools.py 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-01 22:34:16 +01:00
Csaba Kiraly
68fdaf3572
add method descriptions 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-01 22:21:31 +01:00
Csaba Kiraly
e611b5143c
refactor dumbRandomScheduler 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-01 11:11:39 +01:00
Csaba Kiraly
3095e440c6
factorize segmentShuffleScheduler code 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-01 11:11:39 +01:00
Csaba Kiraly
2bf85c41a2
factorize send code 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-01 09:54:22 +01:00
Csaba Kiraly
b5368b4e43
factorize restore 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-01 08:41:37 +01:00