codex-storage
/
das-research
mirror of https://github.com/codex-storage/das-research.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
346 Commits 79 Branches 0 Tags 2.6 MiB
Commit Graph

346 Commits

Author SHA1 Message Date
Leonardo Bautista-Gomez 833ae76097 add shape repr 2023-03-15 12:37:23 +01:00
Leo cb2625a284
Merge pull request #24 from status-im/config-py 
Multiproc
 2023-03-15 11:57:30 +01:00
Leo 8c9106d03d
Merge pull request #21 from status-im/multiproc 
Parallel execution of simulations with parameters to select number of threads
 2023-03-14 16:49:04 +01:00
Leonardo Bautista-Gomez 45bdcc3308 Seed by microseconds. Determinisme checked. 2023-03-14 11:25:17 +01:00
Leonardo Bautista-Gomez 377072ef79 Fix deterministic feature 2023-03-14 11:07:59 +01:00
leobago ba94cc8da1 Update requirements 2023-03-14 08:26:37 +01:00
Csaba Kiraly 567d13e370
add numJobs parameter to config 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-13 14:22:14 +01:00
Leo fa2ebccee5
Merge pull request #20 from status-im/config-py 
configure with code
 2023-03-07 16:03:20 +01:00
Csaba Kiraly 16b670e916
fix issues with logging in parallel execution 
For fixing logging issues see
https://stackoverflow.com/questions/58026381/logging-nested-functions-using-joblib-parallel-and-delayed-calls
and https://github.com/joblib/joblib/issues/1017

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-07 13:11:17 +01:00
Csaba Kiraly f5ffb0a07b
use joblib to run in parallel 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-07 13:11:13 +01:00
Csaba Kiraly 0b6cfad967
factorize study code 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-07 13:10:33 +01:00
Csaba Kiraly 0a5afd97de
remove old config files 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-07 12:10:36 +01:00
Csaba Kiraly 84e5482a6d
improve config filename error handling 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-07 12:07:31 +01:00
Csaba Kiraly 4a4f02427c
change config language to simple (or complex) code 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-07 12:06:57 +01:00
Leo 37ff89bd82
Merge pull request #17 from status-im/bandwidth 
Bandwidth limited diffusion
 2023-03-03 11:48:21 +01:00
Leonardo Bautista-Gomez b4348b0005 Cosmetic changes for documentation 2023-03-03 11:47:27 +01:00
Csaba Kiraly 35d1790429
Merge remote-tracking branch 'origin/vis' into develop 
# Conflicts:
#	.gitignore
#	DAS/simulator.py
#	study.py
 2023-03-02 01:32:25 +01:00
Csaba Kiraly daee84b9ea
add more function docustrings 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-01 23:59:35 +01:00
Csaba Kiraly 66a9d66dc6
moving helper functions to tools.py 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-01 22:34:16 +01:00
Csaba Kiraly 68fdaf3572
add method descriptions 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-01 22:21:31 +01:00
Csaba Kiraly e611b5143c
refactor dumbRandomScheduler 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-01 11:11:39 +01:00
Csaba Kiraly 3095e440c6
factorize segmentShuffleScheduler code 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-01 11:11:39 +01:00
Csaba Kiraly 2bf85c41a2
factorize send code 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-01 09:54:22 +01:00
Csaba Kiraly b5368b4e43
factorize restore 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-01 08:41:37 +01:00
Csaba Kiraly 82ee2b5189
simplify dumbRandomScheduler code 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-03-01 08:38:08 +01:00
HajarZaiz 57f3509102 Removed Frontend directory from Git index 2023-02-26 18:38:20 +01:00
HajarZaiz 39c454d3f4 minor plot fixes 2023-02-26 18:36:02 +01:00
HajarZaiz ceb8357034 Merge branch 'vis' of https://github.com/status-im/das-research into vis 2023-02-26 18:27:47 +01:00
Csaba Kiraly f95a393068
improve perNeighborSendQueue 
- improve shuffling between rows and columns
- speed up code execution

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-02-24 12:10:45 +01:00
Csaba Kiraly d9a2d5d606
fixup: ensure bw limit is respected 
Lost meaning of return while factorizing schedulers. Fix it
by checking limits after each call.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-02-24 12:04:07 +01:00
Csaba Kiraly 0a418b35b2
parametrize dumbRandomScheduler 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-02-24 10:43:45 +01:00
Csaba Kiraly a03371cf4e
add logging of TX and RX statistics 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-02-24 10:43:45 +01:00
Csaba Kiraly 300bc19c67
factorize send code 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-02-24 08:55:34 +01:00
Csaba Kiraly fa1818a43b
simplify code 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-02-24 08:43:18 +01:00
Csaba Kiraly ead127e73e
change defaults to queue per p2p link 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-02-24 08:42:35 +01:00
Csaba Kiraly 89a6b1cdf7
remove old scheduler 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-02-24 00:35:17 +01:00
Csaba Kiraly 2707269836
fixup: moving simulator config to resetShape 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-02-23 22:21:16 +01:00
Csaba Kiraly dfacd6bb18
allow push from non-neighbor 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-02-23 21:28:41 +01:00
Csaba Kiraly b33f829b0e
proposer might push segments without participating in mesh 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-02-23 21:28:41 +01:00
Csaba Kiraly 186d430ad1
consider shuffleLines in segmentShuffleScheduler 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-02-23 21:28:40 +01:00
Csaba Kiraly a1a8a4282d
fix scheduler to check result of endSegmentToNeigh 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-02-23 21:27:53 +01:00
Csaba Kiraly c0650bf75a
implement partial line sending logic 
On any given p2p link, it only makes sense to send up to
k messages, after that repair kicks in.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-02-23 21:27:53 +01:00
Csaba Kiraly e70740f530
handle duplicates in receiveRow/Column 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-02-23 21:27:53 +01:00
Csaba Kiraly f91f3da5d2
fixup: segmentShuffleScheduler 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-02-23 21:27:53 +01:00
Csaba Kiraly bb55abe2b0
comments only 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-02-23 21:27:52 +01:00
Csaba Kiraly 54d101e5b8
segmentShuffleScheduler: persist scheduler state between timesteps 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-02-23 21:27:52 +01:00
Csaba Kiraly cd2b69149b
add segmentShuffleScheduler 
This scheduler check which owned segments needs sending (at least
one neighbor needing it). Then it sends each segment that's worth sending
once, in shuffled order. This is repeated until bw limit.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-02-23 21:27:52 +01:00
Csaba Kiraly 655f3a6642
fix: avoid looking into the future 
Checking neigh.receiving is cheating in the current model.
If the timeslot is small, information can't propagate that fast.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-02-23 21:27:52 +01:00
Csaba Kiraly af72e58d08
collect receivedQueue only if it is used later 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-02-23 21:27:52 +01:00
Csaba Kiraly 1669ec9236
more debug logging 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2023-02-23 21:27:49 +01:00