699a912991
Merge branch 'develop' into addDiagnostics
2023-03-30 13:56:09 +02:00
4b7bf81cee
Remove stop condition fix
2023-03-30 13:52:40 +02:00
98423d29c0
Typos and sets
2023-03-30 13:49:01 +02:00
1b4a225114
Merge pull request #35 from status-im/issue-29
...
Solving Issue 29 partially. Still remains the problems of stop condition.
2023-03-30 13:46:00 +02:00
7719f84a10
Switch from time steps to miliseconds ( #30 )
...
Switch from time steps to miliseconds
2023-03-30 13:41:50 +02:00
296c4fb762
Rows and columns to sets
2023-03-30 13:36:48 +02:00
89368d5fbe
fixed bug and generalized averaging
2023-03-30 11:24:30 +00:00
795bb1d10d
Move set to simulator for future diagnostic purposes
2023-03-30 13:15:42 +02:00
b5390b9f1b
Remove traffic statsi. Rebase to traffic progress.
2023-03-30 11:43:02 +02:00
41e8399159
Add Tx and Rx stats to resultsi. Rebase to traffic progress.
2023-03-30 11:40:50 +02:00
9800161ac9
Switch from time steps to miliseconds. Rebase to traffic progress.
2023-03-30 11:35:44 +02:00
98db10f7a6
Add more documentation
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-30 00:13:33 +02:00
e285890fa7
Fixes allocation bug, remove duplicates in rowIDs and columnIDs, add diagnostics when the block is not available. Add number of steps without progress to stop condition.
2023-03-29 16:42:09 +02:00
cb0a3ea1ba
fixup: avoid warning on mean if empty
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-29 16:36:40 +02:00
9f3089c232
rowIDs and columnIDs are sets
...
Fixes issue 29, where multiple instances of an ID in rowIDs
created a topology with nodes with a huge degree.
This huge degree then created lots of duplicates, eating up available
bandwidth.
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-29 15:54:44 +02:00
f85cdb401b
fix line allocation when evenLineDistribution=True
...
vector should have chi elements for each validator
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-29 15:49:52 +02:00
75a9b484e9
fixup: close plot to release memory
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-27 23:28:12 +02:00
04ad03f175
fixup: avoid warning on mean if empty
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-27 23:27:49 +02:00
dc51727b32
plot progress per run
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-27 23:16:05 +02:00
eb4f451303
save progress and traffic statistics to XML
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-27 23:15:44 +02:00
23af30e381
add generalized metrics collection
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-27 22:29:22 +02:00
7e4074938a
add duplicate statistics
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-27 22:29:22 +02:00
6616cc799f
move traffic stats calculation to observer
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-27 22:29:20 +02:00
119777787e
add progress meters to observer
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-27 22:28:30 +02:00
037c4cd67a
count number of validators having all rows/columns
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-27 22:28:30 +02:00
3795948564
fix requirements.txt
2023-03-27 11:37:23 +02:00
894009b414
Validator node: add nodeClass property
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-27 11:37:22 +02:00
ff93161b8d
set vpn in validator
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-27 11:37:22 +02:00
122f6a8348
remove resetShape
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-27 11:37:22 +02:00
6e4b37a3d2
adding log level TRACE
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-27 11:37:22 +02:00
af124c0755
simplify code
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-27 11:37:22 +02:00
952d191ccd
remove unused golden data
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-27 11:37:22 +02:00
aa99601826
Merge branch 'develop' into vis
2023-03-26 13:01:07 +00:00
680817b97b
Remove global randomness parameter for validator
2023-03-23 20:10:27 +01:00
57af48bf0e
Average runs
2023-03-22 23:17:19 +00:00
3642083f22
Fixing the global random uniform distribution of topics across validators
2023-03-21 15:16:19 +01:00
125ba2ad4a
Fix visualizer to loop over all parameters
2023-03-21 10:41:52 +01:00
64fcbff0ef
warn if no nodes for a row/column
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-20 19:36:42 +01:00
353efec141
fixup: use vpn1 and vpn2 correctly
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-16 20:38:58 +01:00
c366c05616
handle overlap for multiple validators per node correctly
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-16 20:38:57 +01:00
9d9612fd34
rename numberValidators to numberNodes
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-16 20:38:53 +01:00
065086f88c
configure using validatorsPerNode1/2 instead of chi1/2
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-16 20:00:52 +01:00
45fe454204
WIP: extend Vis with new parameters
...
Current code assumes 6 parameters in some places.
Still needs some work
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-16 19:57:46 +01:00
ef4e32ed53
introduce node classes
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-16 19:57:43 +01:00
dc7a4d3c03
generate row/column interest locally in validator
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-16 19:54:16 +01:00
4a5d410f6a
fixup: add bwUplink to save file name
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-16 19:53:02 +01:00
99e051e6fc
adding uplinkBw configuration
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-16 19:50:48 +01:00
b39d75125e
Merge branch 'develop' into fix-deterministic
2023-03-15 13:37:02 +01:00
0a92ef7071
Adding and logging random seed in the shape for each simulation
2023-03-15 13:18:02 +01:00
833ae76097
add shape repr
2023-03-15 12:37:23 +01:00
ba94cc8da1
Update requirements
2023-03-14 08:26:37 +01:00
16b670e916
fix issues with logging in parallel execution
...
For fixing logging issues see
https://stackoverflow.com/questions/58026381/logging-nested-functions-using-joblib-parallel-and-delayed-calls
and https://github.com/joblib/joblib/issues/1017
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-07 13:11:17 +01:00
0a5afd97de
remove old config files
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-07 12:10:36 +01:00
37ff89bd82
Merge pull request #17 from status-im/bandwidth
...
Bandwidth limited diffusion
2023-03-03 11:48:21 +01:00
b4348b0005
Cosmetic changes for documentation
2023-03-03 11:47:27 +01:00
35d1790429
Merge remote-tracking branch 'origin/vis' into develop
...
# Conflicts:
# .gitignore
# DAS/simulator.py
# study.py
2023-03-02 01:32:25 +01:00
daee84b9ea
add more function docustrings
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-01 23:59:35 +01:00
66a9d66dc6
moving helper functions to tools.py
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-01 22:34:16 +01:00
68fdaf3572
add method descriptions
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-01 22:21:31 +01:00
e611b5143c
refactor dumbRandomScheduler
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-01 11:11:39 +01:00
3095e440c6
factorize segmentShuffleScheduler code
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-01 11:11:39 +01:00
2bf85c41a2
factorize send code
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-01 09:54:22 +01:00
b5368b4e43
factorize restore
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-01 08:41:37 +01:00
82ee2b5189
simplify dumbRandomScheduler code
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-03-01 08:38:08 +01:00
39c454d3f4
minor plot fixes
2023-02-26 18:36:02 +01:00
ceb8357034
Merge branch 'vis' of https://github.com/status-im/das-research into vis
2023-02-26 18:27:47 +01:00
f95a393068
improve perNeighborSendQueue
...
- improve shuffling between rows and columns
- speed up code execution
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-24 12:10:45 +01:00
d9a2d5d606
fixup: ensure bw limit is respected
...
Lost meaning of return while factorizing schedulers. Fix it
by checking limits after each call.
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-24 12:04:07 +01:00
0a418b35b2
parametrize dumbRandomScheduler
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-24 10:43:45 +01:00
a03371cf4e
add logging of TX and RX statistics
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-24 10:43:45 +01:00
300bc19c67
factorize send code
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-24 08:55:34 +01:00
fa1818a43b
simplify code
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-24 08:43:18 +01:00
ead127e73e
change defaults to queue per p2p link
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-24 08:42:35 +01:00
89a6b1cdf7
remove old scheduler
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-24 00:35:17 +01:00
2707269836
fixup: moving simulator config to resetShape
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 22:21:16 +01:00
dfacd6bb18
allow push from non-neighbor
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:28:41 +01:00
b33f829b0e
proposer might push segments without participating in mesh
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:28:41 +01:00
186d430ad1
consider shuffleLines in segmentShuffleScheduler
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:28:40 +01:00
a1a8a4282d
fix scheduler to check result of endSegmentToNeigh
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:27:53 +01:00
c0650bf75a
implement partial line sending logic
...
On any given p2p link, it only makes sense to send up to
k messages, after that repair kicks in.
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:27:53 +01:00
e70740f530
handle duplicates in receiveRow/Column
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:27:53 +01:00
f91f3da5d2
fixup: segmentShuffleScheduler
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:27:53 +01:00
bb55abe2b0
comments only
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:27:52 +01:00
54d101e5b8
segmentShuffleScheduler: persist scheduler state between timesteps
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:27:52 +01:00
cd2b69149b
add segmentShuffleScheduler
...
This scheduler check which owned segments needs sending (at least
one neighbor needing it). Then it sends each segment that's worth sending
once, in shuffled order. This is repeated until bw limit.
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:27:52 +01:00
655f3a6642
fix: avoid looking into the future
...
Checking neigh.receiving is cheating in the current model.
If the timeslot is small, information can't propagate that fast.
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:27:52 +01:00
af72e58d08
collect receivedQueue only if it is used later
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:27:52 +01:00
1669ec9236
more debug logging
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:27:49 +01:00
5383c59f6f
add shuffleLines and shuffleNeighbors params
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:25:15 +01:00
d0641e4568
add repairOnTheFly parameter
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:25:09 +01:00
f05c3cd233
fix queuing: should queue after repair
...
If operation is based on send queues, segments should
be queued after successful repair.
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:21:56 +01:00
dff0e5523a
factorize addToSendQueue
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:15:45 +01:00
0c91eff67b
add dumbRandomScheduler parameter
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:15:45 +01:00
b7dab5bad9
fix sendSegmentToNeigh: specify dimension
...
Specify along which dimension (row/column) a segment was
sent.
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:15:39 +01:00
23e40693f1
add perNeighborQueue option
...
If enabled, queue incoming messages to outgoing connections on arrival,
as typical in some GossipSub implementations.
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:13:39 +01:00
7c0fcaba78
add validator.perNodeQueue conf option
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:13:39 +01:00
f67c70896c
add to receivedQueue also in row/column receive code
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:13:39 +01:00
9ab51278c8
add shuffledDict helper
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:13:39 +01:00
1403ca7ad0
add random scheduler
2023-02-23 21:13:39 +01:00
0f4883bf26
add node level send queue
...
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-02-23 21:13:35 +01:00
Leonardo Bautista-Gomez
Csaba Kiraly
Leo
HajarZaiz
Csaba Kiraly
HajarZaiz