Commit Graph

4315 Commits

Author SHA1 Message Date
Agnish Ghosh 287bf72fbb simplify data column reconstruct and broadcast logic 2024-08-19 12:04:05 +05:30
Agnish Ghosh 9ed7a193cd disable rman for blobs 2024-08-19 02:58:50 +05:30
Agnish Ghosh 80b3e70673 slight fix 2024-08-19 01:30:34 +05:30
Agnish Ghosh 15a302cea2 fixes related to reconstruction 2024-08-19 01:23:37 +05:30
Agnish Ghosh 831089825c activate data column verification before storing backfill block 2024-08-18 23:07:33 +05:30
Agnish Ghosh 7e7c14989e bump up data column gossip wait time 2024-08-18 22:23:33 +05:30
Agnish Ghosh 8bb1145b43 populate dataColumnSidecarByRange with remote custody columns 2024-08-18 21:42:10 +05:30
Agnish Ghosh 032f544d65
revert constants until all cl agree 2024-08-18 14:17:33 +05:30
Agnish Ghosh 8eaa903c43
bump devnet constants 2024-08-18 13:41:32 +05:30
Agnish Ghosh 75e1bb616b
handling reconstruction condition internally 2024-08-18 13:35:16 +05:30
Agnish Ghosh e7610ece97
ignore peers with low csc 2024-08-18 13:10:21 +05:30
Agnish Ghosh f5354eb46d
bump parallel requests for data columns upto 64 for supernode 2024-08-18 12:17:25 +05:30
Agnish Ghosh 04fd58efbf
disable valid csc set experimentally 2024-08-18 04:11:39 +05:30
Agnish Ghosh 121a3687b8
disable blob quarantine checks 2024-08-18 02:10:51 +05:30
Agnish Ghosh 7c39d79e7e
remove metadata v3 call from handleStatus 2024-08-18 01:06:05 +05:30
Agnish Ghosh 7c33d3f0f4
prevent blob proposal in pd devnet 2024-08-17 23:42:31 +05:30
Agnish Ghosh cd5532c078 add filtering to peer pool and protocol, remove filtering from SyncManager level 2024-08-17 20:51:36 +05:30
Agnish Ghosh f3f61cbbff conditionally reconstruct and broadcast only when supernode 2024-08-16 01:03:53 +05:30
Agnish Ghosh cc21a2a630 fix: enr bitfield logic for custody subnet count 2024-08-14 16:29:28 +05:30
Agnish Ghosh 1ebba1fac4
add: metadata-v3 for custody subnet count (#6486)
* init: add metadatav3, save progress

* fix import issues

* fix spec version

* fix: metadata_v2 to return altair.MetaData

* update metata function backward compatible now
2024-08-12 21:27:37 +05:30
Agnish Ghosh 9be615dff9
add: data column reconstruction and broadcast (#6481)
* save commit, decouples reconstruction and broadcasting

* save progress

* add: reconstruction event loop, previous reconstruction related cleanups
2024-08-08 17:44:55 +05:30
Agnish Ghosh b32205de7c
upstream peerdas alpha3 related spec changes + fix upstream related issues (#6468)
* reworked some of the das core specs, pr'd to check whether whether the conflicting type issue is centric to my machine or not

* bumped nim-blscurve to 9c6e80c6109133c0af3025654f5a8820282cff05, same as unstable

* bumped nim-eth2-scenarios, nim-nat-traversal at par with unstable, added more pathches, made peerdas devnet branch backward compatible, peerdas passing new ssz tests as per alpha3, disabled electra fixture tests, as branch hasn't been rebased for a while

* refactor test fixture files

* rm: serializeDataColumn

* refactor: took data columns extracted from blobs during block proposal to the heap

* disable blob broadcast in pd devnet

* fix addBlock in message router

* fix: data column iterator

* added debug checkpoints to check CI

* refactor if else conditions

* add: updated das core specs to alpha 3, and unit tests pass
2024-08-05 19:27:39 +05:30
Agnish Ghosh 20e6b189e8
resovle merge conflicts 2024-07-29 19:02:52 +05:30
Agnish Ghosh 329fc21eab
add: condition for being able to selfReconstruct 2024-07-29 19:00:43 +05:30
Agnish Ghosh e80bd36f76 add: verify data column kzg proof during storeBlock, added serializeDataColumn for better logs 2024-07-26 14:19:44 +05:30
Agnish Ghosh 2e9750ba76 add: get_extended_sample_count for lossy sampler and it's unit test 2024-07-25 18:56:10 +05:30
Agnish Ghosh 874488843a add: hypergeom cdf 2024-07-25 17:58:29 +05:30
Agnish Ghosh 7faec9b7a3 nits 2024-07-23 16:42:36 +05:30
Agnish Ghosh c6513122b9 fix reviewed code 2024-07-23 12:38:16 +05:30
Agnish Ghosh b001499281 add: valid custody peer set to RequestManager 2024-07-23 12:36:42 +05:30
Agnish Ghosh d2c7208492 Eth2Node not needed in sync man 2024-07-22 19:07:51 +05:30
Agnish Ghosh 5265eeb6ea refactor: sync manager to range request only valid peers if not supernode 2024-07-22 19:04:04 +05:30
Agnish Ghosh fa5b15436c
add: logic constructing valid set of peers 2024-07-19 13:58:26 +05:30
Agnish Ghosh e034f30b5e
add: subscribeAllSubnets feature 2024-07-15 13:50:41 +05:30
Agnish Ghosh b33900bd35
added enr struct 2024-07-15 13:11:09 +05:30
Agnish Ghosh cf40d7faae
cleanup for debugs, complete 2024-07-08 11:39:50 +05:30
Agnish Ghosh 5eb854bfcd
checkpoint 2 cleanup 2024-07-07 19:48:56 +05:30
Agnish Ghosh aaba448bc9
convert to List add 2024-07-07 15:59:27 +05:30
Agnish Ghosh 0bffdd0dca
change checkpoints 2024-07-07 05:02:58 +05:30
Agnish Ghosh 7a891f13d6
rework on cell and proof 2024-07-07 04:35:06 +05:30
Agnish Ghosh c14b59264c
add: checkpoints for debug support 2024-07-07 03:01:32 +05:30
Agnish Ghosh e5237d19a3
cell and proof extraction 2024-07-07 02:06:13 +05:30
Agnish Ghosh abf58928cf
handle empty blobs 2024-07-07 00:43:24 +05:30
Agnish Ghosh 755c24d6aa
fix: blob len 0 case 2024-07-06 21:54:29 +05:30
Agnish Ghosh 7b9c68b19d
oops 2024-07-06 13:55:33 +05:30
Agnish Ghosh 74ee8bb74d
refactor cells and proofs logic + fix edge cases 2024-07-06 13:24:51 +05:30
Agnish Ghosh 85db9ca99c
regressive fix 2024-07-06 00:33:15 +05:30
Agnish Ghosh 1729bdcf57
reduce parallel requests 2024-07-06 00:30:18 +05:30
Agnish Ghosh adc717c3d0
change return type for get dc 2024-07-06 00:28:55 +05:30
Agnish Ghosh 7101f93ff4
gindex issue fix 2024-07-05 23:03:37 +05:30
Agnish Ghosh a92eda56d0
prevent pulling const values from deneb preset 2024-07-05 17:16:20 +05:30
Agnish Ghosh 778ea9f094
fix build proof in get dc 2024-07-05 16:15:38 +05:30
Agnish Ghosh 086d3f1fdc
fix: get dc sidecar 2024-07-05 15:56:03 +05:30
Agnish Ghosh fab427d65f
enable dc in gossip and message router 2024-07-05 14:34:51 +05:30
Agnish Ghosh 318d656942
fix: gindex 2024-07-05 14:26:42 +05:30
Agnish Ghosh 14afc82a1d
gindex fix 2024-07-05 00:05:37 +05:30
Agnish Ghosh eb46f4c486
fix kzg inclusion proof logic 2024-07-04 23:51:38 +05:30
Agnish Ghosh 6cdc6bfa32
debug: verify data column kzg proofs via kurtosis 2024-07-04 20:24:11 +05:30
Agnish Ghosh 93c3525955
increase ops cost 2024-07-04 19:23:24 +05:30
Agnish Ghosh 24b30a9cec
test kurtosis 2024-07-04 19:20:53 +05:30
Agnish Ghosh fe183e7465
change timings 2024-07-04 19:14:16 +05:30
Agnish Ghosh 41b35b9799
fix: get data column fixes 2024-07-04 16:49:55 +05:30
Agnish Ghosh 13029d952e
fix: get data column 2024-07-04 14:45:23 +05:30
Agnish Ghosh b927ddd65a
fix: get_data_column_sidecars 2024-07-04 13:55:51 +05:30
Agnish Ghosh 3bea5742f2
reduce data column response cost 2024-07-04 13:03:48 +05:30
Agnish Ghosh a755dbaaa9
inclusion proof depth 2024-07-04 11:51:54 +05:30
Agnish Ghosh c6662bd914
reenable blobs in block proposal 2024-07-04 11:46:40 +05:30
Agnish Ghosh 1a857603b2
fix: cell and proof aggregator 2024-07-04 00:08:11 +05:30
Agnish Ghosh d49b1a1d47
fix: ckzg function change 2024-07-03 23:51:41 +05:30
Agnish Ghosh 706373909f
fix: get_data_column_sidecars 2024-07-03 23:45:20 +05:30
Agnish Ghosh 887a44a82f
revert gossip val 2024-07-03 21:59:54 +05:30
Agnish Ghosh 53f7175cd8
minor fix 2024-07-03 21:54:40 +05:30
Agnish Ghosh 9f42196c75
exp disable of some gossip conditions 2024-07-03 21:51:59 +05:30
Agnish Ghosh 152d276d78
added reconstruction logic 2024-07-03 21:50:52 +05:30
Agnish Ghosh d292e94560
sync queue 2024-07-02 22:37:09 +05:30
Agnish Ghosh ad64b22485
shortLog for dc 2024-07-02 22:26:31 +05:30
Agnish Ghosh 8f9f6544d9
intentionally increase custody requirement 2024-07-02 21:41:46 +05:30
Agnish Ghosh 0e710da37b
update timings 2024-07-02 21:27:50 +05:30
Agnish Ghosh d0722cd465
update constants 2024-07-02 20:33:09 +05:30
Agnish Ghosh d8e1beff45
debug 4 2024-07-02 19:56:41 +05:30
Agnish Ghosh 8d2c4893d8
debug3 2024-07-02 19:28:23 +05:30
Agnish Ghosh 77cc2ef358
debug2 2024-07-02 18:35:58 +05:30
Agnish Ghosh 75c3e0b6d3
debug 2024-07-02 18:05:39 +05:30
Agnish Ghosh 2f7a3d0ff2
reenable checkpoints to debug exception 2024-07-02 17:15:08 +05:30
Agnish Ghosh 67fe8aca0b
disable blob activity (exp), improve gossip validation 2024-07-02 14:36:44 +05:30
Agnish Ghosh 9e6cad4105
bit more disabling for kurtosis 2024-07-02 04:51:58 +05:30
Agnish Ghosh a8e2c3e9a2
exp: disable some gossip conditions 2024-07-02 04:45:23 +05:30
Agnish Ghosh 0e02eb4ce7
fix test_sync_manager 2024-07-02 04:01:37 +05:30
Agnish Ghosh 9325423a6b
strangely disable this line makes it go away :) 2024-07-02 03:15:25 +05:30
Agnish Ghosh 3b1f5b48dd
weird fix 2024-07-02 03:09:49 +05:30
Agnish Ghosh 8e28654a24
exp: build failing, checking if failing on other machines with these changes 2024-07-02 00:14:58 +05:30
Agnish Ghosh 8ac4cc9152
add: data column grouping conditions for range request 2024-07-01 17:42:29 +05:30
Agnish Ghosh 26ac58716b
request man for data columns 2024-07-01 15:00:18 +05:30
Agnish Ghosh 510d988933
disable subnet gossip condition, fixed inclusion proof 2024-06-30 12:16:17 +05:30
Agnish Ghosh c8d957a495
add: experimental checkpoints on gossip validation to localize failing conditions 2024-06-29 14:26:52 +05:30
Agnish Ghosh 742669052a
dc quarantine activation, keeping blobs compatible 2024-06-29 14:07:09 +05:30
Agnish Ghosh ca3bd3eeee
experimental: disable scoring for data columns 2024-06-28 18:04:46 +05:30
Agnish Ghosh 0e01d2fb15
experimental disable for inclusion proofs 2024-06-28 18:01:14 +05:30
Agnish Ghosh 27b0705e0d
verify kzg disable 2024-06-28 15:29:57 +05:30
Agnish Ghosh 5bf1e021a7
initiate data column quarantine 2024-06-28 14:53:08 +05:30