Commit Graph

4488 Commits

Author SHA1 Message Date
Agnish Ghosh 1d4924190d refine rman dc 2024-10-03 04:31:40 +05:30
Agnish Ghosh baa959f978 rman fixes 2024-10-03 04:21:52 +05:30
Agnish Ghosh 9fb3d47d7c stop root requests 2024-10-03 04:01:24 +05:30
Agnish Ghosh 1ed21231fe parallel req bump down 2024-10-03 03:34:46 +05:30
Agnish Ghosh 9a4a6066c8 set poll interval 2024-10-03 03:07:22 +05:30
Agnish Ghosh 6d837bcdaa fix more quarantine issues 2024-10-03 02:33:25 +05:30
Agnish Ghosh 52e8249e82 fix quarantine 2024-10-03 01:39:18 +05:30
Agnish Ghosh 2fb732a9f7 refactor change 2024-10-03 00:54:25 +05:30
Agnish Ghosh b239ce9808 fix verification logic 2024-10-03 00:40:49 +05:30
Agnish Ghosh 9954da73d5 fix gossip val 2024-10-03 00:05:25 +05:30
Agnish Ghosh 5bcc599b66 fix 2 2024-10-02 23:22:40 +05:30
Agnish Ghosh c0c04a9528 quite some changes 2024-10-02 22:36:18 +05:30
Agnish Ghosh a789bdfaa6 storeBackfill fix 2024-10-02 22:22:13 +05:30
Agnish Ghosh 878cfc5d5c other fixes 2024-10-02 21:44:40 +05:30
Agnish Ghosh e7f66a6ace add to deneb view 2024-10-02 19:56:16 +05:30
Agnish Ghosh 190a1dd1c4 logging 2024-10-02 19:54:17 +05:30
Agnish Ghosh bc780a003f fix logic 2024-10-02 19:03:55 +05:30
Agnish Ghosh 4f7eedb72b more logging 2024-10-02 18:21:05 +05:30
Agnish Ghosh 6af7918b78 fix 2024-10-02 17:49:58 +05:30
Agnish Ghosh b9d8699c52 added more logging 2024-10-02 17:24:16 +05:30
Agnish Ghosh d71edad9c0 fixes around message router 2024-10-02 16:53:32 +05:30
Agnish Ghosh 3a6593647c verification before persistence 2024-10-02 16:12:48 +05:30
Agnish Ghosh 83dcd82ce1 fix 2024-10-02 15:52:59 +05:30
Agnish Ghosh 183fbd9f3e bump up response cost 2024-10-02 15:50:56 +05:30
Agnish Ghosh 0386f9459f dc response cost 2024-10-02 15:12:15 +05:30
Agnish Ghosh 75eda5341f adding more logging around db 2024-10-02 13:41:54 +05:30
Agnish Ghosh 5b467d060c kzg proof before inclusion proof 2024-10-02 11:41:08 +05:30
Agnish Ghosh beb539faa9 remove break condition 2024-10-02 02:03:56 +05:30
Agnish Ghosh e540c67fd1 enable warning for inconsistent data columns 2024-10-02 01:40:35 +05:30
Agnish Ghosh 2ecdb078b9 fixes 2024-10-02 00:18:00 +05:30
Agnish Ghosh 0b9c48cadb response cost 2024-10-01 23:02:17 +05:30
Agnish Ghosh 668a92c8ed increase time 2024-10-01 22:00:08 +05:30
Agnish Ghosh eeb5319bc6 enable peer quota 2024-10-01 21:13:09 +05:30
Agnish Ghosh 8c2fdecc4e add column support fully to db 2024-10-01 21:06:51 +05:30
Agnish Ghosh 1ba8948466 fix for put operations 2024-10-01 20:57:15 +05:30
Agnish Ghosh b6e89eb7e3 change db key 2024-10-01 20:51:17 +05:30
Agnish Ghosh 4bd6ead294 disable peer quota 2024-10-01 20:43:30 +05:30
Agnish Ghosh 1b5e9cf891 couple of more fixes 2024-10-01 18:40:50 +05:30
Agnish Ghosh b8d03ef913
refactor peer selection 2024-09-30 22:17:30 +05:30
Agnish Ghosh 9cab15e2cf
enable better logging for dc by range 2024-09-30 20:19:05 +05:30
Agnish Ghosh bef7d07105
fix range request issue 2024-09-30 15:48:27 +05:30
Agnish Ghosh 30f8ef9154 bump parallel dc requests 2024-09-23 15:22:08 +05:30
Agnish Ghosh eaf4b261c1 fix enr update issue 2024-09-23 14:09:16 +05:30
Agnish Ghosh 2ff165d304 fix enr advertising 2024-09-23 14:06:33 +05:30
Agnish Ghosh d08044f584 rework filtering for root request 2024-09-23 00:03:37 +05:30
Agnish Ghosh 916ac0eb58 revamp condition 2024-09-22 03:01:18 +05:30
Agnish Ghosh 6db298bca0 fix helpers 2024-09-21 17:32:07 +05:30
Agnish Ghosh 727bfc51b2 fix indent 2024-09-21 17:27:34 +05:30
Agnish Ghosh 32ca2d2645 fix subscription issue 2024-09-21 15:58:23 +05:30
Agnish Ghosh 4d0c66657f rework some of the reconstruction publishing logic 2024-09-21 15:47:58 +05:30
Agnish Ghosh c1cb673d6c fix bug 2024-09-19 17:22:08 +05:30
Agnish Ghosh 2608c19a13 change lookupcsc logic 2024-09-19 15:02:41 +05:30
Agnish Ghosh e8b64d8dd2 subnet sampling part 2 2024-09-19 13:36:56 +05:30
Agnish Ghosh 53af511488 add debug for lookupCscFromPeer 2024-09-18 16:34:53 +05:30
Agnish Ghosh 9b79a9df5b gossip wait time 2024-09-18 14:51:48 +05:30
Agnish Ghosh 2fff2ca38f enable subnet sampling 2024-09-18 13:30:14 +05:30
Agnish Ghosh 76b53483ff
make debug more crispp 2024-09-12 19:18:13 +05:30
Agnish Ghosh 880d253dee fix 2024-09-12 16:19:55 +05:30
Agnish Ghosh ee33ec0683 rework reconstruction publishing strategy 2024-09-12 11:59:59 +05:30
Agnish Ghosh dc4a1df2d7
fix SSZ issues 2024-09-11 10:41:19 +05:30
Agnish Ghosh c09d801fb2 all fixes related to enr 2024-09-09 22:09:10 +05:30
Agnish Ghosh d461d3f25f revert Csc to big endian uint64 2024-09-09 02:27:59 +05:30
Agnish Ghosh 4a006b5584 fix every place to make new custody column function work 2024-09-09 02:09:24 +05:30
Agnish Ghosh 7d04142e4a simplify usage of get custody columns 2024-09-09 01:58:21 +05:30
Agnish Ghosh cf0cf815b6 rework on subnet calculation 2024-09-03 01:03:15 +05:30
Agnish Ghosh 8c7d18ec51 rework reconstruction conditions 2024-09-02 17:23:31 +05:30
Agnish Ghosh 3aa85100f9 move lookupCscFromPeer to Eth2Network, add reqColumns check in data column sidecar range request 2024-09-02 16:45:10 +05:30
Agnish Ghosh e81ce6cf20 reconstruct and broadcast before validator duties 2024-09-02 13:40:15 +05:30
Agnish Ghosh 077b59249d enable logging for root and range requests on successful responses 2024-09-02 13:37:07 +05:30
Agnish Ghosh 84da290669 apply column subnet filtering only in Deneb message handlers 2024-08-28 20:38:19 +05:30
Agnish Ghosh 6796b19883 subscribe to all subnets in deneb message handlers 2024-08-28 19:49:12 +05:30
Agnish Ghosh 07d1a26e59 include data columns in addBlock that care about custody 2024-08-28 18:53:28 +05:30
Agnish Ghosh 66cb3ff54e only persist data columns after block validity 2024-08-28 18:03:21 +05:30
Agnish Ghosh b573caccc7 no verify kzg during addBlock 2024-08-28 14:29:19 +05:30
Agnish Ghosh 66bf20e7f0 logging of subscription of topics 2024-08-28 13:42:32 +05:30
Agnish Ghosh 547abaacf7 have getLowSubnets accept only BitArrays 2024-08-27 22:17:50 +05:30
Agnish Ghosh c42d54144a stick to newSeqOfCap 2024-08-27 14:33:37 +05:30
Agnish Ghosh b0710806da added more logging 2024-08-27 14:28:38 +05:30
Agnish Ghosh cafbeef543 more logging for clarity 2024-08-27 13:01:29 +05:30
Agnish Ghosh ef6901f0a9 enable more logging 2024-08-27 12:13:12 +05:30
Agnish Ghosh a26258a6db echo out column, kzg proof and commitment length to debug 2024-08-27 11:59:00 +05:30
Agnish Ghosh b9a3aa8277 logging in block processor 2024-08-27 02:51:58 +05:30
Agnish Ghosh fa5f47c468 bug fix 2024-08-27 01:43:35 +05:30
Agnish Ghosh 93b043f88a changed data column extraction logic during block proposals 2024-08-27 01:06:54 +05:30
Agnish Ghosh e9c7a758a0 change get data column sidecar logic in proposer to alpha 4 2024-08-27 00:08:05 +05:30
Agnish Ghosh 4730a4b246 enable blob length clarity for more clarity over the bug 2024-08-26 21:47:26 +05:30
Agnish Ghosh 6c6f33a925 investigate potential disparity between das workers length and extracted data columns length 2024-08-26 21:08:39 +05:30
Agnish Ghosh 9048b2c3f6 revert make data column ref 2024-08-26 14:02:33 +05:30
Agnish Ghosh 23355dc049 bug fix in enqueueBlock for data column verification 2024-08-26 13:59:46 +05:30
Agnish Ghosh d860373044 enable logging columns in enqueueBlock 2024-08-26 13:57:27 +05:30
Agnish Ghosh 993067c841 enable column logging in storeBlock 2024-08-26 13:55:06 +05:30
Agnish Ghosh 879a04d66a prevent exact data column logging 2024-08-26 13:02:19 +05:30
Agnish Ghosh 3e0992e0be enabled logging of subnet ids, and data column indices to prevent mismatch 2024-08-26 12:55:17 +05:30
Agnish Ghosh 0f28477295
added debug to subnet ids 2024-08-26 12:19:13 +05:30
Agnish Ghosh 7d5139c44c
trailing fixes 2024-08-24 20:22:14 +05:30
Agnish Ghosh 6a2ba50c19
enable logging of columns, conditionally subscribe to only topics that i custody for 2024-08-24 19:56:52 +05:30
Agnish Ghosh da735abd35
patched in pr #6505, for devnet readiness 2024-08-24 18:50:19 +05:30
Agnish Ghosh 5c3b360b2d
add condition for when >= deneb 2024-08-24 18:21:00 +05:30
Agnish Ghosh e9586fe879
enable data column kzg proof verification in storeBlock 2024-08-24 17:23:10 +05:30
Agnish Ghosh 237dc3adfb
subscribe to only those gossips that return via get_custody_column_subnet 2024-08-24 16:41:38 +05:30
Agnish Ghosh c2962b65c9
turned CscCount to uint8 according to the latest EF spec 2024-08-24 16:23:27 +05:30
Agnish Ghosh e87871e2cb save work 2024-08-23 15:56:36 +05:30
Agnish Ghosh ee9889ca91 disable blobs for syncing because of fake deneb fork 2024-08-23 12:31:54 +05:30
Agnish Ghosh de7cbd8ca4 reduce gossip wait time for data columns 2024-08-23 11:46:06 +05:30
Agnish Ghosh bf60839e67 increase waiting time for data column root request 2024-08-23 02:59:45 +05:30
Agnish Ghosh b4ba4bd7d1 lookup csc fix 2024-08-22 20:09:43 +05:30
Agnish Ghosh 6127bd6ed2 make all csc uint64 2024-08-22 18:34:26 +05:30
Agnish Ghosh fd1ae50830 revert to initially subscribing to all subnets 2024-08-22 16:04:15 +05:30
Agnish Ghosh 46c72d8cbe fix: initial subscription condition 2024-08-22 15:23:37 +05:30
Agnish Ghosh 2805ffe242 update constants 2024-08-22 14:56:44 +05:30
Agnish Ghosh 3f56281351 fixed lookup csc from peer 2024-08-22 13:52:41 +05:30
Agnish Ghosh e163aa08c0 enable filtering of peers during data column root request 2024-08-21 18:23:05 +05:30
Agnish Ghosh 5b8abc5f1b slight fix 2024-08-20 12:53:23 +05:30
Agnish Ghosh d9e019e2e2 sidecar verification disable at block processing 2024-08-20 12:52:29 +05:30
Agnish Ghosh fcdd7ee492 enable data column verification in block processor, other general fixes 2024-08-20 12:19:54 +05:30
Agnish Ghosh 70029545ea added support for customisable custody subnet count on boot, from bn startup flags 2024-08-20 03:58:27 +05:30
Agnish Ghosh f142dcd36f make reconstruct and broadcast right after performing validator duties 2024-08-19 21:33:40 +05:30
Agnish Ghosh 0034fea1d6 reduce outgoing root requests 2024-08-19 20:42:58 +05:30
Agnish Ghosh 2444b3190e init flag for csc, reduce debug messages 2024-08-19 20:08:01 +05:30
Agnish Ghosh e5da0681b3 get data column sidecar cell and proof variant 2024-08-19 19:24:59 +05:30
Agnish Ghosh 61033ed99b
fix: recover cells and proofs 2024-08-19 18:50:20 +05:30
Agnish Ghosh f71354b307 experimentally disable verifying data column sidecar before storing block 2024-08-19 13:52:47 +05:30
Agnish Ghosh 635eebbed1 fix response time 2024-08-19 13:17:33 +05:30
Agnish Ghosh 287bf72fbb simplify data column reconstruct and broadcast logic 2024-08-19 12:04:05 +05:30
Agnish Ghosh 9ed7a193cd disable rman for blobs 2024-08-19 02:58:50 +05:30
Agnish Ghosh 80b3e70673 slight fix 2024-08-19 01:30:34 +05:30
Agnish Ghosh 15a302cea2 fixes related to reconstruction 2024-08-19 01:23:37 +05:30
Agnish Ghosh 831089825c activate data column verification before storing backfill block 2024-08-18 23:07:33 +05:30
Agnish Ghosh 7e7c14989e bump up data column gossip wait time 2024-08-18 22:23:33 +05:30
Agnish Ghosh 8bb1145b43 populate dataColumnSidecarByRange with remote custody columns 2024-08-18 21:42:10 +05:30
Agnish Ghosh 032f544d65
revert constants until all cl agree 2024-08-18 14:17:33 +05:30
Agnish Ghosh 8eaa903c43
bump devnet constants 2024-08-18 13:41:32 +05:30
Agnish Ghosh 75e1bb616b
handling reconstruction condition internally 2024-08-18 13:35:16 +05:30
Agnish Ghosh e7610ece97
ignore peers with low csc 2024-08-18 13:10:21 +05:30
Agnish Ghosh f5354eb46d
bump parallel requests for data columns upto 64 for supernode 2024-08-18 12:17:25 +05:30
Agnish Ghosh 04fd58efbf
disable valid csc set experimentally 2024-08-18 04:11:39 +05:30
Agnish Ghosh 121a3687b8
disable blob quarantine checks 2024-08-18 02:10:51 +05:30
Agnish Ghosh 7c39d79e7e
remove metadata v3 call from handleStatus 2024-08-18 01:06:05 +05:30
Agnish Ghosh 7c33d3f0f4
prevent blob proposal in pd devnet 2024-08-17 23:42:31 +05:30
Agnish Ghosh cd5532c078 add filtering to peer pool and protocol, remove filtering from SyncManager level 2024-08-17 20:51:36 +05:30
Agnish Ghosh f3f61cbbff conditionally reconstruct and broadcast only when supernode 2024-08-16 01:03:53 +05:30
Agnish Ghosh cc21a2a630 fix: enr bitfield logic for custody subnet count 2024-08-14 16:29:28 +05:30
Agnish Ghosh 1ebba1fac4
add: metadata-v3 for custody subnet count (#6486)
* init: add metadatav3, save progress

* fix import issues

* fix spec version

* fix: metadata_v2 to return altair.MetaData

* update metata function backward compatible now
2024-08-12 21:27:37 +05:30
Agnish Ghosh 9be615dff9
add: data column reconstruction and broadcast (#6481)
* save commit, decouples reconstruction and broadcasting

* save progress

* add: reconstruction event loop, previous reconstruction related cleanups
2024-08-08 17:44:55 +05:30
Agnish Ghosh b32205de7c
upstream peerdas alpha3 related spec changes + fix upstream related issues (#6468)
* reworked some of the das core specs, pr'd to check whether whether the conflicting type issue is centric to my machine or not

* bumped nim-blscurve to 9c6e80c6109133c0af3025654f5a8820282cff05, same as unstable

* bumped nim-eth2-scenarios, nim-nat-traversal at par with unstable, added more pathches, made peerdas devnet branch backward compatible, peerdas passing new ssz tests as per alpha3, disabled electra fixture tests, as branch hasn't been rebased for a while

* refactor test fixture files

* rm: serializeDataColumn

* refactor: took data columns extracted from blobs during block proposal to the heap

* disable blob broadcast in pd devnet

* fix addBlock in message router

* fix: data column iterator

* added debug checkpoints to check CI

* refactor if else conditions

* add: updated das core specs to alpha 3, and unit tests pass
2024-08-05 19:27:39 +05:30
Agnish Ghosh 20e6b189e8
resovle merge conflicts 2024-07-29 19:02:52 +05:30
Agnish Ghosh 329fc21eab
add: condition for being able to selfReconstruct 2024-07-29 19:00:43 +05:30
Agnish Ghosh e80bd36f76 add: verify data column kzg proof during storeBlock, added serializeDataColumn for better logs 2024-07-26 14:19:44 +05:30
Agnish Ghosh 2e9750ba76 add: get_extended_sample_count for lossy sampler and it's unit test 2024-07-25 18:56:10 +05:30
Agnish Ghosh 874488843a add: hypergeom cdf 2024-07-25 17:58:29 +05:30