Commit Graph

10 Commits

Author SHA1 Message Date
Bulat-Ziganshin 7d6330b938 Disable advertiseQueueLoop completely
It's the main CPU/memory eater
2022-09-08 12:25:13 +03:00
Bulat-Ziganshin 84536744d4 Disable only advertiseQueue.put
when listBlocks isn't called by advertiseQueueLoop, we have 0% of CPU usage. so
1) idle node burns cpu exclusively inside advertiseQueueLoop
2) idle node memleaks exclusively via advertiseQueue.put
2022-09-08 06:04:20 +03:00
Dmitriy Ryajov 4bc701652f
Adding metrics (#203)
* adding inflight discovery metric

* adding metrics instructions and default dashboard

* spelling fixes
2022-08-23 10:11:21 -06:00
Michael Bradley, Jr 3d823dcbc6 [stores] update getBlock return type
change return type for `method getBlock` from `Future[?!(?Block)]` to
`Future[?!Block]`

use `type BlockNotFoundError = object of CodexError` to differentiate between
"block not found in the store" and other errors

also make some logic and error handling/messages more consistent across
BlockStore implementations

closes #177
closes #182
closes #210

alternative to #205, #209
2022-08-22 20:52:51 -05:00
Dmitriy Ryajov 48368893c9
Add limits for inflight requests (#169)
* convert network to async

* use async network api

* test with async network

* test concurrent send limits
2022-07-29 10:19:34 -06:00
Bulat-Ziganshin 0bfe26440e
WIP: Harmonize BlockStore API (remaining functions) (#123) (#130)
BlockStore API got new return types (rationale in https://github.com/status-im/nim-codex/issues/123#issuecomment-1163797753):
- getBlock: Future[?! (?Block)]
- putBlock/delBlock/listBlocks: Future[?!void]
- hasBlock: Future[?!bool]

Plus refactored readOnce(StoreStream) and check received data in its tests.

And replaced local use of AsyncHeapQueue with seq.sort.
2022-07-28 03:39:17 +03:00
Dmitriy Ryajov 6ce7e23767
Upload authenticators (#108)
* initial implementation of storage proofs upload

* make sure proof verifies with after deserializing

* add por store

* rename por store to stp store

* rename porstore to stpstore

* add support for host discovery to discovery mock

* add tags upload network tests
2022-05-25 20:29:31 -06:00
Dmitriy Ryajov 6ad7a6bb96
Batched blocks (#105)
* remove sleep

* throttle prefetch

* break loop if conn closed

* logging

* prefetch blocks in batches

* add blocks batch constant
2022-05-20 10:53:34 -06:00
Dmitriy Ryajov d4f3ebc867
reorg engine directory structure (#104) 2022-05-19 16:28:53 -06:00
Michael Bradley 209343087c
Change every dagger to codex (#102) 2022-05-19 13:56:03 -06:00