Commit Graph

17 Commits

Author SHA1 Message Date
Dmitriy Ryajov 8d61391073
Light cleanup (#616)
mostly whitespaces cleanup
2023-11-14 09:52:27 -08:00
Tomasz Bekas 2396c4d76d
Blockexchange uses merkle root and index to fetch blocks (#566)
* Blockexchange uses merkle root and index to fetch blocks

* Links the network store getTree to the local store.

* Update codex/stores/repostore.nim

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
Signed-off-by: Tomasz Bekas <tomasz.bekas@gmail.com>

* Rework erasure.nim to include recent cleanup

* Revert accidential changes to lib versions

* Addressing review comments

* Storing proofs instead of trees

* Fix a comment

* Fix broken tests

* Fix for broken testerasure.nim

* Addressing PR comments

---------

Signed-off-by: Tomasz Bekas <tomasz.bekas@gmail.com>
Co-authored-by: benbierens <thatbenbierens@gmail.com>
Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2023-11-14 13:02:17 +01:00
Slava 11cd2c46ad
Rename metrics (#478) (#603) 2023-11-03 17:21:54 +02:00
Ben Bierens 040434aa19
renames log properties named 'count' (#560) 2023-10-03 13:33:01 +02:00
Jaremy Creechley 7efa9177df
Bump deps take2 (#492)
* extra utilities and tweaks
* add atlas lock
* update ignores
* break build into it's own script
* update url rules
* base off codexdht's
* compile fixes for Nim 1.6.14
* update submodules
* convert mapFailure to procs to work around type resolution issues
* add toml parser for multiaddress
* change error type on keyutils
* bump nimbus build to use 1.6.14
* update gitignore
* adding new deps submodules
* bump nim ci version
* even more fixes
* more libp2p changes
* update keys
* fix eventually function
* adding coverage test file
* move coverage to build.nims
* use nimcache/coverage
* move libp2p import for tests into helper.nim
* remove named bin
* bug fixes for networkpeers (from Dmitriy)

---------

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2023-08-01 16:47:57 -07:00
Ben Bierens 14c5270e83
Add metrics (#478)
* Adds metrics to block exchange.

* Adds metrics to purchasing

* Adds metrics to upload and download API

* Adds metrics to the repo store

* Fixes exception in repostore start.

* Merge managed to mess up indentation.
2023-07-20 09:56:28 +02:00
jessiebroke ac2aca42b1
Block exchange debug (#426)
* Adds sequence diagram for local download

* Adds network download flow

* makes node properly request block when presence-list is received

* starting to work :O

* Fixes fully connected download test up to 20 nodes.

* Better sequence diagram of network block resolution

* Cleans up comments and traces.

* Accidentally deleted important else

* Review comments by Dmitriy

---------

Co-authored-by: benbierens <thatbenbierens@gmail.com>
2023-07-18 07:50:47 +02:00
Jaremy Creechley e47b38af11
Improving proc/func formatting consistency (#454)
* Fixes/workarounds for nimsuggest failures in codex.nim.

* remove rng prefix - it appears to work now

* format new's to be more consistent

* making proc formatting a bit more consistent
2023-06-22 08:11:18 -07:00
Ben Bierens da79660f8e
Enable stylecheck (#353)
* applying styleCheck

* stuck on vendor folder

* Applies style check

* Turns styleCheck back off

* switches to stylecheck:usages

* Fixes empty template casing

* rolls up nim-blscurve, nim-datastore, nim-ethers, nim-leopard, and nim-taskpools.

* bumps nim-confutils and removes unused import from fileutils.nim

* Unused using in fileutils.nim is required by CI

* Reverts bump of nim-confutils module
2023-03-10 08:02:54 +01:00
markspanbroek 7a0a48e4a5
Fix warnings (drops Nim 1.2) (#348)
* [build] disable XCannotRaiseY hint

There are too many {.raises:[Defect].} in the
libraries that we use, drowning out all other
warnings and hints

* [build] disable BareExcept warning

Not yet enabled in a released version of Nim,
so libraries that we depend on have not fixed
this yet, drowning out our own hints and warnings

* [build] disable DotLikeOps warning

dot-like ops were an experiment that is not going
land in Nim

* [build] compile log statements in tests

When running tests, all log statements are compiled.
They are filtered out at runtime during a test run.

* [build] do not build executable when running unit test

It's already built in the integration test

* [build] Fix warnings

- remove unused code
- remove unused imports
- stop using deprecated stuff

* [build] Put compiler flags behind nim version checks

* [CI] remove Nim 1.2 compatibility
2023-03-09 12:23:45 +01:00
Dmitriy Ryajov 5abf80cc69
Block download (#304)
* track inflight requests

* preperly handle precense updates

* trace number of of scheduled blocks

* invoke `payForBlocks` at the correct time

* reduntant block info on want list updates

* don't update prices in task handler

* PeerID -> PeerId

* cleanup

* proper log topic

* better chronicles topic filtering

* more trace logging

* sort want blocks

* wip - fix tests

* wip - fix tests, presence changes

* fix small test issue

* return price

* payment related changes

* misc

* re-enable payment tests

* fix warn wording

* fix `u256` conversion

* minor misc changes

* don't idle for so long on `encode`

* logging

* move buff

* disable cache by default

* disable cache by default

* fix streamOneBlock

* log node stopping/exiting

* trace logging

* don't stringify cid

* use `self`

* quick cleanup

* rename enums

* rename enums

* turns out we don't needs this test

* fix wording
2022-11-15 09:46:21 -06:00
Michael Bradley c893b1f0cb
track latest nim-libp2p's master branch (#248)
* [build] track nim-libp2p's unstable branch

* cid formatIt change

* track nim-libp2p-unstable

* don't stringify cid

* Fixed testblockexc.nim

1. Assign price to blocks
2. Delete on node1 cached blocks from node2 before buying them again

* add trace logging

* init pricing info

* remove duplicate price setting

* cid serialization in trace logs

* bumping dht to latest main

* bump

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
Co-authored-by: Bulat-Ziganshin <bulat.ziganshin@gmail.com>
2022-11-02 11:40:28 -06:00
Dmitriy Ryajov e50ea88411
Remove protobuf serialization (#289)
* add format for cid

* cid formatIt change

* track nim-libp2p-unstable

* rework probuf serialization for por

* add missing include

* removing nim protobuf serialization

* rollback to dht to main

* remove protobuf serialization import
2022-10-27 07:41:34 -06:00
Michael Bradley, Jr 3d823dcbc6 [stores] update getBlock return type
change return type for `method getBlock` from `Future[?!(?Block)]` to
`Future[?!Block]`

use `type BlockNotFoundError = object of CodexError` to differentiate between
"block not found in the store" and other errors

also make some logic and error handling/messages more consistent across
BlockStore implementations

closes #177
closes #182
closes #210

alternative to #205, #209
2022-08-22 20:52:51 -05:00
Dmitriy Ryajov 48368893c9
Add limits for inflight requests (#169)
* convert network to async

* use async network api

* test with async network

* test concurrent send limits
2022-07-29 10:19:34 -06:00
Bulat-Ziganshin 0bfe26440e
WIP: Harmonize BlockStore API (remaining functions) (#123) (#130)
BlockStore API got new return types (rationale in https://github.com/status-im/nim-codex/issues/123#issuecomment-1163797753):
- getBlock: Future[?! (?Block)]
- putBlock/delBlock/listBlocks: Future[?!void]
- hasBlock: Future[?!bool]

Plus refactored readOnce(StoreStream) and check received data in its tests.

And replaced local use of AsyncHeapQueue with seq.sort.
2022-07-28 03:39:17 +03:00
Dmitriy Ryajov d4f3ebc867
reorg engine directory structure (#104) 2022-05-19 16:28:53 -06:00