9ca4f90cf3
safe tables access ( #95 )
...
* safer table access
* use withValue
2022-05-12 14:23:05 -06:00
cf062670f6
rename addOrAwait to getWantHandle ( #97 )
2022-05-12 14:09:40 -06:00
46197f957b
add onblock callback to list blocks visitor ( #96 )
...
* add onblock callback to list blocks visitor
* tests for listBlocks
* remove legacy blockList
2022-05-12 14:02:30 -06:00
ce59dbd4a2
Revert "Cleanup engine and rework discovery ( #87 )"
...
This reverts commit 4740ffc144
2022-04-20 14:28:11 +02:00
4740ffc144
Cleanup engine and rework discovery ( #87 )
...
* rework discovery with async queues
* misc style changes
* increase max message size for large manifests
* use upraises and avoid exceptions on key access
* increase sleep time to 100 millis
* pass config
* make list blocks trigger a callback on each block
* check for nil on start/stop
* fix tests and split out discovery tests
* don't auto mount network
* add list block tests
* add discovery tests
* rework moc discovery
* move discovery moc to disc dir
* don't force logging syncs
* don't force moc discovery on all tests
* rework discovery with methods
* add top level utils file
* don't use asyncCheck
* don't pass entire blocks to list blocks calback
* spelling
2022-04-19 21:46:44 -06:00
c06c9b578c
Various dev-ops things ( #85 )
...
* Add metrics server
* Add --version
* Fixes for chronicles=trace + ci
2022-04-14 12:49:03 +02:00
4d681102e5
Add DHT ( #75 )
...
* First implem
* Add persistent net key option
* Working DHT setup
* Bootstrap nodes
* Implement DaggerNode.findPeer
* Remove unrelevant comment
* Added discovery to blockexchange requestBlock
* add FSStore.blockList
* Block advertisement
* Tests compiles
* Green tests
* toDiscoveryId instead of toNodeId
* remove stopAdvertisingBlock
* Removed nim-eth dependency
* Move discovery stuff to discovery.nim
* Add missing file, start of discovery tests
* Better discovery logic
* Add tests
* Address comment
* Better E2E test
2022-04-13 10:32:35 -06:00
d204b518e8
Make block a ref ( #56 )
...
* cleanup manifest
* make block a ref type
2022-03-18 13:50:53 -06:00
2188f69273
Minor cleanup and update deps ( #42 )
...
* abort download if atleast one block failed
* make chunker a regular object
* updating questionable
* updating vendor deps
* cleanup unused imports
* Remove mention of Waku
2022-01-12 16:19:37 -06:00
a9721dd65f
Block.init should return Result ( #37 )
...
* Block.init should return Result
* use get() instead of tryGet()
2022-01-10 20:25:13 -06:00
fbe161a073
Node setup ( #32 )
...
* settup basic nim node
* adding http utils
* adding confutils
* rough rest api proto
* adding missing deps
* turn tls emulation off
* adding toml serialization
* wip
* adding missing deps
* make sure to clean old state in teardown
* adding file upload rest endpoint
* renaming blockexchange to networkstore
* updating nim-presto
* updating libp2p
* wip adding streaming upload
* reworked chunking
* bump to latest unstable
* adding asyncfutures stream
* make streamable
* deleting unused files
* reworking stores api
* use new stores api
* rework blockset and remove blockstream
* don't return option from constructor
* rework chunker
* wip implement upload
* fix tests
* move unrelated logic to engine
* don't print entire message
* logging
* basic encode/decode to/from dag-pb
* add basic upload/download support
* fix tests
* renaming blockset to manifest
* don't pass config to node
* remove config and use new manifest
* wip: make endpoints more reliable
* wip: adding node tests
* include correct manifest test
* removing asyncfutures
* proper chunking of files
* simplify stream reading
* test with encoding/decoding with many blocks
* add block storing tests
* adding retrieval test
* add logging
* tidy up chunker
* tidy up manifest and node
* use default chunk size
* fix tests
* fix tests
* make sure Eof is set properly
* wip
* minor cleanup
* add file utils
* cleanup config
* splitout DaggerServer and "main"
* remove events since they are not used
* add broadcast method to network peer
* add and wire localstore
* use localstore in the node
* wip
* logging
* move file utils
* use the constant
* updating deps
* fix memstore
* use latest libp2p unstable
* fix tests
* rework block streaming
* don't fail storing if the block already exists
* add helper info endpoint
* correct comment
* rename localstore to fsstore
* fix tests
* remove unused tests
* add test to retrieve one block
* move some test files around
* consolidate setup
* Update dagger/blockexchange/engine.nim
Co-authored-by: Tanguy <tanguy@status.im>
* typo
* better block path handling
* don't inherit rootobj
* remove useless template
* Update tests/dagger/blockexc/testblockexc.nim
Co-authored-by: markspanbroek <mark@spanbroek.net>
* use isMainModule
* use proper flag for starter/stoped
* cleanup optional use
* wrap in isMainModule
* use `cancelAndAwait`
* remove unused imports
* wip
* don't use optional
* use functional error api
* rework store tests and add fs tests
* Block.new() to Block.init()
* don't use optional for engine blocks
* use result instead of optional for getBlock
* remove unused imports
* move stopping servers to `shutdown`
* use result instead of optional
* rework with results
* fix tests
* use waitFor in signal handlers
* error helper
* use `?` and mapFailure where possible
* remove unnecesary `=?`
* improve empty cid digest initialization
Co-authored-by: Tanguy <tanguy@status.im>
Co-authored-by: markspanbroek <mark@spanbroek.net>
2022-01-10 09:32:56 -06:00
ef5fa96c7d
Fix unused import warnings
2021-11-16 16:51:24 +01:00
ce66e43440
fix compilation issues with latest libp2p ( #27 )
...
* fix compilation issues with latest libp2p
* change package description
* fix CI compiler
* try with nim 1.4.2
* try with 1.4.2
* use more standardized ci config
* set reasonable log level
* remove nim version dependency
* reverting to 1.4.6 as otherwise it fails on amd64
2021-10-29 13:30:52 -06:00
2fb39ca4a3
Rename bitswap ( #25 )
...
* use PeerInfo in event handlers
* use CidV1 and raw multicodec as default
* add block stream abstraction
* raises defect
* adding dataset abstraction
* move blockstream into own dir
* reorg files and fix tests
* rename dataset to blockset
* wip
* wip
* adding basic test for treehash algo
* run blockset tests along with with the rest
* remove obsolete contents
* fix chunker tests
* rename bitswap and move to stores
* rename bitwsap to blockexc and move to stores
* moare project structure reorg
2021-08-30 13:25:20 -06:00
Dmitriy Ryajov
Tanguy
Mark Spanbroek