5f48de6a44
cleanup manifest ( #44 )
...
* cleanup manifest
* Attach decode to `Manifest`
* move manifest container to it's own file
* rebase main
* update header license year
2022-01-12 19:55:51 -06:00
68a45d8f57
use tryGet, which is more informative than get ( #45 )
2022-01-12 18:42:18 -06:00
2188f69273
Minor cleanup and update deps ( #42 )
...
* abort download if atleast one block failed
* make chunker a regular object
* updating questionable
* updating vendor deps
* cleanup unused imports
* Remove mention of Waku
2022-01-12 16:19:37 -06:00
15ef6c78eb
fix treehash generation ( #39 )
2022-01-10 21:04:00 -06:00
47a0d4ef7a
Move empty digest ( #38 )
...
* improve empty digest initialization
* cleanup warnings
2022-01-10 20:25:27 -06:00
a9721dd65f
Block.init should return Result ( #37 )
...
* Block.init should return Result
* use get() instead of tryGet()
2022-01-10 20:25:13 -06:00
fbe161a073
Node setup ( #32 )
...
* settup basic nim node
* adding http utils
* adding confutils
* rough rest api proto
* adding missing deps
* turn tls emulation off
* adding toml serialization
* wip
* adding missing deps
* make sure to clean old state in teardown
* adding file upload rest endpoint
* renaming blockexchange to networkstore
* updating nim-presto
* updating libp2p
* wip adding streaming upload
* reworked chunking
* bump to latest unstable
* adding asyncfutures stream
* make streamable
* deleting unused files
* reworking stores api
* use new stores api
* rework blockset and remove blockstream
* don't return option from constructor
* rework chunker
* wip implement upload
* fix tests
* move unrelated logic to engine
* don't print entire message
* logging
* basic encode/decode to/from dag-pb
* add basic upload/download support
* fix tests
* renaming blockset to manifest
* don't pass config to node
* remove config and use new manifest
* wip: make endpoints more reliable
* wip: adding node tests
* include correct manifest test
* removing asyncfutures
* proper chunking of files
* simplify stream reading
* test with encoding/decoding with many blocks
* add block storing tests
* adding retrieval test
* add logging
* tidy up chunker
* tidy up manifest and node
* use default chunk size
* fix tests
* fix tests
* make sure Eof is set properly
* wip
* minor cleanup
* add file utils
* cleanup config
* splitout DaggerServer and "main"
* remove events since they are not used
* add broadcast method to network peer
* add and wire localstore
* use localstore in the node
* wip
* logging
* move file utils
* use the constant
* updating deps
* fix memstore
* use latest libp2p unstable
* fix tests
* rework block streaming
* don't fail storing if the block already exists
* add helper info endpoint
* correct comment
* rename localstore to fsstore
* fix tests
* remove unused tests
* add test to retrieve one block
* move some test files around
* consolidate setup
* Update dagger/blockexchange/engine.nim
Co-authored-by: Tanguy <tanguy@status.im>
* typo
* better block path handling
* don't inherit rootobj
* remove useless template
* Update tests/dagger/blockexc/testblockexc.nim
Co-authored-by: markspanbroek <mark@spanbroek.net>
* use isMainModule
* use proper flag for starter/stoped
* cleanup optional use
* wrap in isMainModule
* use `cancelAndAwait`
* remove unused imports
* wip
* don't use optional
* use functional error api
* rework store tests and add fs tests
* Block.new() to Block.init()
* don't use optional for engine blocks
* use result instead of optional for getBlock
* remove unused imports
* move stopping servers to `shutdown`
* use result instead of optional
* rework with results
* fix tests
* use waitFor in signal handlers
* error helper
* use `?` and mapFailure where possible
* remove unnecesary `=?`
* improve empty cid digest initialization
Co-authored-by: Tanguy <tanguy@status.im>
Co-authored-by: markspanbroek <mark@spanbroek.net>
2022-01-10 09:32:56 -06:00
90a6131d7d
Fix occasional test failure due to timeout
2021-11-16 16:51:40 +01:00
ef5fa96c7d
Fix unused import warnings
2021-11-16 16:51:24 +01:00
ce66e43440
fix compilation issues with latest libp2p ( #27 )
...
* fix compilation issues with latest libp2p
* change package description
* fix CI compiler
* try with nim 1.4.2
* try with 1.4.2
* use more standardized ci config
* set reasonable log level
* remove nim version dependency
* reverting to 1.4.6 as otherwise it fails on amd64
2021-10-29 13:30:52 -06:00
2fb39ca4a3
Rename bitswap ( #25 )
...
* use PeerInfo in event handlers
* use CidV1 and raw multicodec as default
* add block stream abstraction
* raises defect
* adding dataset abstraction
* move blockstream into own dir
* reorg files and fix tests
* rename dataset to blockset
* wip
* wip
* adding basic test for treehash algo
* run blockset tests along with with the rest
* remove obsolete contents
* fix chunker tests
* rename bitswap and move to stores
* rename bitwsap to blockexc and move to stores
* moare project structure reorg
2021-08-30 13:25:20 -06:00
6bd4260911
Replace pricing exchange by account exchange
...
Rationale: price is no longer set per peer, but per chunk.
Only the Ethereum accounts of the peers needs to be exchanged.
2021-06-07 09:50:17 +02:00
d4dd15734e
Remove double bookkeeping in peerHave and peerPrices
2021-06-07 09:50:17 +02:00
e3b68f96d8
Remove debt ratio
...
Rationale: selecting peers based on debt ratio in bytes
is no longer needed when the bandwidth for each chunk is
compensated.
2021-06-07 09:50:17 +02:00
fa05bffeac
Pay per-block price instead of per-peer price
2021-06-07 09:50:17 +02:00
f69827d582
Send block prices
2021-06-07 09:50:17 +02:00
0a28338136
Simplify test
2021-06-07 09:50:17 +02:00
0e34b69d7d
Add block prices to peer context
2021-06-07 09:50:17 +02:00
0ac876dbc3
Add pricing to block presence messages
2021-06-07 09:50:17 +02:00
11da2339de
Hard-code asset address
...
We're only going to support a single asset for now.
2021-06-07 09:50:17 +02:00
811b29fb5e
Receive payments for blocks that were sent
2021-06-07 09:50:17 +02:00
5160b267b6
Use reference semantics for wallets
2021-06-07 09:50:17 +02:00
2d4a735e5f
Update to latest versions of nitro and questionable
2021-06-07 09:50:17 +02:00
ee978f354e
Engine sends payments for received blocks
2021-06-07 09:50:17 +02:00
9512bbc50b
Add proc to engine that pays peers for bytes
2021-06-07 09:50:17 +02:00
ad48f55aaa
Update to latest versions of nitro and questionable
2021-06-07 09:50:17 +02:00
4ce3f6d3da
Add nitro wallet to BitswapEngine
2021-06-07 09:50:17 +02:00
22f4c277dd
Revert "Add nitro wallet to BitswapNetwork"
...
This reverts commit e55ac4e9de
2021-06-07 09:50:17 +02:00
3c4d4dc29f
Exchange pricing when connecting to new peer
2021-06-07 09:50:17 +02:00
0897e615e6
Broadcast payment
2021-06-07 09:50:17 +02:00
6e7220efbb
Broadcast pricing
2021-06-07 09:50:17 +02:00
e02c40db80
Add handler for payment messages
2021-06-07 09:50:17 +02:00
5f9215a371
Add handler for pricing messages
2021-06-07 09:50:17 +02:00
364d3a9277
Fix typo
2021-06-07 09:50:17 +02:00
9635f26b90
Fix tests on 32 bit platforms
2021-06-07 09:50:17 +02:00
c819ba1be1
protobuf message for state channel updates
2021-06-07 09:50:17 +02:00
1f2a9f90d5
protobuf message for bandwidth pricing
2021-06-07 09:50:17 +02:00
8adea94196
Add nitro wallet to BitswapNetwork
2021-06-07 09:50:17 +02:00
5cd714ea83
Remove deprecated libp2p import
2021-06-07 09:50:17 +02:00
339f7f04bc
Fix warnings
2021-06-07 09:50:17 +02:00
5bb87c4ecc
Add missing test
2021-06-07 09:50:17 +02:00
6c92b3dc25
Poc 2 ( #7 )
...
* moving protobuf into bitswap
* adding block type
* reworking bitswap
* adding chunker
* adding license header
* use 1.2.6
* adding fixed size chunker
* add blockstore
* add iterator to chunker
* more bitswap changes
* rename ipfs to dagger
* rename to dagger
* blockstore inherits from BlockProvider
* wip - add core block handling logic
* smal changes
* use proper block store methods
* adding asynq heapqueue
* wip prepare for bitswap task runner
* adding `$`
* adding memory store and tests
* fixed chunking
* extracted desicion engine from bitswap
* added helper random funcs
* adding testing helpers
* only handle seqs
* add peer events
* cleanup pending blocks on blockstore event
* allow nil handlers
* move protobuf type helpers
* allow initializing block from Cid
* testing and fixes
* small fixes
* expose `<`
* spelling
* default value
* spelling
* pending blocks manager
* adding stores manager
* more tests a wip around bitswap
* small changes
* merge bitswap and engine for now
* for now run only the new poc's tests
* add a more complete ci setup
* use template in map
* remove p2pd
* remove go
* dont use asyncCheck
* few small changes
* adding ability to update items
* adding multiple task runners
* handle cancelation properly
* use Result instead of throwing
* wip bitswap tests
* moving things around
* split out engine again
* add request and handlers interface
* fix tests
* wip - engine tests
* remove unused imports
* fix tests
* cleanup block requesting logic
* add block request tests
* more block requests
* add support for max heap
* don't use result
* use max heap & send block presence in task handler
* add task handler tests
* rename store to localStore
* cleanup & logging
* cancel task on stop
* don't depend on local store for events
* dont use heap queue for wants
* add chronicles
* fix issue with peer wants
* add test for delayed block sends
* remove obsolete tests
* wip chunker
* run all tests
* add todo
* misc
* remove irrelevant files
* removing more files
* adding helpers for bitswap tests
* moved bitswap file
* misc
* make blocks timeout longer
* adjust block timeout
* speedup test
* compile with threads
* import missing crypto
* misc
* disable threads for now
* fix 32 bit platforms
* re-enable threads support in tests
2021-02-25 18:23:22 -06:00
8e76ecfa9f
Fix: adapt to latest libp2p api
...
Fixes compilation error.
2021-02-08 14:39:27 +01:00
b0a1166c3c
Bitswap: retrieve() waits until IPFS object has been retrieved
...
Workaround that always waited for exactly 1 second has been removed.
2021-02-02 19:29:52 +01:00
b54c3d9585
Implement Ipfs API
2021-02-02 19:29:52 +01:00
a63000f602
Exchange objects between peers using bitswap
2021-02-02 19:29:52 +01:00
28d1ddf7e3
Repo: use Option to signal retrieval success or failure
2021-02-02 19:29:52 +01:00
520f3f3bc9
Add Bitswap protocol for libp2p
2021-02-02 19:29:52 +01:00
e6d84b9c0d
Add initializers for bitswap messages
2021-02-02 19:29:52 +01:00
fa15c92158
Serialize bitswap messages using protobuf
2021-02-02 19:29:52 +01:00
b3f5599925
Add absolutely minimal implementation of a DHT routing table
2021-02-02 19:29:52 +01:00
7b15a6ae83
Disable failing test for now
2021-02-02 19:29:52 +01:00
dc06106ae1
Change MerkleDag to IpfsObject to better match IPFS API
2021-02-02 19:29:52 +01:00
5a36e26e7c
Change type and proc names to better match IPFS API
2021-02-02 19:29:52 +01:00
05a7f47113
Rename dagger->ipfs
...
This is a storage engine based on IPFS, which is only a part of Dagger
2021-02-02 19:29:52 +01:00
dd18f50ec6
Replace root hash by root content id
2021-02-02 19:29:52 +01:00
096fe0e40d
Repo for storing and retrieving Merkle DAGs
2021-02-02 19:29:52 +01:00
ecc4e50d28
Add root hash to Merkle DAG
2021-02-02 19:29:52 +01:00
c795c698fe
Compile all tests in one go
2021-02-02 19:29:52 +01:00
5f5153820f
Add absolutely minimal implementation of chunking
2021-02-02 19:29:52 +01:00
2471423197
Added (failing) test for transferring file from one peer to another
2021-02-02 19:29:52 +01:00
bd29797aba
Project setup
2021-02-02 19:29:52 +01:00
Dmitriy Ryajov
Mark Spanbroek