* use storestream for retrieval
* use `len` prop on manifest
* remove read pattern
* make blocksize and chunksize consistent
* fix tests
* fix chunker tests - now padded by default
* initial commit of the Shacham RSA-based public scheme
Minimal working version with lots of error checks and corrections
still needed.
- using Bearssl RSA code through libp2p
- with selecteble BigInt library for experimentation
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* better proc names
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* separating demo code from library
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* using normal file io instead of memfiles
mmap has serveral potential issues and we do not really need it, so
changing to use the normal system file interface is better.
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* draft version of bls proofs
Implementation of the BLS-based public PoS scheme from
Shacham H., Waters B., "Compact Proofs of Retrievability"
using pairing over BLS12-381 ECC
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* draft test and benchmark code for BLS PoS
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* improve documentation of BLS scheme
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* fix getSector
* fixing DST tag in hashToG1
The DST tag should be unique to achieve domain separation
of hash functions as defined in:
https://tools.ietf.org/id/draft-irtf-cfrg-hash-to-curve-06.html#domain-separation
Changed DST tag to one that indicates the PoC status of this code.
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* add verifyPairings abstraction
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* change random number generator to a secure one
Use Rng based on BrHmacDrbgContext
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* fix benchmark template
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* exchange parameter order in pairing
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* add optimized verifyPairing implementation
When verifying two pairings, one final exponentiation
can be spared through the use of cneg.
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* speed up tag generation by a factor of s
Scalar multiplications in tag generation can be rearranged
to benefit from the way random points are being generated.
Since random points are themselves generated using scalar
multiplication and the base is common, the sum of multiplications
becomes a single multiplication with the scalar sum, resulting in
a nice speedup.
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* use blst_p1_add_or_double instead of blst_p1_add
blst exposes two add functions: one that works for the corner case
of doubling, and one that isn't. It seems safer to use the one that
works, even if it is highly improbable in these cases that doubling
would occur.
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* sectorsperblock should be an external parameter
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* parametrize sectorsblock and querylen
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* improving benchmark messages
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* rebasing main
* generateAuthenticator: remove unused ubase parameter from naive impl
No need to have the same interface on the two implementations, so
we can remove this parameter.
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* generateAuthenticator: add some more explanation
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* renaming pos.nim to rsa.nim
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* sign and verify metadata in Tau
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* adding more comments
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* remove code of slow RSA based version
Removed RSA-based version to ease maintenance, as it is
highly unlikely we would use it.
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* formatting: use just one type section
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* more comments added
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* make `namelen` a const
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* generalize hashToG1
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
* hashNameI: switch to faster implementation
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
Co-authored-by: Tanguy <tanguy@status.im>
Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
* feat: introduce LRU cache
Replace `MemoryStore` with LRU caching mechanism.
`lrucache` library was forked to https://github.com/status-im/lrucache.nim.
Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
# Conflicts:
# dagger/dagger.nim
# dagger/stores.nim
# dagger/stores/manager.nim
# tests/dagger/blockexc/testengine.nim
# tests/dagger/helpers/nodeutils.nim
# tests/dagger/testnode.nim
# tests/dagger/teststores.nim
* feat: introduce --cache-size CLI option
Allow for a value of `0` to disable the cache.
# Conflicts:
# dagger/dagger.nim
* allow dynamic block size in cache
allow block size to be variable/dynamic
update lrucache to use updated lrucache dep
Using removeLru proc, added tests
* Refactor CacheStore init block
Co-authored-by: Michael Bradley, Jr <michaelsbradleyjr@gmail.com>
This PR achieves the following:
1. Improves the exception handling when dialling a peer fails or an unknown error occurs.
2. Add a `Content-Type` header to the `/download` endpoint of `application/octet-stream`, which is [defined by MDN](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types#types) as meant to be used for "generic binary data (or binary data whose true type is unknown)".
Co-authored-by: Michael Bradley <michaelsbradleyjr@gmail.com>
Integrate dagger contracts from `nim-dagger-contracts` repo.
Add `dagger-contracts`, `nim-web3`, and all of `nim-web3`’s transitive deps as submodule deps to `nim-dagger`. Note: `nim-web3` and its transitive deps may no longer be needed when we switch to `nim-ethers`.
Add a `testContracts` nimble task to test all of the contracts functionality. Namely, this spins up an ethereum simulator, deploys the contracts (in `dagger-contracts`), runs the contract tests, and finally, regardless of success/error, kills the ethereum sim processes. The nimble task can be run with `./env.sh nimble testContracts`.
We also tested `nim-dagger-contracts` as a submodule dep of `nim-dagger`, and while the tests run as expected, the preference is to merge `nim-dagger-contracts` inside of `nim-dagger` for ease of parallel development. There’s also a high probability that `nim-dagger-contracts` is not being used as a dep by other projects. Are there any strong objections to this?
Co-authored-by: Michael Bradley <michaelsbradleyjr@gmail.com>
Downloading a CID that is unknown to the network would wait for the blockexchange to try to get the data. However, a subsequent API call, eg upload for that CID, or a blockexchange timeout would cause the dagger node to crash, due to an attempt to complete a future that had already been completed.
We were able to narrow this down to an attempt to call `resp.finish()` when the download block had already returned an `RestApiResponse` (and never sent any chunks to the response from the libp2p stream).
Change the HTTP response code for an unknown cid from 400 to 404.
Fix spelling mistake “cunk”.
Co-authored-by: Michael Bradley <michaelsbradleyjr@gmail.com>
Co-authored-by: Michael Bradley <michaelsbradleyjr@gmail.com>
* adding basic comments to rest api
* Update dagger/rest/api.nim
Co-authored-by: Michael Bradley <michaelsbradleyjr@gmail.com>
Co-authored-by: Michael Bradley <michaelsbradleyjr@gmail.com>
* fix compilation issues with latest libp2p
* change package description
* fix CI compiler
* try with nim 1.4.2
* try with 1.4.2
* use more standardized ci config
* set reasonable log level
* remove nim version dependency
* reverting to 1.4.6 as otherwise it fails on amd64
* use PeerInfo in event handlers
* use CidV1 and raw multicodec as default
* add block stream abstraction
* raises defect
* adding dataset abstraction
* move blockstream into own dir
* reorg files and fix tests
* rename dataset to blockset
* wip
* wip
* adding basic test for treehash algo
* run blockset tests along with with the rest
* remove obsolete contents
* fix chunker tests
* rename bitswap and move to stores
* rename bitwsap to blockexc and move to stores
* moare project structure reorg
* moving protobuf into bitswap
* adding block type
* reworking bitswap
* adding chunker
* adding license header
* use 1.2.6
* adding fixed size chunker
* add blockstore
* add iterator to chunker
* more bitswap changes
* rename ipfs to dagger
* rename to dagger
* blockstore inherits from BlockProvider
* wip - add core block handling logic
* smal changes
* use proper block store methods
* adding asynq heapqueue
* wip prepare for bitswap task runner
* adding `$`
* adding memory store and tests
* fixed chunking
* extracted desicion engine from bitswap
* added helper random funcs
* adding testing helpers
* only handle seqs
* add peer events
* cleanup pending blocks on blockstore event
* allow nil handlers
* move protobuf type helpers
* allow initializing block from Cid
* testing and fixes
* small fixes
* expose `<`
* spelling
* default value
* spelling
* pending blocks manager
* adding stores manager
* more tests a wip around bitswap
* small changes
* merge bitswap and engine for now
* for now run only the new poc's tests
* add a more complete ci setup
* use template in map
* remove p2pd
* remove go
* dont use asyncCheck
* few small changes
* adding ability to update items
* adding multiple task runners
* handle cancelation properly
* use Result instead of throwing
* wip bitswap tests
* moving things around
* split out engine again
* add request and handlers interface
* fix tests
* wip - engine tests
* remove unused imports
* fix tests
* cleanup block requesting logic
* add block request tests
* more block requests
* add support for max heap
* don't use result
* use max heap & send block presence in task handler
* add task handler tests
* rename store to localStore
* cleanup & logging
* cancel task on stop
* don't depend on local store for events
* dont use heap queue for wants
* add chronicles
* fix issue with peer wants
* add test for delayed block sends
* remove obsolete tests
* wip chunker
* run all tests
* add todo
* misc
* remove irrelevant files
* removing more files
* adding helpers for bitswap tests
* moved bitswap file
* misc
* make blocks timeout longer
* adjust block timeout
* speedup test
* compile with threads
* import missing crypto
* misc
* disable threads for now
* fix 32 bit platforms
* re-enable threads support in tests
Mark Spanbroek
Dmitriy Ryajov
Csaba Kiraly
Eric Mastro
Eric Mastro