logos-storage/logos-storage-nim
1
0
Fork 0
mirror of https://github.com/logos-storage/logos-storage-nim.git synced 2026-01-04 14:33:09 +00:00
Code Issues Packages Projects Releases Wiki Activity
125 Commits 239 Branches 16 Tags 136 MiB
Commit Graph

79 Commits

Author SHA1 Message Date
Csaba Kiraly
d4e2a0cf43
hashNameI: switch to faster implementation 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:38 -06:00
Csaba Kiraly
cc3a57ba7e
generalize hashToG1 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:38 -06:00
Csaba Kiraly
a5da2ae946
make namelen a const 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:38 -06:00
Csaba Kiraly
5429872772
more comments added 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:38 -06:00
Csaba Kiraly
964276f53f
formatting: use just one type section 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:37 -06:00
Csaba Kiraly
47985bb2d8
remove code of slow RSA based version 
Removed RSA-based version to ease maintenance, as it is
highly unlikely we would use it.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:37 -06:00
Csaba Kiraly
6a03fbac29
adding more comments 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:37 -06:00
Csaba Kiraly
8693453cc9
sign and verify metadata in Tau 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:37 -06:00
Csaba Kiraly
36a4fbdbe5
renaming pos.nim to rsa.nim 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:37 -06:00
Csaba Kiraly
680c0a54b6
generateAuthenticator: add some more explanation 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:37 -06:00
Csaba Kiraly
8158ecf892
generateAuthenticator: remove unused ubase parameter from naive impl 
No need to have the same interface on the two implementations, so
we can remove this parameter.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:37 -06:00
Dmitriy Ryajov
9265c594c1
rebasing main 2022-01-17 11:26:12 -06:00
Csaba Kiraly
7112570b3f
improving benchmark messages 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:12 -06:00
Csaba Kiraly
0c5a581c5a
parametrize sectorsblock and querylen 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:12 -06:00
Csaba Kiraly
b08901c73c
sectorsperblock should be an external parameter 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:12 -06:00
Csaba Kiraly
355d0d2ab8
use blst_p1_add_or_double instead of blst_p1_add 
blst exposes two add functions: one that works for the corner case
of doubling, and one that isn't. It seems safer to use the one that
works, even if it is highly improbable in these cases that doubling
would occur.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:12 -06:00
Csaba Kiraly
6ff0031293
speed up tag generation by a factor of s 
Scalar multiplications in tag generation can be rearranged
to benefit from the way random points are being generated.
Since random points are themselves generated using scalar
multiplication and the base is common, the sum of multiplications
becomes a single multiplication with the scalar sum, resulting in
a nice speedup.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:11 -06:00
Csaba Kiraly
05e343a039
add optimized verifyPairing implementation 
When verifying two pairings, one final exponentiation
can be spared through the use of cneg.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:11 -06:00
Csaba Kiraly
6104782618
exchange parameter order in pairing 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:11 -06:00
Csaba Kiraly
73f90c2dc5
fix benchmark template 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:11 -06:00
Csaba Kiraly
b44cb86800
change random number generator to a secure one 
Use Rng based on BrHmacDrbgContext

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:11 -06:00
Csaba Kiraly
6b8c732fe0
add verifyPairings abstraction 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:11 -06:00
Csaba Kiraly
5eb2408c16
fixing DST tag in hashToG1 
The DST tag should be unique to achieve domain separation
of hash functions as defined in:
https://tools.ietf.org/id/draft-irtf-cfrg-hash-to-curve-06.html#domain-separation

Changed DST tag to one that indicates the PoC status of this code.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:11 -06:00
Tanguy
09996476de
fix getSector 2022-01-17 11:26:10 -06:00
Csaba Kiraly
998aad6378
improve documentation of BLS scheme 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:10 -06:00
Csaba Kiraly
c52b3ac30a
draft test and benchmark code for BLS PoS 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:10 -06:00
Csaba Kiraly
5ee331ac75
draft version of bls proofs 
Implementation of the BLS-based public PoS scheme from
Shacham H., Waters B., "Compact Proofs of Retrievability"
using pairing over BLS12-381 ECC

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:10 -06:00
Csaba Kiraly
0ce3852e4f
using normal file io instead of memfiles 
mmap has serveral potential issues and we do not really need it, so
changing to use the normal system file interface is better.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:10 -06:00
Csaba Kiraly
076528e8cf
separating demo code from library 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:10 -06:00
Csaba Kiraly
4ed5ba18ec
better proc names 
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:09 -06:00
Csaba Kiraly
6559ff6f04
initial commit of the Shacham RSA-based public scheme 
Minimal working version with lots of error checks and corrections
still needed.
- using Bearssl RSA code through libp2p
- with selecteble BigInt library for experimentation

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
 2022-01-17 11:26:09 -06:00
Dmitriy Ryajov
5f48de6a44
cleanup manifest (#44) 
* cleanup manifest

* Attach decode to `Manifest`

* move manifest container to it's own file

* rebase main

* update header license year
 2022-01-12 19:55:51 -06:00
Dmitriy Ryajov
2188f69273
Minor cleanup and update deps (#42) 
* abort download if atleast one block failed

* make chunker a regular object

* updating questionable

* updating vendor deps

* cleanup unused imports

* Remove mention of Waku
 2022-01-12 16:19:37 -06:00
Dmitriy Ryajov
15ef6c78eb
fix treehash generation (#39) 2022-01-10 21:04:00 -06:00
Dmitriy Ryajov
47a0d4ef7a
Move empty digest (#38) 
* improve empty digest initialization

* cleanup warnings
 2022-01-10 20:25:27 -06:00
Dmitriy Ryajov
a9721dd65f
Block.init should return Result (#37) 
* Block.init should return Result

* use get() instead of tryGet()
 2022-01-10 20:25:13 -06:00
Dmitriy Ryajov
9d17c91d4c
adding basic comments to rest api (#36) 
* adding basic comments to rest api

* Update dagger/rest/api.nim

Co-authored-by: Michael Bradley <michaelsbradleyjr@gmail.com>

Co-authored-by: Michael Bradley <michaelsbradleyjr@gmail.com>
 2022-01-10 14:35:52 -06:00
Dmitriy Ryajov
fbe161a073
Node setup (#32) 
* settup basic nim node

* adding http utils

* adding confutils

* rough rest api proto

* adding missing deps

* turn tls emulation off

* adding toml serialization

* wip

* adding missing deps

* make sure to clean old state in teardown

* adding file upload rest endpoint

* renaming blockexchange to networkstore

* updating nim-presto

* updating libp2p

* wip adding streaming upload

* reworked chunking

* bump to latest unstable

* adding asyncfutures stream

* make streamable

* deleting unused files

* reworking stores api

* use new stores api

* rework blockset and remove blockstream

* don't return option from constructor

* rework chunker

* wip implement upload

* fix tests

* move unrelated logic to engine

* don't print entire message

* logging

* basic encode/decode to/from dag-pb

* add basic upload/download support

* fix tests

* renaming blockset to manifest

* don't pass config to node

* remove config and use new manifest

* wip: make endpoints more reliable

* wip: adding node tests

* include correct manifest test

* removing asyncfutures

* proper chunking of files

* simplify stream reading

* test with encoding/decoding with many blocks

* add block storing tests

* adding retrieval test

* add logging

* tidy up chunker

* tidy up manifest and node

* use default chunk size

* fix tests

* fix tests

* make sure Eof is set properly

* wip

* minor cleanup

* add file utils

* cleanup config

* splitout DaggerServer and "main"

* remove events since they are not used

* add broadcast method to network peer

* add and wire localstore

* use localstore in the node

* wip

* logging

* move file utils

* use the constant

* updating deps

* fix memstore

* use latest libp2p unstable

* fix tests

* rework block streaming

* don't fail storing if the block already exists

* add helper info endpoint

* correct comment

* rename localstore to fsstore

* fix tests

* remove unused tests

* add test to retrieve one block

* move some test files around

* consolidate setup

* Update dagger/blockexchange/engine.nim

Co-authored-by: Tanguy <tanguy@status.im>

* typo

* better block path handling

* don't inherit rootobj

* remove useless template

* Update tests/dagger/blockexc/testblockexc.nim

Co-authored-by: markspanbroek <mark@spanbroek.net>

* use isMainModule

* use proper flag for starter/stoped

* cleanup optional use

* wrap in isMainModule

* use `cancelAndAwait`

* remove unused imports

* wip

* don't use optional

* use functional error api

* rework store tests and add fs tests

* Block.new() to Block.init()

* don't use optional for engine blocks

* use result instead of optional for getBlock

* remove unused imports

* move stopping servers to `shutdown`

* use result instead of optional

* rework with results

* fix tests

* use waitFor in signal handlers

* error helper

* use `?` and mapFailure where possible

* remove unnecesary `=?`

* improve empty cid digest initialization

Co-authored-by: Tanguy <tanguy@status.im>
Co-authored-by: markspanbroek <mark@spanbroek.net>
 2022-01-10 09:32:56 -06:00
Dmitriy Ryajov
8522f516df remove unused file 2021-11-25 09:37:12 +01:00
Mark Spanbroek
ef5fa96c7d Fix unused import warnings 2021-11-16 16:51:24 +01:00
Dmitriy Ryajov
ce66e43440
fix compilation issues with latest libp2p (#27) 
* fix compilation issues with latest libp2p

* change package description

* fix CI compiler

* try with nim 1.4.2

* try with 1.4.2

* use more standardized ci config

* set reasonable log level

* remove nim version dependency

* reverting to 1.4.6 as otherwise it fails on amd64
 2021-10-29 13:30:52 -06:00
Dmitriy Ryajov
2fb39ca4a3
Rename bitswap (#25) 
* use PeerInfo in event handlers

* use CidV1 and raw multicodec as default

* add block stream abstraction

* raises defect

* adding dataset abstraction

* move blockstream into own dir

* reorg files and fix tests

* rename dataset to blockset

* wip

* wip

* adding basic test for treehash algo

* run blockset tests along with with the rest

* remove obsolete contents

* fix chunker tests

* rename bitswap and move to stores

* rename bitwsap to blockexc and move to stores

* moare project structure reorg
 2021-08-30 13:25:20 -06:00
Mark Spanbroek
6bd4260911 Replace pricing exchange by account exchange 
Rationale: price is no longer set per peer, but per chunk.
Only the Ethereum accounts of the peers needs to be exchanged.
 2021-06-07 09:50:17 +02:00
Mark Spanbroek
d4dd15734e Remove double bookkeeping in peerHave and peerPrices 2021-06-07 09:50:17 +02:00
Mark Spanbroek
e3b68f96d8 Remove debt ratio 
Rationale: selecting peers based on debt ratio in bytes
is no longer needed when the bandwidth for each chunk is
compensated.
 2021-06-07 09:50:17 +02:00
Mark Spanbroek
fa05bffeac Pay per-block price instead of per-peer price 2021-06-07 09:50:17 +02:00
Mark Spanbroek
f69827d582 Send block prices 2021-06-07 09:50:17 +02:00
Mark Spanbroek
98d8571ceb Simplify 2021-06-07 09:50:17 +02:00
Mark Spanbroek
c57ecde046 Update to version 0.9.1 of questionable 2021-06-07 09:50:17 +02:00
Mark Spanbroek
0e34b69d7d Add block prices to peer context 2021-06-07 09:50:17 +02:00