codex-storage
/
nim-codex
mirror of https://github.com/codex-storage/nim-codex.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
557 Commits 167 Branches 9 Tags 198 MiB
Commit Graph

557 Commits

Author SHA1 Message Date
Michael Bradley, Jr 5aa42541b3 [build] update nim-datastore 
also make changes to `codex/codex.nim` and tests for SQLiteStore re: changes in
nim-datastore

closes #178
 2022-08-10 13:13:07 -05:00
markspanbroek 097a4cfd67
Better command line options for Ethereum (#181) 
* [contracts] ContractInteractions.new() now requires account parameter

* [cli] Only perform Ethereum interactions when --eth-account is specified

* [cli] Add --persistence option that is disabled by default

* [cli] Use Option for ethDeployment parameter

* [node] Better error reporting when Ethereum node cannot be reached
 2022-08-09 06:29:06 +02:00
Michael Bradley, Jr eebcf7e7b2 [node] custom iterator/query for SQLiteStore's `method listBlocks` 2022-08-08 17:48:37 -05:00
Michael Bradley, Jr 2ecf750959 [node] support self.cache=nil in SQLiteStore 
also fix a discrepancy where cli option `--cache-size` is documented as
`0 disables the cache` in `codex/conf.nim`, but previously the value `0` would
result in a cache being constructed with default parameter values for
`CacheStore.new()`

Closes #180
 2022-08-08 16:24:45 -05:00
Mark Spanbroek b7df2d151c [build] Update dagger-contracts 2022-08-08 14:00:55 +02:00
Mark Spanbroek 7f5310d640 [rest api] Allow user to specify amount of hosts 2022-08-08 14:00:55 +02:00
Mark Spanbroek fd6f8a1386 [sales] Pass along request and slot index in onProve callback 2022-08-08 14:00:55 +02:00
Mark Spanbroek cb5427ca95 [sales] Pass along request and slot index in onStore callback 2022-08-08 14:00:55 +02:00
Mark Spanbroek 3a3d083345 [contracts] Replace size by slotSize 2022-08-08 14:00:55 +02:00
Mark Spanbroek 6ec2d3bf8f Update dagger-contracts 2022-08-08 14:00:55 +02:00
Mark Spanbroek bf11ef5b55 [sales] Use correct slot price 2022-08-08 14:00:55 +02:00
Mark Spanbroek 057de4e928 [sales] Choose a slot index at random 2022-08-08 14:00:55 +02:00
Mark Spanbroek b4b6fdb7c6 [contracts] subscribe to SlotFilled event 2022-08-08 14:00:55 +02:00
Mark Spanbroek 7995fd71d5 [market] remove fulfillRequest() 
It is superseded by fillSlot()
 2022-08-08 14:00:55 +02:00
Mark Spanbroek 04e0c54e95 [market] Remove old getHost() overload 
Is superseded by the new overload that includes
a slot id parameter.
 2022-08-08 14:00:55 +02:00
Mark Spanbroek 698baeef35 [purchasing] Remove Purchase.selected 
There no longer exists a single host that
is selected for a contract.
 2022-08-08 14:00:55 +02:00
Mark Spanbroek ac6d17e895 [purchasing] Remove fulfillRequest() from test 2022-08-08 14:00:55 +02:00
Mark Spanbroek dc95c863d2 [sales] replace fulfillRequest() by fillSlot() 2022-08-08 14:00:55 +02:00
Mark Spanbroek 156cd5ba73 [market] Replace fulfillRequest() by fillSlot() 2022-08-08 14:00:55 +02:00
Mark Spanbroek 67e4a28ed0 [contracts] Replace fulfillRequest() by fillSlot() 2022-08-08 14:00:55 +02:00
Mark Spanbroek 0494b3617b [contracts] replace totalNodes and nodeId by slots 2022-08-08 14:00:55 +02:00
Mark Spanbroek 360c03e7c4 [contracts] Rename: maxPrice -> reward 2022-08-08 14:00:55 +02:00
Mark Spanbroek 5a00eddfba Update dagger-contracts 2022-08-08 14:00:55 +02:00
Michael Bradley, Jr 75666d01bf [node] add SQLiteStore backend 
Closes #138
 2022-08-08 02:12:43 -05:00
Michael Bradley, Jr 92d83bb1d2 [build] add github.com/arnetheduck/nim-sqlite3-abi to vendor 2022-08-08 02:12:43 -05:00
Michael Bradley, Jr 679df23b6f [build] add github.com/status-im/nim-datastore to vendor 2022-08-08 02:12:43 -05:00
Dmitriy Ryajov 625facff4a
Better batching (#170) 
* add cleaner batching

* pass block instead of cid

* cleanup manifest fetching

* bug: reference correct cid version

* add batch tests

* make sure batch is always <= batchSize
 2022-07-29 14:04:12 -06:00
Dmitriy Ryajov 48368893c9
Add limits for inflight requests (#169) 
* convert network to async

* use async network api

* test with async network

* test concurrent send limits
 2022-07-29 10:19:34 -06:00
Dmitriy Ryajov 0b786b383f
adding missing `/` to upload example 2022-07-28 12:03:26 -06:00
Dmitriy Ryajov 7ccde112f8
Contract store (#161) 
* split out manifest and block retrieval

* adding test for `fetchManifest`

* raise exceptions in onStore
 2022-07-28 11:44:59 -06:00
Bulat-Ziganshin 0bfe26440e
WIP: Harmonize BlockStore API (remaining functions) (#123) (#130) 
BlockStore API got new return types (rationale in https://github.com/status-im/nim-codex/issues/123#issuecomment-1163797753):
- getBlock: Future[?! (?Block)]
- putBlock/delBlock/listBlocks: Future[?!void]
- hasBlock: Future[?!bool]

Plus refactored readOnce(StoreStream) and check received data in its tests.

And replaced local use of AsyncHeapQueue with seq.sort.
 2022-07-28 03:39:17 +03:00
Mark Spanbroek afdb5be2d4 Add TODOs to indicate wanted architecture 
As discussed in team:
- Retrieving of blocks that are mentioned in the
  manifest should be moved into a BlockStore.
- Sales should be given an instance of BlockStore,
  and an instance of StorageProofs, instead of the
  onStore, onClear, and onProve callbacks
 2022-07-20 13:43:20 +02:00
Mark Spanbroek 13bbf2b052 Fix test that was failing intermittently 
Give test more time to complete but do not increase
sleep time more than necessary, by introducing
waiting loop for test purposes.
 2022-07-20 13:43:20 +02:00
Mark Spanbroek 6df0c9ab40 [integration] set expiry always to high value 
Reasoning: a purchase is now completed when
the first proof comes in, so the test will no
longer wait until the request has expired.
 2022-07-20 13:43:20 +02:00
Mark Spanbroek 97a5237d44 [sales] Add documentation 2022-07-20 13:43:20 +02:00
Mark Spanbroek 571aa6f694 [sales] Consistent names for callbacks 2022-07-20 13:43:20 +02:00
Mark Spanbroek 592a3dce4d [sales] Add callback for clearing storage when no longer needed 2022-07-20 13:43:20 +02:00
Mark Spanbroek adcb91a5d5 [node] Store datasets locally when sales asks for it 2022-07-20 13:43:20 +02:00
Mark Spanbroek 7bc5280596 [node] rename PrefetchBatch -> Batch 
Reason: in a next commit we'll use the same
constant to fetch data in batches
 2022-07-20 13:43:20 +02:00
Mark Spanbroek d4025f4a5f [manifest] Extract logic for decoding a manifest given a cid 2022-07-20 13:43:20 +02:00
Mark Spanbroek e06211f827 [sales] rename: negotiation -> sales agent 
Reason: with the new design there are no offers and
selection, making this less like a negotation, and
more like an agent trying to fulfill a storage request
 2022-07-20 13:43:20 +02:00
Mark Spanbroek 96ca1de768 Cleanup: remove StorageOffer everywhere 2022-07-20 13:43:20 +02:00
Mark Spanbroek bd2fba50c7 [purchasing] Fix integration test 2022-07-20 13:43:20 +02:00
Mark Spanbroek 4a4588e2a0 [node] Set callbacks for Sales to retrieve and prove posession of data 2022-07-20 13:43:20 +02:00
Mark Spanbroek 94f6838d19 [node] Fix: ensure that retrieve works for small files 2022-07-20 13:43:20 +02:00
Mark Spanbroek f14eeb688b [WIP sales] Handle request expiry 2022-07-20 13:43:20 +02:00
Mark Spanbroek 9438aba5d2 [WIP sales] Handle other host fulfilling request 2022-07-20 13:43:20 +02:00
Mark Spanbroek 7f864570bd [WIP sales] call onSale() 2022-07-20 13:43:20 +02:00
Mark Spanbroek a2570f9380 [WIP sales] fulfill request 2022-07-20 13:43:20 +02:00
Mark Spanbroek 5abd5dd674 [WIP sales] make storage available again upon failure 2022-07-20 13:43:20 +02:00