Commit Graph

11 Commits

Author SHA1 Message Date
Ben Bierens f196caf8cb
Download API upgrade (#955)
* Adds API for fetching manifest only and downloading dataset without stream

* Updates openapi.yaml

* Adds tests for downloading manifest-only and without stream.

* review comments by Giuliano

* updates test clients
2024-10-21 13:25:19 +03:00
Dmitriy Ryajov ec8d0c98b2
Fix REST endpoints semantics (#612)
* Fix REST endpoints semantics

* update endpoint description

* update, operation id

* Adding enum support

* make enum descerializer public

* add support for listing manifests

* test `/data` endpoint to list local manifests

* debug leftovers

* remove commented out line
2023-11-20 16:14:06 -08:00
Ben Bierens cb02962231
Adds endpoint for listing files (manifests) in node. Useful for demo UI. (#599)
* Adds endpoint for listing files (manifests) in node. Useful for demo UI.

* Moves upload/download/files into content API calls.

* Cleans up json serialization for manifest

* Cleans up some more json serialization

* Moves block iteration and decoding to node.nim

* Moves api methods into their own init procs.

* Applies RestContent api object.

* Replaces format methods with Rest objects in json.nim

* Unused import

* Review comments by Adam

* Fixes issue where content/local endpoint clashes with content/cid.

* faulty merge resolution

* Renames content API to data.

* Fixes faulty rebase

* Adds test for data/local API

* Renames local and download api.
2023-11-09 08:47:09 +00:00
Adam Uhlíř 0014ffdef5
test: integration tests listen on localhost (#596) 2023-10-24 16:52:06 +02:00
markspanbroek 71cd35112b
Taiko L2 (#483)
* [docs] fix two client scenario: add missing collateral

* [integration] separate step to wait for node to be started

* [cli] add option to specify ethereum private key

* Remove unused imports

* Fix warnings

* [integration] move type definitions to correct place

* [integration] wait a bit longer for a node to start in debug mode

When e.g. running against Taiko test net rpc, the node start
takes longer

* [integration] simplify handling of codex node and client

* [integration] add Taiko integration test

* [contracts] await token approval confirmation before next tx

* [contracts] deployment address of marketplace on Taiko

* [cli] --eth-private-key now takes a file name

Instead of supplying the private key on the command line,
expect the private key to be in a file with the correct
permissions.

* [utils] Fixes undeclared `activeChroniclesStream` on Windows

* [build] update nim-ethers to include PR #52

Co-authored-by: Eric Mastro <eric.mastro@gmail.com>

* [cli] Better error messages when reading eth private key

Co-authored-by: Eric Mastro <eric.mastro@gmail.com>

* [integration] simplify reading of cmd line arguments

Co-authored-by: Eric Mastro <eric.mastro@gmail.com>

* [build] update to latest version of nim-ethers

* [contracts] updated contract address for Taiko L2

* [build] update codex contracts to latest version

---------

Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2023-09-13 16:17:56 +02:00
Ben Bierens 545e0d47e1
Slows down block maintenance iteration (#543)
* Slows down block maintenance iteration

* Fixes unstable integration test.
2023-09-04 11:12:14 +02:00
Jaremy Creechley 7efa9177df
Bump deps take2 (#492)
* extra utilities and tweaks
* add atlas lock
* update ignores
* break build into it's own script
* update url rules
* base off codexdht's
* compile fixes for Nim 1.6.14
* update submodules
* convert mapFailure to procs to work around type resolution issues
* add toml parser for multiaddress
* change error type on keyutils
* bump nimbus build to use 1.6.14
* update gitignore
* adding new deps submodules
* bump nim ci version
* even more fixes
* more libp2p changes
* update keys
* fix eventually function
* adding coverage test file
* move coverage to build.nims
* use nimcache/coverage
* move libp2p import for tests into helper.nim
* remove named bin
* bug fixes for networkpeers (from Dmitriy)

---------

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2023-08-01 16:47:57 -07:00
Dmitriy Ryajov bd594c9aaf
Create memory-leak detecting test suite (#226)
* adding tracker for streamstore

* adding tracker tests

* Sets up tracker helper functions and closes streams in testnode.nim

* Deploying checksuite for memory leak tracker checking.

* Successfully deploys checksuite and asyncchecksuite.

* Fix leak in testpor.nim

* Fixes leaked storestream in testnetwork.nim

* Fixes integration tests

* Cleanup

* cleanup comment by Mark

---------

Co-authored-by: benbierens <thatbenbierens@gmail.com>
2023-06-22 12:01:21 -06:00
markspanbroek 4ffe7b8e06
Generate proofs when required (#383)
* [maintenance] speedup integration test

* [rest api] add proofProbability parameter to storage requests

* [integration] negotiation test ends when contract starts

* [integration] reusable 2 node setup for tests

* [integration] introduce CodexClient for tests

* [node] submit storage proofs when required

* [contracts] Add Slot type

* [proving] replace onProofRequired & submitProof with onProve

Removes duplication between Sales.onProve() and
Proving.onProofRequired()
2023-03-27 15:47:25 +02:00
markspanbroek 7a0a48e4a5
Fix warnings (drops Nim 1.2) (#348)
* [build] disable XCannotRaiseY hint

There are too many {.raises:[Defect].} in the
libraries that we use, drowning out all other
warnings and hints

* [build] disable BareExcept warning

Not yet enabled in a released version of Nim,
so libraries that we depend on have not fixed
this yet, drowning out our own hints and warnings

* [build] disable DotLikeOps warning

dot-like ops were an experiment that is not going
land in Nim

* [build] compile log statements in tests

When running tests, all log statements are compiled.
They are filtered out at runtime during a test run.

* [build] do not build executable when running unit test

It's already built in the integration test

* [build] Fix warnings

- remove unused code
- remove unused imports
- stop using deprecated stuff

* [build] Put compiler flags behind nim version checks

* [CI] remove Nim 1.2 compatibility
2023-03-09 12:23:45 +01:00
Ben Bierens 9c8a59d150
Blockstore maintenance (#347)
* setting up

* Implements timer utility

* applies async and cancellation to timer loop

* Sets up mocktimer and mockblockstore to set up first maintenance module test

* wip: first test that calls blockChecker

* wip: adding user type to timer callback

* Chronicles doesn't like type-arguments? Disables logging in timer module for now

* Implementing block check test for blockMaintainer module

* Sets up additional tests for blockmaintainer

* Removes generic from timer module. Implements numberOfBlocks per interval in blockMaintainer.

* Implements blockMaintainer

* Sets up tests for blockChecker

* Some comments by Mark

* Cleanup repostore tests

* Setting up the use of std/times for block TTL tracking

* repostore adds expiration timestamp

* Defaults the repostore clock to the system clock

* Applies updates to repostore interface.

* Implements retrieving of block expiration information from repostore

* Sets up tests for maintenance module behavior

* Implements block maintenance module

* Wires maintenance module into codex. Sets up integration tests for block expiration

* Sets up test for missing behavior: removing timestamp metadata on block delete

* Implements removing of expiration metadata in repoStore

* Fixes integration tests for block expiration

* Adds block expiration tests to integration test run

* Handled some comments by Dmitriy

* Review comment by Dmitriy: Removes seq[cid] from runBlockCheck

* Review comment by Dmitriy: Moves key formatting methods to keyutils.

* Review comment by Dmitriy: Encodes durations using chronos

* Fixes conversion of TTL type in conf.

* Review comments by Dmitriy

* Adds unit tests for keyUtils.

* Adds test coverage for exception in maintenance module
2023-03-08 16:04:54 +01:00