codex-storage
/
nim-codex
mirror of https://github.com/codex-storage/nim-codex.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
nim-codex/dagger.nimble

95 lines
2.8 KiB
Plaintext
Raw Normal View History

adding nimbus build system
2021-11-23 16:55:16 +00:00
mode = ScriptMode.Verbose
Poc 2 (#7) * moving protobuf into bitswap * adding block type * reworking bitswap * adding chunker * adding license header * use 1.2.6 * adding fixed size chunker * add blockstore * add iterator to chunker * more bitswap changes * rename ipfs to dagger * rename to dagger * blockstore inherits from BlockProvider * wip - add core block handling logic * smal changes * use proper block store methods * adding asynq heapqueue * wip prepare for bitswap task runner * adding `$` * adding memory store and tests * fixed chunking * extracted desicion engine from bitswap * added helper random funcs * adding testing helpers * only handle seqs * add peer events * cleanup pending blocks on blockstore event * allow nil handlers * move protobuf type helpers * allow initializing block from Cid * testing and fixes * small fixes * expose `<` * spelling * default value * spelling * pending blocks manager * adding stores manager * more tests a wip around bitswap * small changes * merge bitswap and engine for now * for now run only the new poc's tests * add a more complete ci setup * use template in map * remove p2pd * remove go * dont use asyncCheck * few small changes * adding ability to update items * adding multiple task runners * handle cancelation properly * use Result instead of throwing * wip bitswap tests * moving things around * split out engine again * add request and handlers interface * fix tests * wip - engine tests * remove unused imports * fix tests * cleanup block requesting logic * add block request tests * more block requests * add support for max heap * don't use result * use max heap & send block presence in task handler * add task handler tests * rename store to localStore * cleanup & logging * cancel task on stop * don't depend on local store for events * dont use heap queue for wants * add chronicles * fix issue with peer wants * add test for delayed block sends * remove obsolete tests * wip chunker * run all tests * add todo * misc * remove irrelevant files * removing more files * adding helpers for bitswap tests * moved bitswap file * misc * make blocks timeout longer * adjust block timeout * speedup test * compile with threads * import missing crypto * misc * disable threads for now * fix 32 bit platforms * re-enable threads support in tests
2021-02-26 00:23:22 +00:00
version = "0.1.0"
author = "Dagger Team"
fix compilation issues with latest libp2p (#27) * fix compilation issues with latest libp2p * change package description * fix CI compiler * try with nim 1.4.2 * try with 1.4.2 * use more standardized ci config * set reasonable log level * remove nim version dependency * reverting to 1.4.6 as otherwise it fails on amd64
2021-10-29 19:30:52 +00:00
description = "p2p data durability engine"
Poc 2 (#7) * moving protobuf into bitswap * adding block type * reworking bitswap * adding chunker * adding license header * use 1.2.6 * adding fixed size chunker * add blockstore * add iterator to chunker * more bitswap changes * rename ipfs to dagger * rename to dagger * blockstore inherits from BlockProvider * wip - add core block handling logic * smal changes * use proper block store methods * adding asynq heapqueue * wip prepare for bitswap task runner * adding `$` * adding memory store and tests * fixed chunking * extracted desicion engine from bitswap * added helper random funcs * adding testing helpers * only handle seqs * add peer events * cleanup pending blocks on blockstore event * allow nil handlers * move protobuf type helpers * allow initializing block from Cid * testing and fixes * small fixes * expose `<` * spelling * default value * spelling * pending blocks manager * adding stores manager * more tests a wip around bitswap * small changes * merge bitswap and engine for now * for now run only the new poc's tests * add a more complete ci setup * use template in map * remove p2pd * remove go * dont use asyncCheck * few small changes * adding ability to update items * adding multiple task runners * handle cancelation properly * use Result instead of throwing * wip bitswap tests * moving things around * split out engine again * add request and handlers interface * fix tests * wip - engine tests * remove unused imports * fix tests * cleanup block requesting logic * add block request tests * more block requests * add support for max heap * don't use result * use max heap & send block presence in task handler * add task handler tests * rename store to localStore * cleanup & logging * cancel task on stop * don't depend on local store for events * dont use heap queue for wants * add chronicles * fix issue with peer wants * add test for delayed block sends * remove obsolete tests * wip chunker * run all tests * add todo * misc * remove irrelevant files * removing more files * adding helpers for bitswap tests * moved bitswap file * misc * make blocks timeout longer * adjust block timeout * speedup test * compile with threads * import missing crypto * misc * disable threads for now * fix 32 bit platforms * re-enable threads support in tests
2021-02-26 00:23:22 +00:00
license = "MIT"
fix compilation issues with latest libp2p (#27) * fix compilation issues with latest libp2p * change package description * fix CI compiler * try with nim 1.4.2 * try with 1.4.2 * use more standardized ci config * set reasonable log level * remove nim version dependency * reverting to 1.4.6 as otherwise it fails on amd64
2021-10-29 19:30:52 +00:00
requires "libp2p#unstable",
Poc 2 (#7) * moving protobuf into bitswap * adding block type * reworking bitswap * adding chunker * adding license header * use 1.2.6 * adding fixed size chunker * add blockstore * add iterator to chunker * more bitswap changes * rename ipfs to dagger * rename to dagger * blockstore inherits from BlockProvider * wip - add core block handling logic * smal changes * use proper block store methods * adding asynq heapqueue * wip prepare for bitswap task runner * adding `$` * adding memory store and tests * fixed chunking * extracted desicion engine from bitswap * added helper random funcs * adding testing helpers * only handle seqs * add peer events * cleanup pending blocks on blockstore event * allow nil handlers * move protobuf type helpers * allow initializing block from Cid * testing and fixes * small fixes * expose `<` * spelling * default value * spelling * pending blocks manager * adding stores manager * more tests a wip around bitswap * small changes * merge bitswap and engine for now * for now run only the new poc's tests * add a more complete ci setup * use template in map * remove p2pd * remove go * dont use asyncCheck * few small changes * adding ability to update items * adding multiple task runners * handle cancelation properly * use Result instead of throwing * wip bitswap tests * moving things around * split out engine again * add request and handlers interface * fix tests * wip - engine tests * remove unused imports * fix tests * cleanup block requesting logic * add block request tests * more block requests * add support for max heap * don't use result * use max heap & send block presence in task handler * add task handler tests * rename store to localStore * cleanup & logging * cancel task on stop * don't depend on local store for events * dont use heap queue for wants * add chronicles * fix issue with peer wants * add test for delayed block sends * remove obsolete tests * wip chunker * run all tests * add todo * misc * remove irrelevant files * removing more files * adding helpers for bitswap tests * moved bitswap file * misc * make blocks timeout longer * adjust block timeout * speedup test * compile with threads * import missing crypto * misc * disable threads for now * fix 32 bit platforms * re-enable threads support in tests
2021-02-26 00:23:22 +00:00
"nimcrypto >= 0.4.1",
"bearssl >= 0.1.4",
"chronicles >= 0.7.2",
[wip] add `task localstore` to dagger.nimble
2021-11-17 20:21:52 +00:00
"https://github.com/michaelsbradleyjr/nim-chronos.git#export-selector-field",
Poc 2 (#7) * moving protobuf into bitswap * adding block type * reworking bitswap * adding chunker * adding license header * use 1.2.6 * adding fixed size chunker * add blockstore * add iterator to chunker * more bitswap changes * rename ipfs to dagger * rename to dagger * blockstore inherits from BlockProvider * wip - add core block handling logic * smal changes * use proper block store methods * adding asynq heapqueue * wip prepare for bitswap task runner * adding `$` * adding memory store and tests * fixed chunking * extracted desicion engine from bitswap * added helper random funcs * adding testing helpers * only handle seqs * add peer events * cleanup pending blocks on blockstore event * allow nil handlers * move protobuf type helpers * allow initializing block from Cid * testing and fixes * small fixes * expose `<` * spelling * default value * spelling * pending blocks manager * adding stores manager * more tests a wip around bitswap * small changes * merge bitswap and engine for now * for now run only the new poc's tests * add a more complete ci setup * use template in map * remove p2pd * remove go * dont use asyncCheck * few small changes * adding ability to update items * adding multiple task runners * handle cancelation properly * use Result instead of throwing * wip bitswap tests * moving things around * split out engine again * add request and handlers interface * fix tests * wip - engine tests * remove unused imports * fix tests * cleanup block requesting logic * add block request tests * more block requests * add support for max heap * don't use result * use max heap & send block presence in task handler * add task handler tests * rename store to localStore * cleanup & logging * cancel task on stop * don't depend on local store for events * dont use heap queue for wants * add chronicles * fix issue with peer wants * add test for delayed block sends * remove obsolete tests * wip chunker * run all tests * add todo * misc * remove irrelevant files * removing more files * adding helpers for bitswap tests * moved bitswap file * misc * make blocks timeout longer * adjust block timeout * speedup test * compile with threads * import missing crypto * misc * disable threads for now * fix 32 bit platforms * re-enable threads support in tests
2021-02-26 00:23:22 +00:00
"metrics",
"secp256k1",
"stew#head",
"protobufserialization >= 0.2.0 & < 0.3.0",
Update to latest versions of nitro and questionable
2021-04-15 09:23:49 +00:00
"https://github.com/status-im/nim-nitro >= 0.4.0 & < 0.5.0",
Update to version 0.9.1 of questionable
2021-05-10 07:42:38 +00:00
"questionable >= 0.9.1 & < 0.10.0",
Add package upraises
2021-04-07 10:22:02 +00:00
"upraises >= 0.1.0 & < 0.2.0",
[wip] add nim-task-runner dependency
2021-11-17 17:18:55 +00:00
"asynctest >= 0.3.0 & < 0.4.0",
"https://github.com/status-im/nim-task-runner.git#impl_beta2_nimble"
adding nimbus build system
2021-11-23 16:55:16 +00:00
### Helper functions
proc buildBinary(name: string, srcDir = "./", params = "", lang = "c") =
if not dirExists "build":
mkDir "build"
# allow something like "nim nimbus --verbosity:0 --hints:off nimbus.nims"
var extra_params = params
for i in 2..<paramCount():
extra_params &= " " & paramStr(i)
exec "nim " & lang & " --out:build/" & name & " " & extra_params & " " & srcDir & name & ".nim"
proc test(name: string, params = "-d:chronicles_log_level=DEBUG", lang = "c") =
buildBinary name, "tests/", params
exec "build/" & name
task testAll, "Build & run Waku v1 tests":
test "testAll", "-d:chronicles_log_level=WARN"
[wip] add `task localstore` to dagger.nimble
2021-11-17 20:21:52 +00:00
import os
[wip] stream writes to threads in task_runner threadpool It works but there are some strange behaviors that need be sorted out
2021-11-22 02:36:14 +00:00
task localstore, "Build localstore experiment":
const
build_opts =
when defined(danger) or defined(release):
(if defined(danger): " --define:danger" else: " --define:release") &
" --define:strip" &
" --hints:off" &
" --opt:size" &
" --passC:-flto" &
" --passL:-flto"
else:
" --debugger:native" &
" --define:chronicles_line_numbers" &
" --define:debug" &
" --linetrace:on" &
" --stacktrace:on"
[wip] add `task localstore` to dagger.nimble
2021-11-17 20:21:52 +00:00
[wip] stream writes to threads in task_runner threadpool It works but there are some strange behaviors that need be sorted out
2021-11-22 02:36:14 +00:00
common_opts =
" --define:ssl" &
[wip] compile in build dir
2021-12-02 22:20:49 +00:00
" --out:build/localstore" &
[wip] stream writes to threads in task_runner threadpool It works but there are some strange behaviors that need be sorted out
2021-11-22 02:36:14 +00:00
" --threads:on" &
" --tlsEmulation:off"
[wip] add `task localstore` to dagger.nimble
2021-11-17 20:21:52 +00:00
[wip] stream writes to threads in task_runner threadpool It works but there are some strange behaviors that need be sorted out
2021-11-22 02:36:14 +00:00
chronos_preferred =
" --path:\"" &
staticExec("nimble path chronos --silent").parentDir /
"chronos-#export-selector-field\""
chronicles_log_level {.strdefine.} =
when defined(danger) or defined(release):
"INFO"
else:
"DEBUG"
host {.strdefine.} = ""
maxRequestBodySize {.strdefine.} = ""
port {.strdefine.} = ""
[wip] add `task localstore` to dagger.nimble
2021-11-17 20:21:52 +00:00
var commands = [
"nim c" &
build_opts &
common_opts &
chronos_preferred &
[wip] stream writes to threads in task_runner threadpool It works but there are some strange behaviors that need be sorted out
2021-11-22 02:36:14 +00:00
" --define:chronicles_log_level=" & chronicles_log_level &
(when host != "": " --define:host=" & host else: "") &
(when maxRequestBodySize != "": " --define:maxRequestBodySize=" & maxRequestBodySize else: "") &
(when port != "": " --define:port=" & port else: "") &
[wip] add `task localstore` to dagger.nimble
2021-11-17 20:21:52 +00:00
" experiments/localstore.nim",
[wip] compile in build dir
2021-12-02 22:20:49 +00:00
"build/localstore"
[wip] add `task localstore` to dagger.nimble
2021-11-17 20:21:52 +00:00
]
[wip] stream writes to threads in task_runner threadpool It works but there are some strange behaviors that need be sorted out
2021-11-22 02:36:14 +00:00
[wip] add `task localstore` to dagger.nimble
2021-11-17 20:21:52 +00:00
for command in commands:
exec command