gmega
897fd926ad
feat: generalize speedup analysis to multiple groups
2025-11-10 11:02:29 -03:00
gmega
d3a1bef683
feat: generalize final analysis (except speedups) to multiple groups
2025-11-10 08:46:10 -03:00
Chrysostomos Nanakos
0a68259dc7
fix(argo): ensure vector scales up even on workflow failure
...
Signed-off-by: Chrysostomos Nanakos <chris@include.gr>
2025-10-31 13:16:58 +02:00
Chrysostomos Nanakos
b59ddaf648
feat(k8s): add DO (Digital Ocean) Kubernetes cluster terraform
...
Adds Terraform configuration to provision a DO K8s cluster with
auto-scaling node pools for running Codex benchmarks.
Signed-off-by: Chrysostomos Nanakos <chris@include.gr>
2025-10-21 18:17:26 +03:00
Chrysostomos Nanakos
200c749cb5
feat(workflows): add Vector log parsing workflow template and synchronization
...
Add workflow template for parsing logs collected by Vector from Kubernetes
pods, with semaphore synchronization to prevent concurrent access conflicts.
- log-parsing-workflow-template-vector: New workflow template that scales
down Vector aggregator to access RWO PVC, parses JSONL logs, then scales
aggregator back up
- vector-log-parsing-semaphore: ConfigMap semaphore limiting to one log
parsing workflow at a time (prevents RWO PVC mount conflicts)
- codex-workflows-rbac: Added deployment get/patch/update permissions to
executor role (required for scaling Vector aggregator)
Signed-off-by: Chrysostomos Nanakos <chris@include.gr>
2025-10-21 13:25:00 +03:00
Chrysostomos Nanakos
0c8e28fa46
feat(k8s): add Vector logging infrastructure for benchmarks
...
Add Vector agent/aggregator deployment for collecting logs from Codex
benchmark experiments in K8s. Includes PVC for log storage, S3 secret
template and RBAC.
Vector collects logs from benchmark pods and writes JSONL files for
post-processing by the log-parsing workflow.
Signed-off-by: Chrysostomos Nanakos <chris@include.gr>
2025-10-21 13:13:49 +03:00
Chrysostomos Nanakos
8d11207e73
fix(k8s): make codex-node memory resources conditional
...
Make the resources block for codex-node container conditional on
experiment.memory being set and non-empty. If codexMemory is not
provided in the workflow parameters or is set to an empty string,
no resource limits will be set on the pod.
Signed-off-by: Chrysostomos Nanakos <chris@include.gr>
2025-10-20 11:46:37 +03:00
Giuliano Mega
884b16d8c5
Merge pull request #8 from codex-storage/fix-header-assertion
...
fix: Use startswith for Content-Type header assertion
2025-10-07 15:15:17 -03:00
Chrysostomos Nanakos
162a8f4a54
fix: Use startswith for Content-Type header assertion
...
Signed-off-by: Chrysostomos Nanakos <chris@include.gr>
2025-10-07 11:13:02 +03:00
Chrysostomos Nanakos
b1e4bda940
fix: switch to bitnamilegacy kubectl image
...
The bitnami/kubectl images have been moved to the legacy repository.
Signed-off-by: Chrysostomos Nanakos <chris@include.gr>
2025-10-01 15:45:16 +03:00
gmega
04828514e4
feat: port bug fixes and features from swarm branch
2025-06-09 20:06:19 -03:00
gmega
4cbb401d12
feat: add optional data removal with adjusted quotas
2025-06-09 19:59:00 -03:00
gmega
cd77b20a82
feat: add in-place merging for properly sorting pod logs when dumping from vector flat files
2025-04-24 12:00:07 -03:00
gmega
223276f7c8
feat: allow image names in compose to be overridden by env vars
2025-04-16 14:09:08 -03:00
gmega
67ca362ee7
misc: minor refactor, add simple network perf test deploy
2025-04-16 12:56:34 -03:00
gmega
e5fa036e98
feat: add link to analysis in README
2025-02-27 19:00:49 -03:00
gmega
b2491c26f9
fix: fix workflow expressions
2025-02-27 18:49:48 -03:00
gmega
81cda58a9d
feat: add download speed plot, dedup experiment datasets
2025-02-27 18:47:36 -03:00
gmega
a366f04e7c
feat: allow re-running failed experiments from previous workflow runs
2025-02-25 12:14:15 -03:00
gmega
2061fe6dbe
feat: add supporting scripts to run workflow retries
2025-02-25 11:00:12 -03:00
gmega
5a9543259b
feat: add support for region k8s annotations
2025-02-24 14:16:59 -03:00
gmega
8dbc3faed8
feat: add tunable parallelism
2025-02-23 11:33:49 -03:00
gmega
73219922f6
feat: add Codex chart values for cluster experiments
2025-02-20 12:16:05 -03:00
gmega
48e71a315a
feat: add support for setting the node tag in benchmark workflow
2025-02-20 12:14:49 -03:00
gmega
688091c965
feat: allow use of custom runner and node tags for Codex
2025-02-20 11:59:24 -03:00
gmega
a8c19364b7
fix: minikube env param in workflow
2025-02-20 10:21:45 -03:00
gmega
63501c3b79
fix: simplify and fix adaptation of models to log entries
2025-02-20 09:42:23 -03:00
gmega
1ab2112542
fix: logging for adapted entries should include superclasses
2025-02-19 18:16:26 -03:00
gmega
9563a96373
feat: add explicit logging for experiment stage boundaries
2025-02-19 14:00:27 -03:00
gmega
242097abd3
chore: reify config schema standardization for static dissemination experiments
2025-02-19 12:14:11 -03:00
gmega
ad6e94db7d
feat: generalize final analysis to Codex experiments
2025-02-18 17:58:13 -03:00
gmega
d1077d0972
feat: allow reading multiple Vector files when splitting logs
2025-02-18 15:59:11 -03:00
gmega
e47f8848e4
fix: implement correct timeout behavior for Codex streaming downloads
2025-02-18 15:41:29 -03:00
gmega
a0e4181123
feat: add explicit experiment type to experiment config to ease analysis
2025-02-17 11:28:54 -03:00
gmega
0d08814929
feat: generalize benchmark workflow to run Codex in addition to Deluge
2025-02-17 10:44:00 -03:00
gmega
4c7f1a4899
feat: generalize single-experiment download analysis notebook
2025-02-17 09:41:39 -03:00
gmega
95651ad9d7
feat: add download metric unit to experiment config to ease analysis
2025-02-14 18:37:05 -03:00
gmega
0530b4a10b
feat: standardize download metric
2025-02-14 18:19:45 -03:00
gmega
38434f4590
fix container label for codex experiment runner
2025-02-14 15:59:59 -03:00
gmega
e8441b7bea
fix: respect logger increments even when stream returns less data than expected
2025-02-14 15:59:28 -03:00
gmega
f7adf878eb
feat: add memory parameter to Deluge values file
2025-02-14 14:30:56 -03:00
gmega
205f926f89
feat: add stable bootstrap node
2025-02-14 14:30:18 -03:00
gmega
f336df8da7
fix: adjust Codex logging cooldown, insert polling backoff on download completion, define default Codex experiment
2025-02-14 12:14:52 -03:00
gmega
68ee1bad87
feat: add working Codex helm chart
2025-02-14 11:00:17 -03:00
gmega
6681922e00
feat: add Codex static dissemination experiment
2025-02-12 19:21:34 -03:00
gmega
84f5e1437f
chore: delete redundant test
2025-02-04 19:25:26 -03:00
gmega
74ee71889e
feat: add Codex node and initial integration tests
2025-02-04 19:18:58 -03:00
gmega
820699f001
feat: add Codex agent REST API
2025-02-03 18:00:43 -03:00
gmega
849bcad6c8
refactor: simplify download progress reporting
2025-02-03 16:45:48 -03:00
gmega
bd0ef9ca55
refactor: move Codex agent into its own subpackage
2025-02-03 15:53:15 -03:00
