200c749cb5
Add workflow template for parsing logs collected by Vector from Kubernetes pods, with semaphore synchronization to prevent concurrent access conflicts. - log-parsing-workflow-template-vector: New workflow template that scales down Vector aggregator to access RWO PVC, parses JSONL logs, then scales aggregator back up - vector-log-parsing-semaphore: ConfigMap semaphore limiting to one log parsing workflow at a time (prevents RWO PVC mount conflicts) - codex-workflows-rbac: Added deployment get/patch/update permissions to executor role (required for scaling Vector aggregator) Signed-off-by: Chrysostomos Nanakos <chris@include.gr>
This folder contains the required Kubernetes and Argo Workflow resources required to run experiments in Kubernetes both in local (e.g. Minikube, Kind) and remote clusters.
Prerequisites
Argo Workflows
Whatever cluster you choose must be running Argo Workflows.
Local clusters. For local clusters, you can follow the instructions in the Argo Workflows Quickstart Guide to get Argo Workflows running.
For remote clusters, it's best to consult the Argo Workflows Operator Manual.
Argo CLI Tool. You will also need to install the Argo CLI tool to submit workflows.
Permissions. Codex workflows assume that they are running in a namespace called codex-benchmarks. We
have a sample manifest which creates the namespace as well as the proper service account with RBAC
permissions here. For local clusters, you can apply this manifest
as it is. For remote clusters, you might need to customize it to your needs.
Logs
Experiments require logs to be stored for later parsing during analysis. For local clusters, this can be achieved by running Vector and outputting pods logs to a persistent volume. The manifests for setting the persistent volume, as well as vector, can be found here.
Submitting Workflows
Once everything is set up, workflows can be submitted with:
argo submit -n argo ./deluge-benchmark-workflow.yaml
for local clusters, you should add:
argo submit -n argo ./deluge-benchmark-workflow.yaml --insecure-skip-verify
To observe progress, you can use the Argo Wokflows UI which can be accessed by port-forwarding the Argo Workflows server.