Merge branch 'master' into feature/codex-net-deployer

This commit is contained in:
benbierens 2023-07-12 14:40:14 +02:00
commit 512aeb79a0
No known key found for this signature in database
GPG Key ID: FE44815D96D0A1AA
2 changed files with 132 additions and 78 deletions

View File

@ -78,31 +78,7 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Docker - Build and export to Docker
uses: docker/build-push-action@v4
with:
context: .
file: ${{ env.DOCKER_FILE }}
platforms: ${{ env.PLATFORM }}
load: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
- name: Docker - Minify image
uses: kitabisa/docker-slim-action@v1
id: slim
env:
DSLIM_HTTP_PROBE: false
with:
target: ${{ steps.meta.outputs.tags }}
overwrite: true
- name: Docker - Show slim report
run: echo "${REPORT}" | jq -r
env:
REPORT: ${{ steps.slim.outputs.report }}
- name: Docker - Push to Docker registry
- name: Docker - Build and Push
uses: docker/build-push-action@v4
with:
context: .
@ -145,4 +121,3 @@ jobs:
inputs: ${{ env.TAGS }}
images: ${{ needs.build.outputs.tags-linux-amd64 }},${{ needs.build.outputs.tags-linux-arm64 }}
push: true

View File

@ -1,59 +1,138 @@
# Run tests with Docker in Kubernetes
We may [run tests localy](../LOCALSETUP.MD) using installed Dotnet and inside Kubernetes we may use a [prepared Docker images](https://hub.docker.com/r/codexstorage/cs-codex-dist-tests/tags).
We may [run tests on local](../LOCALSETUP.MD) or remote Kubernetes cluster. Local cluster flow uses direct access to nodes ports and this is why we introduced a different way to check services ports, for more information please see [Tests run modes](../../../issues/20). Configuration option `RUNNERLOCATION` is responsible for that.
Custom [entrypoint](docker-entrypoint.sh) will do the following
For local run it is easier to install .Net and run tests on Docker Desktop Kubernetes cluster. In case of remote run we do not expose services via Ingress Controller and we can't access cluster nodes, this is why we should run tests only inside the Kubernetes.
We can run tests on remote cluster in the following ways
#### Run pod inside the cluster using generic .Net image
<details>
<summary>steps</summary>
1. Create dist-tests-runner.yaml
```yaml
--
apiVersion: v1
kind: Pod
metadata:
name: dist-tests-runner
namespace: default
spec:
containers:
- name: dotnet
image: mcr.microsoft.com/dotnet/sdk:7.0
command: ["sleep", "infinity"]
```
2. Deploy pod in the cluster
```shell
kubectl apply -f dist-tests-runner.yaml
```
3. Copy kubeconfig to the pod
```shell
kubectl cp kubeconfig.yaml dist-tests-runner:/opt
```
4. Exec into the pod via kubectl or [OpenLens](https://github.com/MuhammedKalkan/OpenLens)
```shell
kubectl exec -it dist-tests-runner -- bash
```
5. Clone repository inside the pod
```shell
git clone https://github.com/codex-storage/cs-codex-dist-tests.git
```
6. Update kubeconfig option in config file
```shell
cd cs-codex-dist-tests
vi DistTestCore/Configuration.cs
```
```dotnet
GetNullableEnvVarOrDefault("KUBECONFIG", "/opt/kubeconfig.yaml")
```
7. Run tests
```shell
dotnet test Tests
```
8. Check the results and analyze the logs
</details>
#### Run pod inside the cluster using [prepared Docker images](https://hub.docker.com/r/codexstorage/cs-codex-dist-tests/tags)
Before the run we should create some objects inside the cluster
1. Namespace where we will run the image
2. [Service Account to run tests inside the cluster](https://github.com/codex-storage/cs-codex-dist-tests/issues/21)
3. Secret with kubeconfig for created SA
4. Configmap with custom app config if required
For more information please see [Manual run inside Kubernetes via Job](../../../issues/7)
Then we need to create a manifest to run the pod
<details>
<summary>runner.yaml</summary>
```yaml
---
apiVersion: v1
kind: Pod
metadata:
name: dist-tests-runner
namespace: cs-codex-dist-tests
labels:
name: cs-codex-dist-tests
spec:
containers:
- name: cs-codex-dist-tests
image: codexstorage/cs-codex-dist-tests:sha-671ee4e
env:
- name: RUNNERLOCATION
value: InternalToCluster
- name: KUBECONFIG
value: /opt/kubeconfig.yaml
- name: CONFIG
value: "/opt/Configuration.cs"
- name: CONFIG_SHOW
value: "true"
volumeMounts:
- name: kubeconfig
mountPath: /opt/kubeconfig.yaml
subPath: kubeconfig.yaml
- name: config
mountPath: /opt/Configuration.cs
subPath: Configuration.cs
- name: logs
mountPath: /var/log/cs-codex-dist-tests
# command:
# - "dotnet"
# - "test"
# - "Tests"
restartPolicy: Never
volumes:
- name: kubeconfig
secret:
secretName: cs-codex-dist-tests-app-kubeconfig
- name: config
configMap:
name: cs-codex-dist-tests
- name: logs
hostPath:
path: /var/log/cs-codex-dist-tests
```
For more information about pod variables please see [job.yaml](job.yaml).
</details>
And then apply it
```shell
kubectl apply -f runner.yaml
```
After the pod run, custom [entrypoint](docker-entrypoint.sh) will do the following
1. Clone repository
2. Switch to the specific branch - `master` by default
3. Run all tests - `dotnet test`
**Run with defaults**
```bash
docker run \
--rm \
--name cs-codex-dist-tests \
codexstorage/cs-codex-dist-tests:sha-686757e
```
**Just short tests**
```bash
docker run \
--rm \
--name cs-codex-dist-tests \
codexstorage/cs-codex-dist-tests:sha-686757e \
dotnet test Tests
```
**Custom branch**
```bash
docker run \
--rm \
--name cs-codex-dist-tests \
--env BRANCH=feature/tests \
codexstorage/cs-codex-dist-tests:sha-686757e
```
**Custom local config**
```bash
docker run \
--rm \
--name cs-codex-dist-tests \
--env CONFIG=/opt/Configuration.cs \
--env CONFIG_SHOW=true \
--volume $PWD/DistTestCore/Configuration.cs:/opt/Configuration.cs \
codexstorage/cs-codex-dist-tests:sha-686757e
```
**Local kubeconfig with custom local config**
```bash
docker run \
--rm \
--name cs-codex-dist-tests \
--env CONFIG=/opt/Configuration.cs \
--env CONFIG_SHOW=true \
--env SOURCE=https://github.com/codex-storage/cs-codex-dist-tests.git \
--volume $PWD/DistTestCore/Configuration.cs:/opt/Configuration.cs \
--volume $PWD/kubeconfig.yml:/opt/kubeconfig.yml \
codexstorage/cs-codex-dist-tests:sha-686757e
```