Source: https://api.octoperf.com/doc/on-premise-infra/tutorials/kubernetes-deployment/

# Deploy OctoPerf On-Premise Infra on Kubernetes

![Kubernetes](https://api.octoperf.com/doc/on-premise-infra/tutorials/img/kubernetes-logo.png)

This tutorial installs OctoPerf On-Premise Infra on a [Kubernetes](https://kubernetes.io) cluster with the [OctoPerf Helm charts](https://github.com/OctoPerf/helm-charts), then registers a Kubernetes agent that runs the load generators as pods of the same cluster. It uses [minikube](https://minikube.sigs.k8s.io/), a single-node cluster on your own computer, to try the charts before installing them on a real cluster.

## Prerequisites

This tutorial requires:

- a computer running Linux with at least `4` CPUs and `8 GB` of RAM available for the cluster,
- [Docker](https://docs.docker.com/engine/install/), used by minikube to run the Kubernetes node,
- [minikube](https://minikube.sigs.k8s.io/docs/start/),
- [kubectl](https://kubernetes.io/docs/tasks/tools/), to interact with the cluster,
- [Helm](https://helm.sh/docs/intro/install/) `3`, to install the charts,
- `git` and `make`.

It also supposes you know the basic [Kubernetes concepts](https://kubernetes.io/docs/concepts/): pods, deployments, services and ingresses.

## Overview

The [helm-charts](https://github.com/OctoPerf/helm-charts) repository provides two charts:

| Chart | What it installs |
|-------|------------------|
| `enterprise-edition` | The OctoPerf server: backend, web UI, MCP server, documentation and an Elasticsearch database |
| `kubernetes-agent` | A [Kubernetes agent](https://api.octoperf.com/doc/on-premise-agent/provider-type/on-premise/#kubernetes-agent): it connects to the server and starts each load generator as a pod of the cluster |

The ingresses of the `enterprise-edition` chart are served by a [Traefik v3](https://doc.traefik.io/traefik/) ingress controller. The minikube example of the chart installs it for you.

The tutorial goes through these steps:

1. Start a minikube cluster,
2. Install Traefik and OctoPerf with the minikube example of the `enterprise-edition` chart,
3. Create an account and an on-premise provider in the web UI,
4. Install the `kubernetes-agent` chart with the Helm command that OctoPerf gives,
5. Validate a Virtual User and launch a test on the agent.

## Start Minikube

Start a cluster with enough resources for OctoPerf and its load generators, for example `8` CPUs and `16 GB` of memory:

```bash
minikube start --driver=docker --cpus=8 --memory=16g
```

Then get the IP of the minikube node:

```bash
minikube ip
192.168.49.2
```

With the docker driver, the IP is `192.168.49.2`. The minikube example of the chart reaches OctoPerf on `192.168.49.2.sslip.io`, a name that the public [sslip.io](https://sslip.io/) DNS service resolves to `192.168.49.2`. Your browser, the agent and the other pods of the cluster all use this address.

!!! info
    If `minikube ip` returns another IP, replace `192.168.49.2` with it in the `values.yaml` file of the example (see [Install OctoPerf](#install-octoperf)).

The `ingress` addon of minikube installs another ingress controller (ingress-nginx). Make sure it is disabled:

```bash
minikube addons disable ingress
```

## Install OctoPerf {#install-octoperf}

Clone the helm-charts repository and go to the minikube example of the `enterprise-edition` chart:

```bash
git clone https://github.com/OctoPerf/helm-charts.git
cd helm-charts/enterprise-edition/examples/minikube
make install
```

`make install` runs two steps:

1. It enables the `default-storageclass` and `storage-provisioner` addons of minikube, for the persistent volume of Elasticsearch, and installs Traefik with its official Helm chart and the settings of `traefik-values.yaml`. Traefik listens on port `80` of the minikube node,
2. It installs the `enterprise-edition` chart in the `octoperf` namespace, with the settings of `values.yaml`.

The `values.yaml` of the example sets the public address of the server and the Elasticsearch connection:

```yaml
mcpServer:
  enabled: true
  applicationYml: |-
    server:
      hostname: 192.168.49.2.sslip.io
      public:
        port: 80

backend:
  applicationYml: |-
    server:
      hostname: 192.168.49.2.sslip.io
      public:
        port: 80
    elasticsearch:
      hostname: elasticsearch-master-headless
      # Must match elasticsearch password
      password: "0ct0P3rf"

ingress:
  enabled: true
  hosts:
    - 192.168.49.2.sslip.io
```

The `applicationYml` blocks are the [application.yml](https://api.octoperf.com/doc/on-premise-infra/configuration/#custom-application-yml) of the backend and of the MCP server: any [setting of the server](https://api.octoperf.com/doc/on-premise-infra/configuration/) can be added there. The example also disables the documentation and the utility server, and gives Elasticsearch a single node with a small volume.

The first start takes a few minutes, while the cluster downloads the images. Check the pods of the `octoperf` namespace:

```bash
kubectl -n octoperf get pods
NAME                                            READY   STATUS    RESTARTS   AGE
elasticsearch-master-0                          1/1     Running   0          3m
enterprise-edition-backend-0                    1/1     Running   1          3m
enterprise-edition-frontend-7fb545c9c7-rdc6s    1/1     Running   0          3m
enterprise-edition-mcp-server-b57668fd9-m85wd   1/1     Running   0          2m
```

The backend may restart once and the MCP server a few times while Elasticsearch and the backend start: this is expected. Once all the pods are ready, open `http://192.168.49.2.sslip.io/ui` in your web browser.

!!! info
    The configuration of this example is meant for a test on minikube. The [README of the enterprise-edition chart](https://github.com/OctoPerf/helm-charts/tree/master/enterprise-edition#readme) lists all the settings, see also [Going further](#going-further).

## Create an Account and a Provider

OctoPerf On-Premise Infra starts empty. To run a test, you need an account and load generators:

1. [Sign up](https://api.octoperf.com/doc/account/#sign-up) to create an account. The server comes with a [default license](https://api.octoperf.com/doc/on-premise-infra/license/) for 50 concurrent virtual users,
2. Open **Workspace > On Premises** and [create an On-Premise provider](https://api.octoperf.com/doc/on-premise-agent/providers/#create-a-new-provider), for example named `Minikube`, with its `default` location,
3. At the end of the wizard, open the **Add Agent** dialog, select the location, go to the *Installation* step and choose **Kubernetes**.

![Kubernetes choice of Add Agent](https://api.octoperf.com/doc/on-premise-agent/provider-type/img/host-install-kubernetes.png "The Kubernetes choice of the Installation step"){ width="750" }

The token is masked on the screen, the command copied with **Copy Command** holds it. The token identifies the provider, the location and the name of the agent.

## Install the Kubernetes Agent

In a terminal, run the two commands of the *Installation* step:

1. **Check Helm 3 or later and access to the cluster**: it prints the version of Helm and the address of the minikube cluster,
2. **Install OctoPerf agent**: it adds the OctoPerf chart repository, then installs the `kubernetes-agent` chart in the `octoperf` namespace, next to OctoPerf. Its server URL is the address of the minikube example, `http://192.168.49.2.sslip.io`.

The copied command looks like this:

```bash
helm repo add octoperf https://helm.octoperf.com --force-update
helm install red-john-smith octoperf/kubernetes-agent \
  --namespace octoperf --create-namespace \
  --set token='<token>' \
  --set serverUrl='http://192.168.49.2.sslip.io'
```

The agent appears in the *Agents* table of the **Providers** page after a few seconds. Click **Refresh Agents** if it does not:

![Kubernetes agent](https://api.octoperf.com/doc/on-premise-infra/tutorials/img/kubernetes/kubernetes-agent.png "The Kubernetes agent in the default location of the Minikube provider")

!!! info
    With the docker driver of minikube, the agent reports the memory of the host computer, not the memory given to minikube. The load generators are sized with the *Amount of memory per agent in MB* of the provider (see [Memory configuration](https://api.octoperf.com/doc/on-premise-agent/providers/#configuration)): keep it below the memory of the minikube node.

## Run a Test

The agent is now ready to run load generators:

1. [Validate a Virtual User](https://api.octoperf.com/doc/design/edit-virtual-user/validation/) in the location of the provider: the agent starts a load generator pod, runs one iteration and deletes the pod,
2. Create a scenario that uses this location, then launch it. The [scenario check](https://api.octoperf.com/doc/runtime/edit-scenario/scenario-check/) of the launch wizard tells whether the agents have enough capacity for the load.

During the test, the load generator runs as a pod named after the agent, next to the agent pod:

```bash
kubectl -n octoperf get pods
NAME                                READY   STATUS    RESTARTS   AGE
enterprise-edition-backend-0        1/1     Running   0          6m
kubernetes-agent-684bfcd869-vk9gl   1/1     Running   0          44m
yellow-curt-johns-trev              1/1     Running   0          26s
```

The agent deletes the load generator pods at the end of the test.

## Going Further {#going-further}

Before installing the charts on a production cluster, review these settings of the [enterprise-edition chart](https://github.com/OctoPerf/helm-charts/tree/master/enterprise-edition#readme):

- the public hostname, in `ingress.hosts` and in the `applicationYml` of the backend and of the MCP server, and TLS on the Traefik side,
- the request read timeout of Traefik, `60s` by default, to raise for the upload of large JMeter projects and CSV files,
- Elasticsearch: the cluster of the chart with several nodes and persistent volumes, or an existing cluster with `elasticsearch.enabled: false`,
- your [license](https://api.octoperf.com/doc/on-premise-infra/license/) file, under `backend.licenseFile`.

For the agents:

- install one Kubernetes agent per node that runs load generators: an agent only starts pods on its own node. From the chart `18.0.1`, several agents share a cluster, each one on its own node. Reopen the **Add Agent** dialog for each agent: a command run twice gives two agents the same name,
- uninstall an agent with `helm uninstall <agent name> --namespace octoperf`, see [Stop an agent](https://api.octoperf.com/doc/on-premise-agent/provider-type/on-premise/#stop-an-agent),
- set the resources, tolerations, image pull secrets and security context of the load generator pods in the values of the [kubernetes-agent chart](https://github.com/OctoPerf/helm-charts/tree/master/kubernetes-agent#readme), see [Kubernetes agent](https://api.octoperf.com/doc/on-premise-agent/provider-type/on-premise/#kubernetes-agent).

When upgrading from an earlier version, read the [upgrade procedure](https://api.octoperf.com/doc/on-premise-infra/upgrade/) and the [migration guide](https://api.octoperf.com/doc/on-premise-infra/migration/17.1.x-to-18.0.0/): from `18.0.0`, the chart uses Traefik instead of ingress-nginx, and the agents must be installed again with a new token.

To remove the test installation, delete the minikube cluster:

```bash
minikube delete
```