Skip to content

On Premise Load Testing

Architecture

Architecture

OctoPerf supports using your own computers to run load tests. You can also configure an On-Premise provider to run tests behind your firewall. This is useful to do performance tests on a development environment or on any application that is not available on the Internet.

On-Premise

An OctoPerf On-Premise agent is a docker container installed on a server of your choice. The OctoPerf agent in itself will not execute the test but instead he will create Docker containers for each runtime profile he has to execute. In that way it works similarly to other orchestrators like Docker swarm or Kubernetes but with a much simpler approach dedicated to load testing through OctoPerf:

agent

The perks of this agent is that a single one can execute several tests at once, provided there are enough resources available on the machine. The resources requirement for a test is computed based on available memory on the machine and the memory settings of the provider.

System Requirements

For an optimal load testing experience, we recommend the following configuration for each on-premise agent:

  • CPU: 4 or more vCPUs (Intel Core i7, Intel Xeon or AMD Ryzen 7 recommended),
  • RAM: 8GB+ RAM (16GB recommended),
  • Disk: 16GB+ free hard-disk space (SSD preferred)
  • Operating system: Linux, Ubuntu (most recent LTS). With latest Docker version,
  • Server: both physical servers and virtual machines are supported, (physical servers strongly recommended)
  • Software: Latest version of Docker CE,
  • Network: Have a constant access over HTTPS to https://api.octoperf.com.

Info

While it is possible to use larger machines to run more load, you may run into other issues like the docker engine not being able to create containers fast enough, large garbage collections or network contentions. Using several smaller machines can be a better solution in such situations. It allows concurrent tests to stay independent by using different hardware.

Cloud Providers

Introduction

Instead of running your own nodes, you can connect cloud providers to OctoPerf to benefit from:

  • On-Demand Load Generators: when starting a new test, OctoPerf provisions cloud instances according to your needs,
  • Automated Test Run: provided the cloud instance can run Docker commands, OctoPerf manages the entire test lifecycle remotely,
  • Automatic Shutdown: once no longer required, cloud instances are automatically terminated (so you get billed only for the hours you use them).

The following cloud providers are supported:

Given that most cloud providers bill instances on an hourly rate, one can expect to make significant savings over running your own on-premise grid. That being said, cloud providers require some configuration in order to work.

See Providers for more information.

Info

You can mix both Cloud and On-Premise load generators within the same test.

Auto-scale process

Amazon

  1. The OctoPerf Server (either SaaS or On-Premise Infra) contacts the Cloud API to ask for machines to be started
  2. The Cloud API acknowledges the request for machines and states if it is able to provide them. It then proceeds to start them.
  3. Once the OctoPerf Agents are running, they report their status to the OctoPerf Server. Every once in a while they will connect back to check if they have a greenlight to proceed with the test. All communication is done through outgoing HTTP/S from the agents. Note that after 5 min without response, the instance is rebooted. After 10 min without response, the instance is terminated and another one is started.
  4. Once the last agents is up and running the test is greenlighted and load is generated from the agents as defined in the runtime configuration. Metrics are reported live through outgoing HTTP/S from the agents to the OctoPerf server.

Glossary

  • Load Generator: virtual or physical computer with CPU and memory used to generate the load on your target web application. Those machines run the JMeter containers,
  • Container: Designates a Docker Container. A container image is a lightweight, stand-alone, executable package of a piece of software that includes everything needed to run it,
  • Agent: an agent is a docker container which communicates with OctoPerf servers to monitor your system and starts / stops the JMeter containers.

We chose Docker to run the agents because it provides numerous advantages:

  • Easy installation: run an agent container within minutes,
  • Portability: the container contains all the necessary packages and configuration to run the agent on any machine with Docker installed,
  • Version control: we can manage several agent versions and upgrades are seamless,
  • Lightweight footprint: docker containers are usually small and docker has minimal overhead compared to VMs.