Skip to content

REST API changes: 17.1.0 to 18.0.0

This page lists the changes to the OctoPerf REST API between 17.1.0 and 18.0.0. It is written for the people who call the API directly - CI/CD pipelines, custom reporting jobs, in-house dashboards. If you only use the web interface, the Maven plugin or the MCP server, you have nothing to do: they were updated with the platform.

If you are upgrading from 17.0.1 or earlier, read REST API changes: 17.0.1 to 17.1.0 first, then this page.

18.0.0 brings K6 as a new load engine, and checks every scenario before it runs. Two changes require an update on your side: running a Virtual User validation takes a new body, and a scenario with an error in its scenario check can no longer be launched. Two provider capacity endpoints are removed. Everything else is additive, but new enum values show in the responses: a client that refuses unknown enum values must learn them.


1. TL;DR: what changes

# Change Impact
1 POST /runtime/virtual-user-checks/{projectId}/{virtualUserId} takes a VirtualUserCheckSettings body; the route with {providerId}/{location} is removed Update your validation calls
2 New GET and DELETE /runtime/virtual-user-check-settings/{projectId}/{virtualUserId} Additive
3 New POST /design/imports/k6/{projectId} Additive
4 New POST /runtime/scenarios/k6/{projectId} Additive
5 PUT /design/virtual-users/sample/{projectId} accepts ?type=K6 and ?type=PLAYWRIGHT Additive
6 The Virtual User exports accept K6 Virtual Users Additive
7 TableEntry gains sampleType Additive
8 HttpResponseEntity gains failed Additive
9 New types and enum values: K6, K6ScriptAction, K6UserProfileEngine, CUSTOM, K6_RUNTIME... Check strict enum parsing
10 New GET /design/sla-profiles/threshold-defaults/custom-metrics; an SLA threshold can target a CUSTOM metric Additive
11 Launching a scenario with an error answers 400, from every launch route and the scheduler Handle the refusal
12 New GET /runtime/scenarios/sanity-check/{scenarioId} Additive
13 GET /runtime/scenarios/memory-usage/{scenarioId} and POST /workspaces/docker-providers/usage/all are removed Use the sanity check instead
14 New GET /public/docker/regions, unauthenticated Additive
15 New GET /workspaces/docker-providers/sanity-check/by-workspace/{workspaceId} and GET /workspaces/docker-providers/sanity-check/{workspaceId}/{providerId} Additive

2. Running a Virtual User validation

A validation now runs with the settings of its own, the ones the validation panel shows: provider, location, iterations and the engine settings of a user profile. They are saved as the validation settings of the Virtual User at each run.

Removed:

  • POST /runtime/virtual-user-checks/{projectId}/{virtualUserId}/{providerId}/{location}?iterations=, with a BrowserSettings body,
  • the UserProfile body and the iterations query parameter of POST /runtime/virtual-user-checks/{projectId}/{virtualUserId}.

Now:

POST /runtime/virtual-user-checks/{projectId}/{virtualUserId}
Content-Type: application/json

{
  "projectId": "<projectId>",
  "virtualUserId": "<virtualUserId>",
  "providerId": "<providerId>",
  "location": "<location>",
  "iterations": 1,
  "engine": { "@type": "JmeterUserProfileEngine", ... }
}
  • projectId and virtualUserId must be those of the path: a mismatch answers 400.
  • iterations moves from the query to the body.
  • engine is the engine of a user profile: JmeterUserProfileEngine, PlaywrightUserProfileEngine, SeleniumUserProfileEngine or K6UserProfileEngine, matching the type of the Virtual User.
  • The response is still the BenchResult of the validation.

What to do. Read the saved settings first (§3), change what you need, and post them back:

curl -s -H "Authorization: Bearer ${TOKEN}" \
  "https://api.octoperf.com/runtime/virtual-user-check-settings/${PROJECT_ID}/${VU_ID}" > settings.json
curl -s -X POST -H "Authorization: Bearer ${TOKEN}" -H "Content-Type: application/json" \
  --data @settings.json \
  "https://api.octoperf.com/runtime/virtual-user-checks/${PROJECT_ID}/${VU_ID}"

3. Validation settings

Method Path Does
GET /runtime/virtual-user-check-settings/{projectId}/{virtualUserId} Returns the saved VirtualUserCheckSettings of the Virtual User, or the defaults of its type when none were saved
DELETE /runtime/virtual-user-check-settings/{projectId}/{virtualUserId} Forgets the saved settings: the next GET returns the defaults

There is no POST or PUT: running a validation saves its settings.


4. K6

Import K6 scripts

POST /design/imports/k6/{projectId}
Content-Type: multipart/form-data

One files part per script, .js or .ts, at least one. The first file is the entrypoint. The response is the created VirtualUser, of type K6. See K6 import.

Create a scenario from the K6 options

POST /runtime/scenarios/k6/{projectId}
Content-Type: application/json

{
  "virtualUserId": "<virtualUserId>",
  "entrypointFileName": "script.js",
  "providerId": "<providerId>",
  "location": "<location>",
  "name": "Scenario",
  "options": { "scenarios": { ... } }
}
  • options is the JSON output of k6 inspect --execution-requirements <entrypoint>.
  • entrypointFileName and name are optional.
  • The response is a K6ScenarioConversion: scenario, absent when no K6 scenario could be converted, and unconverted, one {scenario, kind, outcome} entry for each part of the load that did not convert as it is. See the conversion rules.

Samples and exports

  • PUT /design/virtual-users/sample/{projectId} takes an optional type, JMETER (the default), K6 or PLAYWRIGHT, to create the sample K6 or Playwright Virtual User. The Playwright sample is attached to the project's Core Web Vitals SLA profile, created when the project has none.
  • GET /design/exports/{virtualUserId} and GET /design/exports/all/{projectId}?vuType=K6 export K6 Virtual Users, as a ZIP file of their scripts.

5. Scenario check

Every scenario is checked before it runs: the scenario check looks for the settings that would make the test fail, such as a missing provider, a location with no agent up, a load that never ends or a server the load generators cannot reach.

Launching a scenario with an error answers 400

POST /runtime/scenarios/run/{scenarioId}, and every other route that starts a test, refuse a scenario that has an ERROR in its check. The refusal comes before any credit is consumed, answers 400, and its message names the user profile and each blocking problem, separated by ;:

Scenario cannot run: Buyer in Paris: No agent running in this location

The scheduler refuses such a scenario the same way: the run is recorded with this message. WARNING and INFO problems never block a launch.

A scenario that none of your subscriptions can run is still refused as before, with No valid subscription found!. The sanity check reports it too, as the rule SUBSCRIPTION_MISSING, with what stops each subscription.

What to do. Treat a 400 on a launch as a configuration problem of the scenario, not as a transient error: retrying gives the same answer. Show the message, and fix the scenario in the web interface. To fail early, call the sanity check (below) before launching.

Checking a scenario

New:

GET /runtime/scenarios/sanity-check/{scenarioId}

It runs nothing and consumes no credit. It answers the problems of the scenario, from ERROR to INFO, an empty list when the scenario can run as configured:

[
  {
    "@type": "ScenarioError",
    "id": "<id of the entity at fault>",
    "type": "DockerProviderRegion",
    "level": "ERROR",
    "errorMsg": "No agent running in this location",
    "userProfileId": "<userProfileId>",
    "userProfileName": "Buyer in Paris",
    "rule": "AGENT_MISSING",
    "details": "No agent is up in the location 'Paris' of the provider 'My Agents', so the test cannot start its load generators.\n\n**To fix it:**\n- ..."
  }
]
  • level: ERROR blocks the launch, WARNING and INFO do not.
  • errorMsg: the title the web interface shows. It may be reworded between versions: do not parse it.
  • rule: the rule the problem comes from, stable across versions. Each rule is explained on the Scenario Check page, at the rule in lowercase with dashes: AGENT_MISSING is #agent-missing.
  • details: Markdown, the problem with the figures of the scenario, then the list of fixes.
  • userProfileId and userProfileName: the user profile to fix.

Checking the on-premise providers

New:

GET /workspaces/docker-providers/sanity-check/by-workspace/{workspaceId}
GET /workspaces/docker-providers/sanity-check/{workspaceId}/{providerId}

The first checks every on-premise provider of the workspace, the second a single one. They answer the problems of the agents, an empty list when there is none:

[
  {
    "@type": "ProviderError",
    "id": "<location>",
    "type": "DockerProviderRegion",
    "level": "WARNING",
    "errorMsg": "Agents sharing a machine",
    "providerId": "<providerId>",
    "agentIds": ["<agentId>", "<agentId>"],
    "rule": "AGENTS_SHARING_A_MACHINE",
    "details": "**silver_devon_herman** and **tan_chloe_shields** run on the same machine. ...\n\n**To fix it:**\n- ..."
  }
]
  • rule: AGENTS_SHARING_A_MACHINE or MEMORY_UNDER_DECLARED, the same rules as in the sanity check of a scenario.
  • agentIds: the agents the problem concerns, those of the machine, or those of the location for a memory declared too low.
  • level, errorMsg and details: as for a scenario.

Provider capacity endpoints removed

Removed:

  • GET /runtime/scenarios/memory-usage/{scenarioId},
  • POST /workspaces/docker-providers/usage/all.

What to do. The capacity of on-premise providers is now part of the scenario check: the rules AGENT_MISSING, CAPACITY_EXCEEDED and CAPACITY_BUSY say whether the agents of a location can hold the load of the scenario, computed by placing its load generators on the agents the way a test does. Call GET /runtime/scenarios/sanity-check/{scenarioId} instead.

Region names

New:

GET /public/docker/regions

It needs no authentication, and answers the cloud regions as a GeoJSON FeatureCollection: each feature carries the id of a region (eu-west-3), its name (Paris), its longName (EU West (Paris)) and its coordinates. The messages of the scenario check name locations this way. A few region names are corrected, for instance centralus is US Central (Iowa) and westcentralus is US West Central (Wyoming).


6. Additive changes

TableEntry.sampleType

The rows of a statistic table carry the sample type they were computed from, CUSTOM for a custom metric:

{ "actionPath": "...", "sampleType": "BASIC", "values": { ... } }

HttpResponseEntity.failed

The responses of the validation and error exchanges carry failed, true when the engine counted the exchange as an error, a failed assertion on a 200 response included: the status alone cannot tell. false on a response stored before 18.0.0.

SLA thresholds on custom metrics

An SLAMonitorThreshold can target a custom metric: a CUSTOM_* metric of type CUSTOM, narrowed to one name by a metricName filter, its unit given by a UnitReportConfig. Such a threshold is evaluated on the whole Virtual User, when its SLAProfileAction sits at the root of the Virtual User.

{
  "@type": "SLAMonitorThreshold",
  "metric": {
    "id": "CUSTOM_AVG", "type": "CUSTOM", "benchResultId": "",
    "filters": [{"@type": "SingleTermFilter", "field": "metricName", "term": "LCP"}],
    "configs": [{"@type": "UnitReportConfig", "unit": "TIME"}]
  },
  "thresholds": []
}

GET /design/sla-profiles/threshold-defaults/custom-metrics returns the default thresholds of the custom metrics whose meaning is known, keyed by name: the Core Web Vitals LCP, INP, CLS, FCP and TTFB, on Google's bands.

New types and enum values

Where New value Meaning
VirtualUserType K6 A Virtual User made of K6 scripts
Action @type K6ScriptAction {id, name, enabled, fileName, script}: one file of a K6 Virtual User
UserProfile.engine @type K6UserProfileEngine The K6 settings of a user profile ; abortOnFail, false when absent, lets a threshold declaring abortOnFail stop K6 on its load generator
SampleType CUSTOM The custom metrics of the scripts
MetricId CUSTOM_AVG, CUSTOM_MIN, CUSTOM_MAX, CUSTOM_SUM, CUSTOM_COUNT, CUSTOM_RATE The statistics of a custom metric
MetricId PROTOCOL_MESSAGES The messages of a gRPC stream, a WebSocket session or a Kafka call
HttpProtocol SQL, REDIS, KAFKA The protocols of the K6 extensions
MonitorType K6_RUNTIME The K6 Runtime monitor of a K6 load generator
Report filter keys engine, k6SampleType, metricName Filter by engine, by K6 sample type, by custom metric

7. Migration checklist

  1. Post a VirtualUserCheckSettings body to run a validation, and stop calling the route with {providerId}/{location} (§2).
  2. Handle a 400 on a scenario launch as a scenario to fix, not a call to retry, and call the sanity check first to fail early (§5).
  3. Replace the calls to GET /runtime/scenarios/memory-usage/{scenarioId} and POST /workspaces/docker-providers/usage/all with the sanity check (§5).
  4. Accept the new enum values in the responses you parse (§6).