Playwright Spec action¶

This action contains one or several Playwright tests, it can only be used from inside a Playwright virtual user.

Script¶

When you create a new Playwright Spec action it comes with a default script on our demo application:

Info

More details about the creation process of this script can be found on our blog.

For a more complete overview of all the scripting possibilities, check Playwright official documentation.

OctoPerf provides code auto-completion when using CTRL+Space from inside the script:

Variables¶

Read¶

Variables can be used inside a PlayWright action using the following syntax: process.env.variableName where variableName is the name of your variable. Here is a real life example:

  await page.locator('input[name="username"]').click();
  await page.locator('input[name="username"]').fill(process.env.login);
  await page.locator('input[name="username"]').press('Tab');
  await page.locator('input[name="password"]').fill(process.env.password);

In this case, both values will be replaced with the variables upon runtime:

Info

In order to use variables inside text for instance in the title of your test, you can use the following:

test(`test with ${process.env.login}`, async ({page}) => {

Note that the simple quotes around the text have been replaced with backquotes, as instructed by typescript documentation.

Write¶

While inside a PlayWright action, you can also write variables to be saved for later using setVariableValue("name", "value") simply replace name and value with the variable. The same is true for properties using: setPropertyValue("name", "value")

A typical use case would be to use the following code to extract the value of an input inside the page and then write it to a variable for later use:

  const sourcepageLocator = page.locator('input[name="_sourcePage"]').nth(0);
  const sourcepage = await sourcepageLocator.getAttribute('value');
  setVariableValue('_sourcePage',sourcepage);

Info

You can change the nth(0) to another value to get a different occurrence of this input, in case there are several. For more details, check PlayWright documentation.

Files¶

It is possible to use files that were placed in the project files menu, using the path ../resources/.

Here's an example of how you would use them:

await page.locator('input[type="file"]').nth(0).setInputFiles('../resources/myfile.txt');

Core Web Vitals¶

During a load test, OctoPerf measures the Core Web Vitals of every page your script loads. There is nothing to add to the script: the measurement is always active.

Metric	Measures	Unit
`LCP` (Largest Contentful Paint)	when the main content of the page is displayed	milliseconds
`FCP` (First Contentful Paint)	when the first content of the page is displayed	milliseconds
`TTFB` (Time To First Byte)	when the first byte of the page is received	milliseconds
`INP` (Interaction to Next Paint)	how fast the page reacted to the clicks and key presses made on it	milliseconds
`CLS` (Cumulative Layout Shift)	how much the content moves while the page loads	score, no unit

Each measure belongs to the action that loaded the page: a page.goto(), or a click on a link that opens another page. An assertion or a hook never carries Web Vitals.

The Web Vitals show in the report as custom metrics, filtered by action like the response times.

An SLA profile judges them on Google's bands: a Web Vital that needs improvement raises a warning, a poor one a critical alarm. The Playwright sample Virtual User comes with a Core Web Vitals SLA profile attached.

Info

LCP and INP are measured by Chromium-based browsers; other browsers may report only part of the metrics.
INP only exists when the page receives interactions: a page that is only displayed has no INP.
The pages of a browser context created by the script itself, with browser.newContext(), are not measured.
A validation of the Virtual User does not report Web Vitals: run a test to get them.

Debug¶

In order to debug a Playwright virtual user we are first going to create two script actions as described in the tabs below:

Petstore login

PassFail

import {test, expect} from '@playwright/test';

test('test', async ({page}) => {
    await page.goto('https://petstore.octoperf.com/');
    await page.getByRole('link', {name: 'Enter the Store'}).click();
    await page.getByRole('link', {name: 'Sign In'}).click();
    await page.locator('input[name="username"]').click();
    await page.locator('input[name="username"]').fill('user1');
    await page.locator('input[name="password"]').click();
    await page.locator('input[name="password"]').fill('pass');
    await page.getByRole('button', {name: 'Login'}).click();
    await page.locator('#SidebarContent').getByRole('link').first().click();
    await page.getByRole('link', {name: 'FI-SW-01'}).click();
    await page.getByRole('link', {name: 'EST-1'}).click();
    await page.getByRole('link', {name: 'Add to Cart'}).click();
});

import {test, expect} from '@playwright/test';

test('test fail', async ({page}) => {
    await page.goto('https://petstore.octoperf.com/');
    await page.getByRole('link', {name: 'Enter the Store'}).click();
    await page.getByRole('link', {name: 'Sign In'}).click();
    await page.locator('#stripes--1962445802').click();
    await page.locator('#stripes--1962445802').fill('user1');
    await page.locator('input[name="password"]').click();
    await page.locator('input[name="password"]').fill('pass');
});

Now when you run a validation you can check the Debug panel to see the details:

Warning

If you are getting certificates errors like ERR_CERT_AUTHORITY_INVALID make sure to add the proper configuration in order to ignore https errors in your script.

Trace¶

A nice way to see the details of a run is to use the Playwright Trace viewer. In OctoPerf you use the trace button or when you click on the icon next to your playwright action name in the debug menu. This opens the trace viewer in a new tab:

Note

Each trace is also available along with the logs and files of the validation.

Aggregators¶

With no aggregation, results like response times will be reported for every hostname of your playwright script. That means any URL with a dynamic parameter won't be reported individually.

Aggregators allow you to bypass that and group requests based on a regex expression. That way you can have the averaged values for a group of similar requests.

To experiment with aggregators, it is best to use the debug tab after a validation, the difference is visible using the Raw/Aggregated buttons in the bottom section:

Raw resourcesAggregated resources

The Raw resources displays all network call to help you write aggregators.

The Aggregated resources displays aggregated network calls. By default simply using hostnames.

You can use aggregators to get precise results. For example, with the PetStore:

Regular Expression	Template
`petstore\.octoperf\.com/$`	`Home`
`petstore\.octoperf\.com/actions/Account\.action`	`Account`
`petstore\.octoperf\.com/actions/Catalog\.action`	`Catalog`
`petstore\.octoperf\.com/actions/Order\.action`	`Order`
`petstore\.octoperf\.com/actions/Cart\.action`	`Cart`
`petstore\.octoperf\.com/(.+)`	`Asset $1$`

Aggregators are executed in sequence. So make sure the first one does not match all resources (Here using the $ at the end of the regexp).

Warning

There is a hard limit on the number of network resources that a Playwright action can generate. Use aggregators to get detailed metrics for your website or application only (not for external resources like CDN calls, analytics tags, etc.).