> OctoPerf load testing - Jenkins continuous integration documentation page. How to run OctoPerf from Jenkins. Source: https://api.octoperf.com/doc/runtime/edit-scenario/integrations-automation/ci-cd/ci/jenkins/ # Jenkins Getting started OctoPerf integrates with the most used **Continuous Integration** server: [Jenkins](https://jenkins-ci.org/). [Continuous Integration](https://en.wikipedia.org/wiki/Continuous_integration) is the heart of Agile companies who run tests on a daily basis. By integrating load tests in their continuous integration cycles, companies can quickly **spot performance regressions** during the development phase. This is crucial to drill down to the issue very quickly, and **fix performance hogs as soon as they occurs**. ## What is Jenkins ![Jenkins Automation server](https://api.octoperf.com/doc/runtime/edit-scenario/integrations-automation/ci-cd/ci/img/jenkins-logo.png "Jenkins") Jenkins is the lead open-source automation server. Thousands of companies use it to run software builds, tests and deployments every day. Jenkins can be extended by using **plugins**. To run load tests on Octoperf directly from Jenkins, you need to install [Octoperf Jenkins Plugin](https://wiki.jenkins-ci.org/display/JENKINS/OctoPerf+Plugin). ### Plugin The [OctoPerf Plugin](https://wiki.jenkins-ci.org/display/JENKINS/OctoPerf+Plugin) allows to launch load tests on Octoperf directly from Jenkins. The job will not only launch the test but also: - Run the Jenkins job until the test is finished, - Fetch various test statistics during the test run and output them in the build console, - Download [JUnit](http://junit.org/) report and store it in the build, - Download [JMeter](http://jmeter.apache.org/) logs and store them in the build. The various reports and logs downloaded when the test is finished allow to have performance trends graphs and build status according to the test health (errors, assertion failures and more). ## Installation Our load testing Jenkins plugin can be installed directly from Jenkins UI. The following procedure explains how to install it: - Login to Jenkins and select *Manage Jenkins* in the left menu, - Select *Manage Plugins*, - Select *Available* tab and enter **OctoPerf** in the *Filter* box in the top right corner, - Finally, select one of the install options and restart Jenkins. ![OctoPerf Jenkins Plugin installation](https://api.octoperf.com/doc/runtime/edit-scenario/integrations-automation/ci-cd/ci/img/install-octoperf-jenkins-plugin.png "Octoperf Jenkins Plugin") Now, an [OctoPerf](https://octoperf.com) account must be provided to connect Jenkins to our Cloud load testing platform. ## System Settings ![OctoPerf Jenkins Plugin Settings](https://api.octoperf.com/doc/runtime/edit-scenario/integrations-automation/ci-cd/ci/img/jenkins-settings.png "Octoperf Jenkins Plugin Settings") The server URL is fully configurable. To use the Jenkins plugin with [OctoPerf On Prem Infra](https://api.octoperf.com/doc/on-premise-infra/), enter the appropriate Server URL according to your setup. By default, the plugin is configured to use the Saas platform. ## Credentials This section explains how to add an OctoPerf account to Jenkins. This step is required to connect to Octoperf and launch load tests from a Jenkins job: - Login to Jenkins and select *Credentials* in the left menu, - Select *Global Credentials* in the credentials table, ![Jenkins credentials](https://api.octoperf.com/doc/runtime/edit-scenario/integrations-automation/ci-cd/ci/img/jenkins-credentials.png "Jenkins credentials setup") - Select *Add Credentials*, - In kind select *Octoperf Account* and then provide your account username and password, - Test the credentials by clicking on *Test OctoPerf Credentials*, this will try to login on our platform, - When finished, click on *OK* to add the credentials. ![Jenkins OctoPerf credentials](https://api.octoperf.com/doc/runtime/edit-scenario/integrations-automation/ci-cd/ci/img/jenkins-octoperf-credentials.png "Jenkins Octoperf Credentials") You should end up with the account being added to the global credentials list. ![Jenkins Global credentials](https://api.octoperf.com/doc/runtime/edit-scenario/integrations-automation/ci-cd/ci/img/jenkins-global-credentials.png "Jenkins Global Credentials") *Jenkins Global credentials configuration* Now, it's time to create a **Jenkins job** which will execute a load test on **OctoPerf**. ## Job Setup ### Freestyle The following procedure explains how to setup a Jenkins job which runs a load test and collects some test results for reporting: - Login to Jenkins and click on *New Item* in the left menu, - Give the job a name and select *Freestyle project* or any relevant project type, - Select *OK* to create the job, you should be redirected to the project configuration page, - In the *Build* section, select *Add Build Step* and Select *OctoPerf*. ![Add Build Step](https://api.octoperf.com/doc/runtime/edit-scenario/integrations-automation/ci-cd/ci/img/add-build-step.png "Add Build Step") Now, we have a build configured to launch a load test on our load testing platform. It's time to configure the build step: - Select the *Credentials* used to login, - Select the *Workspace*, - Select the *Scenario* to run, the combobox should autofill with information gathered from our servers, - Optionally define **Test Stop Conditions**. ![Octoperf Build Step](https://api.octoperf.com/doc/runtime/edit-scenario/integrations-automation/ci-cd/ci/img/octoperf-build-step.png "Octoperf Build Step") *Build step configuration* ### Pipeline OctoPerf Jenkins plugin supports the pipeline syntax since version *3.2.0*. Here is an example Jenkins Pipeline Script written in Groovy: ```groovy node { stage ('OctoPerf') { octoPerfTest credentialsId: 'XXXXXX-XXX-XXX-XXXX', scenarioId: 'XXXXXXX' } stage ('JunitReport') { junit 'junit-report.xml' } stage ('PerfReport') { perfReport modeThroughput: true, sourceDataFiles: '**/*.jtl' } } ``` !!! note `credentialsId` corresponds to the [Jenkins credentials Id](#credentials). If you are unsure of the syntax or various Ids, you can use the [pipeline snippetizer](https://www.jenkins.io/blog/2016/05/31/pipeline-snippetizer/) to generate your pipeline syntax. The script above defines three stages: - *OctoPerf*: runs a given scenario on OctoPerf, collects Junit and JMeter JTL reports, - *JunitReport*: analyzes and publishes Junit reports using JUnit Plugin, - *PerfReport*: analyzes and publishes JMeter JTL reports using Performance Plugin. ![Octoperf Pipeline](https://api.octoperf.com/doc/runtime/edit-scenario/integrations-automation/ci-cd/ci/img/octoperf-pipeline.png "Octoperf Pipeline") Thanks to pipeline support, OctoPerf tests can be fully integrated into a **devops** continuous integration cycle. ## Execution We can now run a load test from Jenkins by launching the job. The job execution should display various information about the test run including: - Scenario being run, - Basic statistics like Hits, Errors and Response time, - And build status. ![Octoperf Build Console](https://api.octoperf.com/doc/runtime/edit-scenario/integrations-automation/ci-cd/ci/img/octoperf-build-console.png "Octoperf Build Console") *Various logs and metrics about the test* The build status indicates if the test has been successfully run on Octoperf. It does not reflect if errors or assertion failures happened during the load test. In order to get performance trends and build status depending on the errors faced during the test, our plugin stores JMeter logs and junit reporting. ## JUnit Reporting The Jenkins plugin generates and stores a JUnit report at the end of the test in the file **junit-report.xml**. This is especially useful if the build should be marked as unstable or failed when errors or assertion failures are encountered during the test run. **JUnit Report configuration** - Edit the job configuration, - Select *Add Post-build step*, then select *Publish JUnit result report*, - Specify *junit-report.xml* in the **Test Report XMLs**. ![JUnit Reporting](https://api.octoperf.com/doc/runtime/edit-scenario/integrations-automation/ci-cd/ci/img/junit-post-build-step.png "JUnit reporting") *JUnit Post build step configuration* This instructs the Jenkins job to take the JUnit results generated by our servers and convert them to readable test results. **The generated junit report only includes containers**. ![JUnit Report Trends](https://api.octoperf.com/doc/runtime/edit-scenario/integrations-automation/ci-cd/ci/img/build-report-trend.png "JUnit report trends") *Build Test Trends showing a regression on last build.* The screenshot above shows a sample load test project with **Test result trends**. As seen above, the last build encountered some failures. By clicking on *Latest Test Results* link, detailed test results are shown. ![JUnit Test Results](https://api.octoperf.com/doc/runtime/edit-scenario/integrations-automation/ci-cd/ci/img/junit-test-results.png "JUnit Test Results") Thanks to these results, users can quickly spot **performance regressions**. Triggering tests with assertions from Jenkins can be used to run **functional tests**. The complete test report can be viewed in our [online reporting](https://api.octoperf.com/doc/analysis/edit-bench-report/) for further analysis. ## Performance Reporting The [Jenkins Performance plugin](https://wiki.jenkins-ci.org/display/JENKINS/Performance+Plugin) combined with [Octoperf Jenkins Plugin](https://wiki.jenkins-ci.org/display/JENKINS/OctoPerf+Plugin) fully unleashes the power of running load tests inside Jenkins. The *Performance plugin* allows to capture performance metrics from JMeter logs. Jenkins will generate **graphic charts with performance trends**. Based on the performance metrics, the build status can be set as *good*, *unstable* or failed. Octoperf's Jenkins plugin downloads and stores the JMeter logs in the **logs** folder. to get performance graph trends and metrics inside Jenkins when running load tests on Octoperf: - Install the [Performance Plugin](https://wiki.jenkins-ci.org/display/JENKINS/Performance+Plugin), - Edit the job configuration and add a *Post-build action* named **Publish Performance test result report**, - Configure the performance plugin as desired. ![Performance Plugin](https://api.octoperf.com/doc/runtime/edit-scenario/integrations-automation/ci-cd/ci/img/octoperf-performance-plugin.png "Performance Plugin integration") After running a few performance tests, it should start showing up performance metrics and trends on the job page like shown on the screenshot below. ![Jenkins Performance Trends](https://api.octoperf.com/doc/runtime/edit-scenario/integrations-automation/ci-cd/ci/img/octoperf-jenkins-performance-trends.png "Performance Trend Graphs") The performance trend graphs allow to have an history of the build performance. This is particularly useful to spot slow growing performance issues. ## Online reporting Link to the online report is published at the beginning of the build logs. Every test run from Jenkins has an associated analysis report on our cloud platform.