How to automate JMeter tests with Maven and Jenkins (Hudson)

So I’ve just had a great success getting my JMeter tests to run in Maven.  The next step: I want my continuous build server to run the tests automatically.  I’m using Jenkins as my continuous build server.  I used to use Hudson.  Then they forked, and I picked one.

1) Get your Jenkins running

I have a Jenkins server that is dedicated and was installed with the Ubuntu distro.  But for the purposes of this howto I’m going to run an instance locally.  This is pretty easy if you have a java environment already going.  Just download the latest jenkins war, then run it on the command line with

java -jar jenkins.war

You’ll have a fresh Jenkins instance running at http://localhost:8080.

Next you need to add the Performance Plugin.

1. Click the Manage Jenkins link.
2. Click the Manage Plugins link.
3. Select the Available tab.
4. Find the Performance Plugin, select it, then install it.

I also added the git plugin so that I can have Jenkins pull my demo project directly from GitHub.  You have to have a command line git client installed for the plugin to work, and you might have to configure the location in Manage Jenkins -> Configure the System

2) Create the Job in Jenkins

1. Go to the Jenkins main page.
2. Click New Job.
3. Give the job a name.
4. Choose free-style software project.
5. Click OK
6. Under source code management I chose Git.
7. I set the repository URL to git@github.com/jribble/jmeter-demo.git
8. I set the branch to build to Stage_1_JMeter_and_Maven 
9. Add a Maven build step with “verify” as the goal.
10. Down in Post-build Actions check the Publish Performance test result report checkbox.
11. Click the Add a new report box and choose JMeter.
12. For Report files specify **/*.jtl
13. Also under Post-build Actions check the Archive the artifacts checkbox.
14. Under Files to archive specify **/*jtl-report.html.  This will keep a copy of the html formatted reports for each build so you can look at them in Jenkins.
15. Click Save
16. Click Build Now.

After a short wait you should get a successful build.  If you click into a specific build you will see a Performance Report link as well as a link under Build Artifacts that shows you the JMeter generated html report.

Jenkins Performance Report

Once you have more than one build Jenkins will show a Performance Trend link on the project, which will show you graphs correlating the results of multiple builds.

3) Fix the project

You might notice in the Performance Trend picture above that the result file has a time stamp appended to the end.  This is the default behavior of the JMeter Maven plugin, and it’s a problem for Jenkins.  Jenkins uses the test result file name to correlate results and if the name of the file changes every day Jenkins won’t know that yesterday’s results should be reported with today’s results.

We can suppress this behavior with a simple pom file change. Simply add this configuration xml to the jmeter-maven-plugin definition:

<configuration>
	<testResultsTimestamp>false</testResultsTimestamp>
</configuration>

So the resulting pom file looks like this:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>com.example</groupId>
	<artifactId>jmeter-demo</artifactId>
	<packaging>jar</packaging>
	<version>1.0-SNAPSHOT</version>
	<name>jmeter-demo</name>
	<url>http://maven.apache.org</url>

	<build>
		<plugins>
			<plugin>
				<groupId>com.lazerycode.jmeter</groupId>
				<artifactId>jmeter-maven-plugin</artifactId>
				<version>1.4.1</version>
				<configuration>
					<testResultsTimestamp>false</testResultsTimestamp>
				</configuration>
				<executions>
					<execution>
						<id>jmeter-tests</id>
						<phase>verify</phase>
						<goals>
							<goal>jmeter</goal>
						</goals>
					</execution>
				</executions>
			</plugin>
		</plugins>
	</build>
</project>

Now if you build the project again with the settings given here you will get two sets of results, because the old results aren’t cleaned up.  Might be worth changing the project configuration to start clean every time.

That’s it.  You can grab the sample project from my jmeter-demo project on github.  The Stage_2_Automate_In_Jenkins branch has the final project settings for Jenkins.