How to write JMeter tests for your GWT-RPC servlet

I wrote this post more than a year ago and apparently never finished it and published it.  So I’m just publishing it as is.  It may not be done, but it may be useful to someone…

I work on a pretty sizable GWT project that we would like to be able to load test on a regular basis using JMeter.  The client uses GWT-RPC for communication to the server. GWT-RPC is just a servlet that uses the HTTP protocol.  It’s pretty easy to inspect the payload of a request packet and replay that with JMeter, but there are a few problems with that approach.  The first is that the encoding of the request and response are unique and somewhat opaque.  It’s not that easy to create tests with fuzzed data.  I found this post on the GWT-RPC protocol, and it was helpful, but it also told me that wasn’t the way I wanted to go.  The second problem is that the GWT-RPC protocol includes hash values that reflect the state of the code when it was compiled.  It seems that I could create some recorded tests, and they would work, until I modified the code such that the hash changed, at which point I would have to update all of my requests.

While looking for possible approaches I came across this post that describes a method of creating JMeter tests for GWT-RPC using Java for the client.  The post is light on specifics, but it pointed me in the right direction.  I’m happy with what I ended up with.  This post describes in detail how to make it work.

1) Make sure you have a GWT project

For this howto I’m using the basic project that’s created by default.  Specifically I used the Maven GWT plugin to generate the project using an archetype.

2) Create a project for the JMeter tests written in Java

You’re going to end up with two projects for the JMeter tests.  I like to create them as modules of a parent.  To make things easy I’m also going to make the GWT project a module so all three build together with one command.  I do this by starting with a project directory with just a pom.  The pom describes the parent and has packaging “pom”.

Here’s my initial parent pom:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.example</groupId>
  <artifactId>jmeter-demo</artifactId>
  <version>1.0.0-SNAPSHOT</version>
  <packaging>pom</packaging>

  <name>jmeter-demo</name>
  <url>http://maven.apache.org</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>
  <modules>
  </modules>

</project>

Now inside of the directory with the parent pom I create a new quickstart project.  Maven will automatically create the project as a module of the parent.

mvn archetype:generate -DgroupId=com.example -DartifactId=jmeter-demo-java -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false

3) Build a JAR from your GWT project

I’m not 100% sure this is the only solution, but the JMeter Java project is going to depend on your GWT project as a Java component.  My thinking is that means I need to tell Maven to package it as a jar.  You can do this one of two ways: 1) you can split the GWT project in two, one that generates a jar and another that consumes the jar and generates a war; 2) you can configure your GWT project to build both a jar and a war.  The first solution is more the maven way, but I didn’t want to go through that effort on our big project, so instead I added a profile that would build a jar.  Here’s the xml I added:

	<profiles>
		<profile>
			<id>jar</id>
			<activation>
				<activeByDefault>true</activeByDefault>
			</activation>
			<build>
				<plugins>
					<plugin>
						<groupId>org.apache.maven.plugins</groupId>
						<artifactId>maven-jar-plugin</artifactId>
						<executions>
							<execution>
								<id>make-a-jar</id>
								<phase>compile</phase>
								<goals>
									<goal>jar</goal>
								</goals>
							</execution>
						</executions>
					</plugin>
					<plugin>
						<groupId>org.apache.maven.plugins</groupId>
						<artifactId>maven-install-plugin</artifactId>
						<executions>
							<execution>
								<phase>install</phase>
								<goals>
									<goal>install-file</goal>
								</goals>
								<configuration>
									<packaging>jar</packaging>
									<artifactId>${project.artifactId}</artifactId>
									<groupId>${project.groupId}</groupId>
									<version>${project.version}</version>
									<file>
										${project.build.directory}/${project.build.finalName}.jar
			                </file>
								</configuration>
							</execution>
						</executions>
					</plugin>
					<plugin>
						<groupId>org.apache.maven.plugins</groupId>
						<artifactId>maven-deploy-plugin</artifactId>
						<executions>
							<execution>
								<phase>deploy</phase>
								<goals>
									<goal>deploy-file</goal>
								</goals>
								<configuration>
									<packaging>jar</packaging>
									<generatePom>true</generatePom>
									<url>${project.distributionManagement.repository.url}</url>
									<artifactId>${project.artifactId}</artifactId>
									<groupId>${project.groupId}</groupId>
									<version>${project.version}</version>
									<file>${project.build.directory}/${project.artifactId}-${project.version}.jar</file>
								</configuration>
							</execution>
						</executions>
					</plugin>
				</plugins>
			</build>
		</profile>
	</profiles>

Now when I run mvn install I will get a jar of the GWT project installed in my local repo, not just the war.  And now I can add a dependency on the GWT project in my JMeter Java project:

    <dependency>
      <groupId>com.example</groupId>
      <artifactId>gwt-demo</artifactId>
      <version>1.0-SNAPSHOT</version>
    </dependency>

4) Add GWT SyncProxy to the JMeter Java project

GWT SyncProxy is a project that lets you create GWT-RPC clients in Java that run in Java.  The intent of the project is really to make it easy to write tests against GWT-RPC servers.  And it does a good job.

However, it’s not a project that exists in a central Maven repository, so we have to do something to make it available.  There are a few ways to tackle this problem – I’m going to do it with an in-project repository.  This means I need to add this xml to my JMeter Java project’s pom.xml:

	<repositories>
		<repository>
			<id>lib</id>
			<name>lib</name>
			<releases>
				<enabled>true</enabled>
				<checksumPolicy>ignore</checksumPolicy>
			</releases>
			<snapshots>
				<enabled>false</enabled>
			</snapshots>
			<url>file://${project.basedir}/lib</url>
		</repository>
	</repositories>

I add a dependency to the project to the gwtsyncproxy jar, making up most of the information like groupId and artifactId (I also need Google’s gdata jar as gwtsyncproxy depends on it):

		<dependency>
			<groupId>com.gdevelop</groupId>
			<artifactId>gwt-syncproxy</artifactId>
			<version>0.3.1</version>
		</dependency>
		<dependency>
			<groupId>com.google</groupId>
			<artifactId>gdata</artifactId>
			<version>1.0</version>
		</dependency>

I copy the jars into correct locations under ${project.basedir}/lib:

gwt-syncproxy.jar -> lib/com/gdevelop/gwt-syncproxy/0.3.1/gwt-syncproxy-0.3.1.jar
gdata.jar -> lib/com/google/gdata/1.0/gdata-1.0.jar

But wait, where do I get those jars?

You can grab the jars from my demo project on github, along with a full working example of this thing.  Or you can use Google to go find them.

5) Write some Java code

We have to add one more dependency to the JMeter Java project – for JMeter itself:


			org.apache.jmeter
			ApacheJMeter_java
			2.6

And now we create a Java class that extends org.apache.jmeter.protocol.java.sampler.AbstractJavaSamplerClient, creates a sync proxy on the GreetingService, and calls the one method as part of a test, returning an org.apache.jmeter.samplers.SampleResult.

package com.example;

import org.apache.jmeter.protocol.java.sampler.AbstractJavaSamplerClient;
import org.apache.jmeter.protocol.java.sampler.JavaSamplerContext;
import org.apache.jmeter.samplers.SampleResult;

import com.example.client.GreetingService;
import com.gdevelop.gwt.syncrpc.SyncProxy;

public class DemoSampler extends AbstractJavaSamplerClient {

	   private GreetingService svc = null;

	   @Override
	   public void setupTest( JavaSamplerContext context )
	   {
	      super.setupTest ( context );

	      try
	      {
	         svc = (GreetingService) SyncProxy.newProxyInstance ( GreetingService.class, "http://localhost:8888/GwtDemo/", "greet" );
	      }
	      catch ( Exception e )
	      {
	         throw new RuntimeException ( e );
	      }
	   }

	public SampleResult runTest(JavaSamplerContext arg0) {
	      SampleResult result =  new SampleResult ();
	      result.sampleStart();

	      String greeting = svc.greetServer( "Test" );

	      result.setSuccessful ( true );
	      result.setSampleLabel ( greeting );
	      result.sampleEnd();
	      return result;
	}

}

If you want to do a quick test you can add a main method to the DemoSampler:

	public static void main(String[] args) {
		DemoSampler test = new DemoSampler();
		JavaSamplerContext ctx = new JavaSamplerContext(new Arguments());
		test.setupTest(ctx);
		SampleResult result = test.runTest(ctx);
		System.out.println((result.isSuccessful() ? "SUCCESS: " : "FAILURE: ")
				+ result.getSampleLabel());
	}

Get the GWT project running by executing “mvn gwt:run” in the GWT project directory, then run DemoSampler as a Java application (I do that in Eclipse).
This is the result I get:

SUCCESS: Hello, Test!

I am running jetty-6.1.x.

It looks like you are using:
Java/1.6.0_24

6) Create a project for the XML JMeter tests

From this point you’re pretty much going to follow my post on How to run JMeter tests in Maven, with a few modifications:

  1. You’ll need to add the JMeter Java project as a dependency on the jmeter plugin in the test project pom.xml.
  2. In order to be able to use your Java sampler from the JMeter GUI you need the GUI to recognize your jar as an extension.  The way I know how to do this is to copy the jar into the JMeter lib/ext directory.  You will also need to make all of the jars that the sampler jar is dependent on available to the GUI.  You can either copy them all to the JMeter lib directory or you can use maven to make an uber jar that contains all dependencies.  I actually didn’t like either of these solutions.  I forked the jmeter plugin and added a goal that would open the gui for me.
  3. When you create the test case add a Java Request sampler and select your sample from the drop down box.

That’s it.  Write some load tests.  Run them automatically in Jenkins. Have fun.

Advertisements

Subversion branches with Git and SourceTree

Where I work we use Subversion for source control.  I really like Git, and I use the git-svn functionality so I use git locally against a shared subversion server.

I am also using a Mac and prefer SourceTree as a frontend for my source control.  It handles the git-svn stuff automatically and very nicely.

The problem I have is that we use a nonstandard branching organization in subversion, but I want to be able to switch between different subversion branches inside of my local git repository.  I couldn’t figure out how to do this for the longest time, and finally ran into a post somewhere that gave me enough of a hint to figure out how to make it work.  It’s not pretty, and I may be doing some unnecessary bits, but it does seem to work.  Here’s how I do it:

  1. Checkout the trunk subversion repository (using SourceTree) as a git repository.
  2. Open the configuration file at .git/config.
  3. Add a new remote section like this:
    [svn-remote "1.0svn"]
       url = https://my.svn.server/svn/branches/1.0/myproject/
       fetch = :refs/remotes/git-svn-1.0
    
  4. On a command line, in the local git repo, run
    git svn fetch 1.0svn
    
  5. Refresh the view in SourceTree. You will now see git-svn-1.0 listed as a remote under Subversion. You can check out the remote branch as a local branch that tracks the remote branch.

Bulk deleting from Google Calendar

I use Novell Groupwise at work and up until a couple days ago used this Cool Solution to sync my Groupwise calendar to my Google Calendar.  The gist of it is that Groupwise automatically forwards every new appointment to my gmail, which automatically adds the appointment to my calendar.  This worked fine and kept me happy except for one glaring problem.  Appointments don’t automatically delete.

Over time this has gotten to be a bigger and bigger problem as I have more and more old recurring appointments that have been deleted from Groupwise, but are still happily recurring in Google Calendar.

Groupwise 8 has a new feature by which you can publish the calendar as an ICS feed.  You can subscribe to it as a separate calendar in Google Calendar and it will all show up nice and cleanly along with your other calendars.  It’s a much better solution that forwarding rules, and it will keep in sync so I don’t have delete problems.

So I turned off my forwarding rule for calendar events, published my Groupwise calendar and subscribed to it in Google Calendar.  But I still have the problem of a bunch of old recurring Groupwise events in my main Google Calendar.   Since I followed the Cool Solution exactly, each of my Groupwise events has “GWDelegate” in the description in Google Calendar.  So when I do a search in the calendar for GWDelegate all of the Groupwise events show up in the results.  This is nice and simple.  Now I just have to delete them.

One by one.  Because I can’t delete multiple items in Google Calendar.  Ugh.

Okay, new plan.  I export the calendar to an ICS file.  I run the following awk script on the ICS file, creating a new file that has all of the GWDelegate events removed:


#!/usr/bin/env ksh

awk '
# At the beginning of an event we start caching the lines until we figure out
# if we are going to keep it or drop it
 /BEGIN:VEVENT/ {
 cache = 1;
 drop = 0;
 }

# If we match the drop criteria (a line includes the work GWDelegate)
# we flag the record as to drop, we clear the cache and we stop caching
 /GWDelegate/ {
 drop = 1;
 cached_lines = "";
 cache = 0;
 }

# If we find the end of an event we print out the cache, clear the cache, and stop caching
# print the current line if not dropping
# stop dropping and go on to the next line
 /END:VEVENT/ {
 printf( "%s", cached_lines );
 cached_lines = "";
 cache = 0;
 if ( !drop )
 print;
 drop = 0;
 next;
 }

# If we are caching we add the current line to the cache and go on to the next line
 cache {
 cached_lines = cached_lines $0 "\n";
 next;
 };

# If we are not caching and we are not dropping the current event we print the current line
 !drop { print; }
'

This awk script is a modification of one I found in this forum thread, so thanks to agama for the original solution.

Now that I have a full export of my Google Calendar with the events removed that I no longer want I can delete all events from the calendar and import the modified ICS file, restoring the ones I still want.  Not quite as nice as being able to delete all search results, but it does work.  This solution should be easy to modify to search for events with any attribute, not just GWDelegate somewhere in the event.

How to automate JMeter tests with Maven and Jenkins (Hudson)

So I’ve just had a great success getting my JMeter tests to run in Maven.  The next step: I want my continuous build server to run the tests automatically.  I’m using Jenkins as my continuous build server.  I used to use Hudson.  Then they forked, and I picked one.

1) Get your Jenkins running

I have a Jenkins server that is dedicated and was installed with the Ubuntu distro.  But for the purposes of this howto I’m going to run an instance locally.  This is pretty easy if you have a java environment already going.  Just download the latest jenkins war, then run it on the command line with

java -jar jenkins.war

You’ll have a fresh Jenkins instance running at http://localhost:8080.

Next you need to add the Performance Plugin.

  1. Click the Manage Jenkins link.
  2. Click the Manage Plugins link.
  3. Select the Available tab.
  4. Find the Performance Plugin, select it, then install it.

I also added the git plugin so that I can have Jenkins pull my demo project directly from GitHub.  You have to have a command line git client installed for the plugin to work, and you might have to configure the location in Manage Jenkins -> Configure the System

2) Create the Job in Jenkins

  1. Go to the Jenkins main page.
  2. Click New Job.
  3. Give the job a name.
  4. Choose free-style software project.
  5. Click OK
  6. Under source code management I chose Git.
  7. I set the repository URL to git@github.com/jribble/jmeter-demo.git
  8. I set the branch to build to Stage_1_JMeter_and_Maven 
  9. Add a Maven build step with “verify” as the goal.
  10. Down in Post-build Actions check the Publish Performance test result report checkbox.
  11. Click the Add a new report box and choose JMeter.
  12. For Report files specify **/*.jtl
  13. Also under Post-build Actions check the Archive the artifacts checkbox.
  14. Under Files to archive specify **/*jtl-report.html.  This will keep a copy of the html formatted reports for each build so you can look at them in Jenkins.
  15. Click Save
  16. Click Build Now.

After a short wait you should get a successful build.  If you click into a specific build you will see a Performance Report link as well as a link under Build Artifacts that shows you the JMeter generated html report.

Jenkins Performance Report

Once you have more than one build Jenkins will show a Performance Trend link on the project, which will show you graphs correlating the results of multiple builds.

3) Fix the project

You might notice in the Performance Trend picture above that the result file has a time stamp appended to the end.  This is the default behavior of the JMeter Maven plugin, and it’s a problem for Jenkins.  Jenkins uses the test result file name to correlate results and if the name of the file changes every day Jenkins won’t know that yesterday’s results should be reported with today’s results.

We can suppress this behavior with a simple pom file change. Simply add this configuration xml to the jmeter-maven-plugin definition:

<configuration>
	<testResultsTimestamp>false</testResultsTimestamp>
</configuration>

So the resulting pom file looks like this:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>com.example</groupId>
	<artifactId>jmeter-demo</artifactId>
	<packaging>jar</packaging>
	<version>1.0-SNAPSHOT</version>
	<name>jmeter-demo</name>
	<url>http://maven.apache.org</url>

	<build>
		<plugins>
			<plugin>
				<groupId>com.lazerycode.jmeter</groupId>
				<artifactId>jmeter-maven-plugin</artifactId>
				<version>1.4.1</version>
				<configuration>
					<testResultsTimestamp>false</testResultsTimestamp>
				</configuration>
				<executions>
					<execution>
						<id>jmeter-tests</id>
						<phase>verify</phase>
						<goals>
							<goal>jmeter</goal>
						</goals>
					</execution>
				</executions>
			</plugin>
		</plugins>
	</build>
</project>

Now if you build the project again with the settings given here you will get two sets of results, because the old results aren’t cleaned up.  Might be worth changing the project configuration to start clean every time.

That’s it.  You can grab the sample project from my jmeter-demo project on github.  The Stage_2_Automate_In_Jenkins branch has the final project settings for Jenkins.

How to run JMeter tests with Maven

I just finished an effort setting up automated load tests against a GWT-RPC server interface.  It took me some time to pull together the pieces from different sources and figure out what was possible and how I wanted to organize the whole thing.  But once I was there the result is pretty simple.  So I’m going to write it down here and hopefully save someone else the effort of figuring it out like I did. So this post is the first part of the puzzle – how to create a project that will run JMeter tests with Maven.  I got a lot of help from some other people’s posts on this subject.  Jacobo Matute’s post Performance Tests with Jmeter, Maven and Hudson was a big help, but fortunately slightly out of date. My goal with this project was to have something that works the maven way.  All I should have to do is:

  • Checkout the project.
  • Run a maven goal.

I don’t want to have to install JMeter first.  I don’t want to have to go get any dependencies. That’s what Maven’s for.  This used to be a problem because there were no JMeter artifacts available, but that’s fortunately not true anymore.

1) Create a simple maven project

I use the quickstart archetype:

mvn archetype:generate -DgroupId=com.example -DartifactId=jmeter-demo -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false

This creates a basic java project named jmeter-demo, with and example App class and an AppTest test class.  We don’t need either of these, so we can delete them.  Make sure you keep src/main/java and src/test/java, but you can delete the com directory in each of these. Now you should be able to

cd jmeter-demo

and

mvn install

and get a build success, but nothing was built.

2) Add the JMeter Maven plugin

All of the heavy lifting will be done for us by the JMeter Maven plugin.  Edit the pom.xml file in the jmeter-demo directory so it looks like this:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>com.example</groupId>
	<artifactId>jmeter-demo</artifactId>
	<packaging>jar</packaging>
	<version>1.0-SNAPSHOT</version>
	<name>jmeter-demo</name>
	<url>http://maven.apache.org</url>

	<build>
		<plugins>
			<plugin>
				<groupId>com.lazerycode.jmeter</groupId>
				<artifactId>jmeter-maven-plugin</artifactId>
				<version>1.4.1</version>
				<executions>
					<execution>
						<id>jmeter-tests</id>
						<phase>verify</phase>
						<goals>
							<goal>jmeter</goal>
						</goals>
					</execution>
				</executions>
			</plugin>
		</plugins>
	</build>
</project>

So you’re removing the junit dependency and adding a build plugin that runs jmeter tests when you do the verify phase of the lifecycle. If you run mvn install now you will get a failure because you don’t have the jmeter test directory created.

3) Create a JMeter test

The JMeter plugin will run any test that it finds under the src/test/jmeter directory.  A JMeter  test is just xml, but it’s easiest to create this xml with the JMeter GUI.  Currently you have to install JMeter in order to run the GUI, but hopefully very soon it will be possible to run the GUI directly from the JMeter plugin. Using the GUI I created a simple test with a Thread Group that runs 10 times, an HTTP Request Sampler that requests the Google Home Page at http://www.google.com, and a View Results in Table that shows the timing results.  Then I ran it in the GUI.  I saved the test in a src/test/jmeter directory that I created.

4) Run it in Maven

Now I run

mvn install

in my project directory.  Here’s a piece of the output:

[INFO] --- jmeter-maven-plugin:1.4.1:jmeter (jmeter-tests) @ jmeter-demo ---
[INFO]
[INFO] -------------------------------------------------------
[INFO]  P E R F O R M A N C E    T E S T S
[INFO] -------------------------------------------------------
[INFO]
[WARNING] Unable to parse the 'jmeter.exit.check.pause' entry in jmeter.properties!  Falling back to a default value of '2500'.
[INFO]
[INFO] Proxy server is not being used.

[info]
[debug] JMeter is called with the following command line arguments: -n -t C:\projects\jmeter-demo\jmeter-demo\src\test\jmeter\Demo Test.jmx -l C:\projects\jmeter-demo\jmeter-demo\target\jmeter\report\Demo Test-120416.jtl -d C:\projects\jmeter-demo\jmeter-demo\target\jmeter
[info] Executing test: Demo Test.jmx
INFO    2012-04-16 14:51:59.725 [jmeter.e] (): Listeners will be started after enabling running version
INFO    2012-04-16 14:51:59.726 [jmeter.e] (): To revert to the earlier behaviour, define jmeterengine.startlistenerslater=false
[info] Completed Test: Demo Test.jmx
[info]
[info] Building JMeter Report(s)...
[info]
[info] Raw results: C:\projects\jmeter-demo\jmeter-demo\target\jmeter\report\Demo Test-120416.jtl
[info] Test report: C:\projects\jmeter-demo\jmeter-demo\target\jmeter\report\Demo Test-120416.jtl-report.html
[INFO]
[INFO] Test Results:
[INFO]
[INFO] Tests Run: 1, Failures: 0, Errors: 0
[INFO]
[INFO]

We can see that the test ran successfully, the raw results are located at target/jmeter/report/Demo Test-120416.jtl and an html report is located at target/jmeter/report/Demo Test-120416.jtl-report.html. You can grab the sample project from the Stage_1_JMeter_and_Maven branch of my jmeter-demo project. Check out my next step in the automated load tests effort:

How to automate JMeter tests with Maven and Jenkins (Hudson)