Trifork Blog

AngularJS training

Performance testing a Flex BlazeDS application

July 14th, 2009 by
| Reply

In the past few years I've seen an increasing interest in Flex applications at our customers. I have to say that I'm not surprised about this trend. Not only do Flex applications generally look great, but they also provide a big boost to user experience. As a developer and architect I am also quite pleased with the programming model and extensive widget library. Sure, Adobe can still improve on a lot of things, but so far I have always worked with pleasure on Flex applications.

Recently, at one of our customers, I bumped into another interesting side of Flex application development, namely performance testing (e.g. load-testing or stress-testing). Surprisingly enough, it is very hard to find an affordable solution for this. There are commercial tools out there that seem to do it quite well (Neoload) or reasonably well (WebLOAD) but I could not find a single viable open source alternative.

Of course, it's not exactly straightforward to write such a tool. The difficulty lies in the ability to generate and send AMF requests to the webserver, plus the interpretation of the AMF response you get back. The standard approach that most performance testing tools use is based on the assumption of HTTP requests and responses. While the AMF protocol technically "hitches a ride" on the HTTP protocol as well, the content is packed into a binary format (AMF) that can't be interpreted the regular way HTTP requests and responses can. Hence the need for a dedicated tool that can interpret AMF. However, not every customer has tens of thousands of dollars to spend. So, how do you cope with this?

AMF request generation

As it turns out, with a little ingenuity and the help of the BlazeDS codebase, you can create a solution that together with the excellent JMeter allows you to execute performance tests just as you are used to doing. Let's start with the generation of AMF requests.

In the blazeds-core.jar there are representations of AMF messages and ways of de-/serializing them. This is actually used by BlazeDS itself to convert between AMF data to a Java Object model and vice versa. Because they kindly open sourced this code, I was able to reuse these classes to generate AMF requests and save those to a file. Here are a couple of code snippets I wrote to do this:

public byte[] generate(AmfRequestParameters params) {
    ActionMessage message = new ActionMessage();
    message.addBody(createMessageBody(params));

    ByteArrayOutputStream out = new ByteArrayOutputStream();
    MessageSerializer serializer = createMessageSerializer(out);
    try {
        serializer.writeMessage(message);
    } catch (IOException e) {
        throw new AmfRequestGenerationException(e);
    }

    return out.toByteArray();
}

First thing to note is the creation of an ActionMessage. This is actually a representation of an AMF request to which we will add a body in the second line of this method. Because we want to save this AMF request to a file we of course need to serialize it first. We do this by using the MessageSerializer class BlazeDS kindly provides. We let it serialize the message to a ByteArrayOutputStream so that we can potentially use it for other purposes than saving it to a File, but you could just as easily pass in a FileOutputStream. Finally, we return the byte array back to the client that called this method.

Building the message

So, let's have a look at the createMessageBody(params) method:

protected MessageBody createMessageBody(AmfRequestParameters params) {
    RemotingMessage message = new RemotingMessage();
    message.setHeader(HEADER_DS_ENDPOINT, channel);
    message.setHeader(HEADER_DS_ID, "1");
    message.setMessageId("1");
    message.setDestination(params.getDestination());
    message.setOperation(params.getOperation());
    message.setBody(params.getParameters());

    return new MessageBody(null, null, new RemotingMessage[] {message});
}

What we do in this method is create an instance of a RemoteMessage. If you have ever inspected the contents of an AMF request, some of the properties might be familiar to you. Two headers are set: "DSEndpoint" and "DSId". The endpoint is actually the channel in your services-config.xml file that you use to send all communication through (by default called something like 'channel-amf'). I am not quite sure what the id header is supposed to do, but BlazeDS seems to require it. However the value does not seem to matter, hence the "1". Neither does the message id for that matter.

The interesting parts are the destination, operation and parameters. These properties relate to the backend service you want to call, the method on that service and any arguments for that method. Finally, we return a MessageBody with this RemotingMessage that will be added to the ActionMessage in the generate() method which we saw earlier.

I introduced a small data holder class to hold the dynamic values of the message, called AmfRequestParameters:

public class AmfRequestParameters {

    private String destination;
    private String operation;
    private Object[] parameters;

    /**
     * Creates a new {@code AmfRequestParameters} with the given fields.
     *
     * @param destination destination of the amf request
     * @param operation operation to call on the destination
     * @param parameters parameters of the operation
     */
    public AmfRequestParameters(String destination, String operation, Object[] parameters) {
        this.destination = destination;
        this.operation = operation;
        this.parameters = parameters;
    }

    // getters and setters omitted
}

Creating a message serializer

The last thing that needs to be done is serializing the message. This can be done with a MessageSerializer:

protected MessageSerializer createMessageSerializer(OutputStream out) {
    SerializationContext context = new SerializationContext();
    context.setSerializerClass(AmfMessageSerializer.class);
    SerializationContext.setSerializationContext(context);

    MessageSerializer serializer = context.newMessageSerializer();
    serializer.initialize(context, out, null);

    return serializer;
}

In order to create a MessageSerializer, BlazeDS needs to know what type of serializer it needs to create. This is done via a SerializationContext on which you specify the AmfMessageSerializer class as the one it should instantiate. We then initialize the serializer with the context and the chosen outputstream, which is a ByteArrayOutputStream in our case. Optionally, you can add an AmfTrace to the initialization procedure, but I didn't find that to be of any value to me.

Adding the AMF request to JMeter

In the end, we now have a byte array that contains a serialized AMF message. This can easily be stored to a file, after which it can be loaded into JMeter:

JMeter AMF request screenshot

A couple of things are important to know when sending AMF requests with JMeter. First of all you need to make sure that you point JMeter to the URL that exposes your backend services to your Flex application. This is actually the endpoint URI under the channel element in your services-config.xml file. Second of all, make sure that you specify the HTTP request method as POST. Makes sense if we're sending a file along with the request. Finally, point JMeter to the AMF request file to include in the request and specify its MIME type as "application/x-amf".

And that's it, you are all set for performance testing your Flex BlazeDS application!

Limitations

There are a couple of limitations to this approach:

  • You can't inspect the response of a request in detail. As the response is also in AMF, JMeter can't inspect it and make assertions on it
  • HTTP response codes are less meaningful. As basically every response from the server is a HTTP 200 response code, we can't detect well if an error was thrown. For example, if we get a BadCredentialsException during login, we could have added an assertion to JMeter to check if the user actually got to the next screen, instead of being redirected to the login screen. Unfortunately, this does not work for Flex applications as the client application itself has to make a decision on what do with this type of error (instead of the server doing this for us)
  • This approach is limited to BlazeDS only. Due to the code we're using from the BlazeDS codebase, we can't for example use WebORB instead of BlazeDS. This is mostly due to the fact that there are subtle differences in the way certain data types are serialized and deserialized
  • Maybe the biggest drawback is that you have to create an AMF request file for each variation on a request. For example, if we want to login with 100 different users, we also need to create 100 login request files. Normally, you would be able to script this in JMeter by adding variables to the request parameters.

Currently I am looking into solving some of these limitations by making use of JMeter's PreProcessors and PostProcessors. You can add custom Java code to JMeter that will execute on each request. You may therefore be able to generate the AMF request file on the fly.

Conclusion

By making use of the BlazeDS codebase and JMeter we are able to performance test a Flex BlazeDS application in a very cost-effective manner. JMeter has proven itself over the years to be an excellent tool for performance testing and it is a relief that, with a little ingenuity, we can use it for Flex applications too.

I think I've only scratched the surface on what can be accomplished with this combination. There is definetely a lot of potential to be explored here and I encourage everyone that has read this blog entry to go out there and try it yourself!

20 Responses

  1. July 14, 2009 at 15:57 by Erik van Oosten

    Just posted on InfoQ:

    http://www.infoq.com/news/2009/07/flex-monkey-1.0-released

    Gorilla Logic, Inc. has announced the first production release of FlexMonkey with version 1.0. FlexMonkey is an open source testing tool for Flex and AIR applications. FlexMonkey provides for the capture, replay, and verification of Flex user interface functionality.

  2. July 15, 2009 at 11:01 by Rob de Boer

    Hey Erik,

    I have used FlexMonkey in the past and indeed it is a nice tool. Unfortunately, it focuses on functional and integration testing. I do like their style of recording test scenarios in an easy way.

    Cheers,

    Rob

  3. July 30, 2009 at 11:27 by iamcoder

    Hi,

    You can get more information about flex testing tools from the following site.

    http://www.rsdhariwal.com/2009/07/29/testing-tool-for-flex-application/
    http://www.rsdhariwal.com/2009/07/30/flex-ui-selenium-for-flex-application-testing/

    I hope it helps

  4. January 13, 2010 at 20:47 by Aaron St. John

    Hello,

    Thank you for this post, I think it will be very useful for our own testing efforts. However, I am having trouble getting it working in my test environment and I was hoping you might be able to offer pointers on my sticking point (I think it's an easy answer)

    I have created a simple program based on the code you provided that links against the BlazeDS jar files, incidentally, in the current version of BlazeDS (3.2.0.397) the imports I had to make (which aren't included above) are as follows:

    flex.messaging.messages.RemotingMessage,flex.messaging.io.amf.ActionMessage,flex.messaging.io.amf.MessageBody,flex.messaging.io.MessageSerializer,flex.messaging.io.SerializationContext and flex.messaging.io.amf.AmfMessageSerializer.

    I am testing this method against the Blaze DS Remoting sample included in adobe's turn-key distribution of BlazeDS found here:

    http://opensource.adobe.com/wiki/display/blazeds/Release+Builds

    The program outputs a file called mysample.amf that targets one of the methods called "getProduct" (which takes no arguments) in the sample webservice whose destination name is "product". The channel name, is set to "my-amf" based on the configuration xml files in the turn-key distribution.

    My current sticking point, is I'm now trying to setup JMeter as your screenshot shows to hit the webservice and pass along the mysample.amf to it, presumably to get a response that's NOT a 404. But I can't figure out what the appropriate "path" is to send the JMeter Request to in order to hit the service.

    The service is hosted on localhost at port 8400 ie [http://localhost:8400]. Based on the screenshot of your example above (and some google snooping), my closest best guess is the path should be something like: "samples/messagebroker/my-amf" which gives me a 404 error when I ping it, but returns the error message "The requested resource () is not available."

    Which tells me I may be on the right track because this response is slightly different than the 404 error I get from a truly non-existent page
    e.g. for "http://localhost:8400/nothing I get the 404" : "The requested resource (/nothing) is not available.", the difference being, in the previous post, the described resource is blank, instead of being an echo of the path I enter e.g. "/nothing"

    I believe I set everything up consistently with your post, so I believe this problem is coming from the pathing issue. The only other possible glitch could be that in the screenshot you provide in JMeter, JMeter detects your ".amf" file as the mime-type "application/x-amf" but when I load my ".amf" file into JMeter it doesn't detect a Mime-Type.

    This may be b.c. of how I'm writing out the byte[] to a file? I am simply doing a FileOutputStream.write(...) which seems like the obvious choice, perhaps there's a better way that flags the mime-type?

    Thanks again for the great post, if you can provide any help or pointers it would be much appreciated.

  5. January 13, 2010 at 20:59 by Aaron St. John

    *Sigh* figured it out 10 mins after posting. The correct path is:

    "samples/messagebroker/amf"

    The channel name is "my-amf". After I made this change, it all works perfectly.

    Apologies for the long post, maybe this will help someone else that comes across this.

    I would still be interested in info about why the application/x-amf mimetype isn't being recognized by JMeter, but it doesn't seem to affect anything.

    Cheers!

  6. February 25, 2010 at 16:58 by Batistuta

    Is it correct if I replace your 2 lines in createMessageBody() like this :

    message.setHeader("HEADER_DS_ENDPOINT", "my-amf");
    message.setHeader("HEADER_DS_ID", "1");

    Because when I send my AMF file, JMeter show me in "response" :

    AppendToGatewayUrlÿÿÿÿ

  7. July 21, 2010 at 23:48 by Vivek

    While searching for tutorials about AMF Flex and Java client, I came across this and posting question with a hope to get answer.

    I'm implementing a Java client which sends AMF request to BlazeDS server using AMFConnection library.
    In my case, I'm calling a remote API (through BlazeDS RemotingMessage library) which needs a Custom Server Object in the AMF request body. But I can't create this object because its class and related libraries are present on Server. I observed a way around to this in a ActionScript, where developers use below syntax, where they create a local class and link to Server Class.

    =================Action Script ==================
    [Bindable]
    [RemoteClass(alias=" com.ABC.PQR.sampleClass")]

    public class SampleClass {
    public function SampleClass() {
    }

    public var var1:String;
    public var var2:String;
    public var var3:ArrayCollection;
    public var var4:Boolean;
    public var var5:Boolean;
    }

    =============================================

    Can you please let me know how to send a custom object using Java client creating a AMF request with a custom object as request body, similar to the above example.

  8. July 28, 2010 at 23:43 by Vivek

    The problem got solved.
    To map local client object to server side object, I used registerAlias() with package names passed as an arguments. My client object was not in a package and hence the API failed.
    The 2nd argument is for client class path and must be present in a package.

    -Vivek

  9. September 10, 2010 at 18:11 by Nick

    Is this a bit outdated? We are using AMFConnection in Java Samplers for JMeter to generate load tests. Works great and pretty straight forward to use.

  10. September 15, 2010 at 15:48 by racsor

    Plugin for JMeter, visualizer deserialize response in format text.

  11. September 15, 2010 at 15:53 by racsor

    JMeter plugin visualizer for deserialize response in text.
    http://code.google.com/p/jmeter-amf-visualizer

  12. September 22, 2010 at 13:55 by oleg

    "Is this a bit outdated? We are using AMFConnection in Java Samplers for JMeter to generate load tests. Works great and pretty straight forward to use."

    Nick, could you please tell more about AMFConnection? Where can I download that plugin?

  13. September 24, 2010 at 14:15 by Ashxen

    Thanks a lot, article helps me very much. Just not forget download and add to your class path flex-messaging-core.jar and flex-messaging-common.jar-s to organize imports...

  14. October 9, 2010 at 16:49 by Pradeep Kumar Tiwari

    HI Ashxen,

    How do you managing the n numbers of virtual users.
    As in JMETER , Normally increasing number of threads increase the numbers of virtual users by giving the vuser names in a csv file and through csv data set conf or through user parameter we can use these users at run time.

    Now for JMETER with flex, I can generate n number of users.
    Now how will I incorportae these generated files for n number of users in JMETER?
    How to run the script for N number of users.

    Regards,
    Pradeep

  15. March 8, 2011 at 14:06 by Veeru

    HI,

    I tried to test my BlazeDS services using AMFConnection but its failing as because few of my services were using FlexCOntext,getFlexClient() which is coming as null when I use AMF Connection... Please help me to resolve this issue...

    Normal return types were working fine...

  16. April 5, 2011 at 19:02 by Derrick

    Rob,

    I'm curious as to whether any more work has been done on this.

    Thanks!

  17. August 14, 2011 at 02:39 by limscoder

    Rob,

    This method seems adequate for testing request/response RPC calls, but I need to test push notifications.

    I have a system where I need to test the latency of Flex Messages being produced and consumed by clients connected over streaming HTTP channels. Do you have any experience with such testing, or any ideas about how it may be accomplished in a way that replicates a real world scenario? Is this possible with JMeter or other testing tools?

  18. August 24, 2011 at 11:25 by Ravi Kishore

    Hi,

    I have reviewed the post. Can you please share the code files(AMFMessage and other) and also explain how to use them.

    Thanks in advance.

    Regards,
    Ravi Kishore.V

  19. September 25, 2011 at 16:01 by Ken

    In case it helps anyone who happens across this article, there is now a JMeter plugin that lets you record and edit AMF messages for use in load testing. The plugin eliminates most of the limitations mentioned in this article. You can even use it to test apps that have unique session ids embedded in the AMF.

    https://github.com/steeltomato/jmeter-amf

    Ken (Project Maintainer)

  20. July 10, 2012 at 22:11 by PMD-UBIK-INGENIERIE

    Hello,
    Just to mention our Company (UBIK-INGENIERIE) distributes an in-house Flex/Air AMF Apache JMeter plugin.
    You can simply, record, variabilize and replay AMF dialogs.

    See:
    http://www.ubik-ingenierie.com/-Solutions-

    Regards
    Philippe

Leave a Reply