Trifork Blog

Posts Tagged ‘Testing’

The Spock Framework

July 31st, 2014 by
(http://blog.trifork.com/2014/07/31/the-spock-framework/)

Groovy Logo

In one of my current projects, I started to play around with Groovy and was fascinated how groovy this language is. I got more curious about the way to perform testing and started googling around a little bit. My google research showed me several frameworks to test groovy applications. One of these frameworks is the Spock framework. Spock is created for testing Java and Groovy applications. It has a very clear syntax that is easy to read and it comes with a JUnit runner that ensures you can use it with any of your existing tooling. Furthermore, Spock combines the best features of proven tools like JUnit, jMock, and RSpec, and innovates on top of them. In the first part of the blog I am going to explain how you get started with the Spock framework and how a test method looks like. After that I am going to show how you can use Spock with the Geb framework and what the difference is between testing with Spock alone and Spock combined with Geb. In the end a conclusion is given.

Read the rest of this entry »

Reporting bugs for Dummies with Google Drive forms

January 8th, 2013 by
(http://blog.trifork.com/2013/01/08/reporting-bugs-for-dummies-with-google-drive-forms/)

There are bugs in every application. Even more so for applications that are in development, there is no question about it. If it is a given that bugs are present then the question is how do we find those bugs?

A simple answer is: look hard and look frequently. The harder you look, the more bugs you will find. This is usually a job for someone who specializes in Quality Assurance (QA). A QA-er should be an analytical and critical and should have the skills and experience to efficiently look for issues in software. If he or she has found a bug then it should be reported in a clear and efficient way. A good QA-er will of course know how to do this.

Read the rest of this entry »

Simulating a bad network for testing

July 3rd, 2012 by
(http://blog.trifork.com/2012/07/03/simulating-a-bad-network-for-testing/)

In a development environment, and often in the test and QA environments as well, we are thankfully blessed with a network that is for all intents and purposes infinitely fast, infinitely reliable and not shared with anyone else. Sometimes this causes you to miss a bug that only becomes apparent once your application has been released into the wild, where it has to deal with latency, packet loss and protocol violations.

To reproduce such bugs, it would be nice to have a network that is bad in a precisely controlled way. On a Linux machine, you can simulate one with netem. There is a wide range of possibilities with this tool, most of which are more useful to a network engineer than to a programmer or software tester, but I’ll give some simple examples, and demonstrate their effect with mtr.

First let’s take a look at the normal state of the network:

$ mtr -c 100 --report orange11.nl
HOST: cartman                    Loss%  Snt   Last   Avg  Best  Wrst StDev
1.|-- lobby                      0.0%   100    0.2   0.2   0.1   0.2   0.0
2.|-- backup1.orange11.nl        0.0%   100    2.4   4.0   2.0   9.1   1.7
3.|-- 10.0.0.30                  0.0%   100    5.0   4.0   2.2  10.4   1.6

That’s not too bad. Now we’ll simulate an average packet delay of 100 ms with a variability of 50ms, and a packet loss of 5%:

$ sudo tc qdisc add dev eth0 root netem delay 100ms 50ms loss 5%
$ mtr -c 100 --report orange11.nl
HOST: cartman                    Loss%  Snt   Last   Avg  Best  Wrst StDev
1.|-- lobby                      8.0%   100  129.3  96.6  50.2 147.8  26.0
2.|-- backup1.orange11.nl        3.0%   100  120.1 103.9  54.4 157.5  27.8
3.|-- 10.0.0.30                  4.0%   100   90.3 103.4  53.9 154.3  29.3

Pretty much as we would expect, the best ping times are around 50ms, the worst around 150ms, with an average around 100ms. The packet loss is a bit more random that I expected, but it should average out around 5% if we left mtr running for much longer than 100 cycles.

I can recommend trying out whatever project you are working on now, with a packet delay of 500ms to see if strange things happen in a reasonable worst case. It is important to realize that this tool can only shape the traffic that we’re sending, not receiving, so if the networked application is running on a different server, only your uploads and ACK packets should be affected.

You don’t have to reboot to get your network back to normal:

$ sudo tc qdisc del dev eth0 root

A great deal more can be done to shape your network traffic for better or for worse, such as rate control, prioritizing one destination over another, introducing packet corruption, duplication, or reordering etc, but these are outside the scope of this post.

A nice tutorial with examples can be found at the linuxfoundation.org and if you are interested in reading more about the background of network traffic control in the Linux kernel, I can recommend the Linux Advanced Routing & Traffic Control HOWTO.

Let me know how you get on won’t you?

Easier mocking with Mockito

August 13th, 2009 by
(http://blog.trifork.com/2009/08/13/easier-mocking-with-mockito/)

If you have been happily using Easymock for the past years, this blog post is for you. Because you need to upgrade to Mockito, the new kid on the block in the mocking universe. Why bother about what mocking framework you’re using to create your unittests? Because it will improve you tests, make them more expressive, and help you write better code. Read on if you want to know more.  Read the rest of this entry »

Automated functional testing with WebDriver

August 5th, 2009 by
(http://blog.trifork.com/2009/08/05/automated-functional-testing-with-webdriver/)

There is nothing nicer than having a functional test suite that checks if your application is still working as it should. It is even nicer to run these tests and see what is happening, while the tests are being executed. Two of the tools that give you this opportunity are Selenium RC and WebDriver.

In the project that I’m working on I use WebDriver to create functional tests. In this post I will try to give you an impression of why we switched from using Selenium RC to WebDriver and how we used WebDriver to test a Content Management System (CMS) and helped us making understandable tests.

Read the rest of this entry »

Performance testing a Flex BlazeDS application

July 14th, 2009 by
(http://blog.trifork.com/2009/07/14/performance-testing-a-flex-blazeds-application/)

In the past few years I’ve seen an increasing interest in Flex applications at our customers. I have to say that I’m not surprised about this trend. Not only do Flex applications generally look great, but they also provide a big boost to user experience. As a developer and architect I am also quite pleased with the programming model and extensive widget library. Sure, Adobe can still improve on a lot of things, but so far I have always worked with pleasure on Flex applications.
Read the rest of this entry »

Testing with factories

July 10th, 2009 by
(http://blog.trifork.com/2009/07/10/testing-with-factories/)

On July 1st, Jelmer added a very useful blog post about testing the database layer in which he suggested to use “insert statement” and “fixture” classes to provide a good way to insert test data into your database. I am also using that technique as I’m writing unit tests for the database layer. I have to say this really makes unit tests less work to write, more focussed and therefore more fun to write!

Although this technique solves the challenges we have in database layer tests, we might encounter similar challenges when writing unit tests for other layers (such as the service or frontend layer). This is because such tests often also require “test data” you need to create. Only this time the test data is not in the form of database rows, but in the form of Java objects. Such objects need to be instantiated in a valid state and often depend on other objects. I find this quite similar to inserting rows into database tables having not null columns and/or foreign key constraints.
Read the rest of this entry »

Testing the database layer

July 1st, 2009 by
(http://blog.trifork.com/2009/07/01/testing-the-database-layer/)

The database is an integral part of many applications and writing queries is often hard. For that reason I have always written integration tests for the data access objects or DAO’s that I use to access the database. The way I write my DAO tests has changed a lot over the years and in this post I’d like to document the ways in which it has changed and why.

Read the rest of this entry »

EasyMock: capturing arguments passed to mock objects

September 22nd, 2008 by
(http://blog.trifork.com/2008/09/22/easymock-capturing-arguments-passed-to-mock-objects/)

EasyMock2 is a library that provides an easy way to create on-the-fly mock objects based on interfaces. It can be used in combination with JUnit to create simple and powerful unit tests. Here’s an overview of the basic EasyMock flow, followed by a brief example that demonstrates a new feature in EasyMock 2.4: capturing arguments passed to MockObjects.

EasyMock2 basics

Lets assume we’re testing a ManagerImpl object that requires a Dao object to do some work. The manager passes a String object to the dao, and receives an Integer in return. Since we only want to test the Manager object in isolation, it seems a good idea to mock the Dao object.

The Dao interface:

interface Dao {
   Integer poseQuestion(String question);
}

The Manager interface + implementation:

interface Manager {
   String poseQuestion(String question);
}

With implementation:

class ManagerImpl1 implements Manager {
   Dao dao;
   public String poseQuestion(String question) {
      return dao.poseQuestion(question).toString();
   }
   // Dao setter etc
}

EasyMock basically does the following:

  • Create a mock object from the Dao interface:
    Dao mockDao = createMock(Dao.class);
  • Inject the mock object where an actual implementation would have been used:
    manager.setDao(mockDao);
  • “Record” all expected method calls to this mock object, and define appropriate responses. So-called argument matchers are used to put constraints on the expected arguments passed to the Dao
    expect(mockDao.poseQuestion(  // the expected call
    	and(	// merges the 2 matchers below
    	   isA(String.class)),	// value is a String?
    	   eq("What's the meaning of life?"))  	// value equals given String?
       .andReturn(new Integer(42))	// define the return value for this method call
       .once();	// This call is expected only once.
  • When all expected calls are recorded, put the mock object in “replay” mode
    replay(mockDao);
  • In replay mode, call the manager which in turn will call the mock object in its natural habitat
    String theCorrectAnswer = manager.poseQuestion("What's the meaning of life?");
  • Afterwards: verify if all expected calls to the mock object were actually made.
    verify(mockDao)
  • Capturing arguments

    Now let’s say the manager in the above example does not pass the String argument directly to the Dao. Instead, it uses a Wrapper object as an argument for the dao method calls. The Wrapper object contains the String argument from the manager call, and some additional arguments only used in the Dao (e.g. for logging).

    interface Dao2 {
       Integer poseQuestion(Wrapper questionWrapper);
    }
    
    class ManagerImpl2 implements Manager {
       Dao2 dao;
    
       public String poseQuestion(String question) {
          Wrapper wrapper = new Wrapper();
          wrapper.setQuestion(question);
          wrapper.setDate(new Date());
          // wrapper.setMore(...)
    
          return dao.poseQuestion(wrapper).toString();
       }
    
       // Dao2 setter etc
    }
    

    To fully test this ManagerImpl2 class, we’d like to know exactly what arguments are stored in the Wrapper object when it’s passed to the dao. The isA(Wrapper.class) matcher is of no help: checking the type of the Wrapper argument is not enough in this case. We want to check the specific values inside it.

    EasyMock 2.4 offers a new feature to do just that. An additional capture(Capture c) matcher was added. This Matcher does not actually match anything, but stores the object passed to the mock object in a Capture object. Calling the getValue() on this Capture object later on will return the original argument. In this example, capturing the Wrapper object as it’s passed to the dao allows us to check whether it still contains the String that was used as an argument in the manager call.

    A complete unit test for ManagerImpl2 would contain a method that looks something like this:

    @Test
    public void testPoseQuestion() {
       ManagerImpl2 manager = new ManagerImpl2();
       Dao2 mockDao = createMock(Dao2.class);
    
       manager.setDao(mockDao);
    
       Capture capturedArgument = new Capture();
    
       expect(mockDao.poseQuestion(
          and(
             // capture the argument for later use
             capture(capturedArgument),
             // check if it is a Wrapper, as expected
             isA(Wrapper.class)))
       .andReturn(new Integer(42))
       .once();
    
       replay(mockDao);
    
       // mockDao is used during manager call,
       // the actual wrapper object is captured
       manager.poseQuestion("What's the meaning of life?");
    
       verify(mockDao);
    
       // Check the captured argument: does it contain the original
       // question, or was it modified by the manager somehow?
       assertEquals("What's the meaning of life?",
                ((Wrapper) capturedArgument.getValue()).getQuestion());
    }
    

    The capture matcher allows for easy argument checking, without the necessity to implement additional mock arguments with custom equals/hashCode methods.

    See the EasyMock2.4 documentation for more examples, and more useful tricks.

EasyMock2 in a Nutshell

October 5th, 2005 by
(http://blog.trifork.com/2005/10/05/easymock2-in-a-nutshell/)

The essential tool for unit testing, EasyMock, has come to a new level. This blog entry introduces the new EasyMock2 release.
While it is still in pre-release stage, I do recommend using EasyMock2 for all your Java 5 projects.

To quote Tammo Freese (Project Founder) from the EasyMock project:

‘I plan no modifications other than bug fixes from this point.’

Read the rest of this entry »