Trifork Blog

Posts by Joris Kuipers

Twitter

Spring Insight plugin for the Axon CQRS framework

November 13th, 2012 by
(http://blog.trifork.com/2012/11/13/axon-insight-plugin/)

Introduction

In a previous blog post we introduced the Spring Insight module that’s part of SpringSource’s tc Server developer edition and gives you, well, insight into what’s taking up the time to process requests in your applications. Insight does this by instrumenting your application at load time using AspectJ: a single call to your application will result in a so-called trace consisting of various operations, and for each of these operations it’s measured how long it takes to execute. An operation can provide additional details: for example, when sending a query to a database using a JDBC Connection the actual SQL statement including prepared statement bindings will be stored in the operation.

Spring Insight is not a profiler: it doesn’t create operations for each and every little thing that happens, but only for ‘significant’ events, like a Spring-MVC controller invocation, the start of a transaction or communication with a database. These events are called ‘operations’. It does allow assigning operations to so-called endpoints. An endpoint could be a controller method, or a method on a class annotated with one of Spring’s stereotype annotations like @Service or @Repository for example. For these endpoints Insight can provide statistics across traces, so that you can measure the health of these endpoints during the lifespan of your application based on configurable thresholds.

Insight’s plugin model

Spring Insight consists of a core plus a set of plugins that are shipped with the tool. These plugins define the actual operations and endpoints that Insight knows about and exposes. One of the nice things about Spring Insight is that the plugins use a public API that’s not restricted to the built-in plugins: you can build your own plugins to teach the tool new tricks.

Although you could do this on a per-application basis to expose metrics relevant to your particular app, you wouldn’t usually write a dedicated plugin for that. Insight already exposes several application-level methods as operations if they’re part of your stereotype-annotated Spring beans, and you can use their set of annotations to expose additional application-specific operations and endpoints easily .

Plugins are much more useful for framework developers: their framework might contain several methods that would be interesting to expose as Insight operations or even endpoints to show users what the framework is doing and how long that takes. Earlier this year, that’s exactly what we did for the Axon CQRS framework that’s being developed within Trifork.

This blog briefly discusses the plugin’s implementation. All source code has been added to the Axon Framework itself and is available on Github.

Update:
After publishing this blog entry, VMware has contacted  us to host the source code for this plugin in their public GitHub repository, so that the plugin can be shipped out-of-the-box with the Spring Insight distribution. That means that the plugin sources are no longer found under the Axon repository. The link above has been updated to reflect this change.

Read the rest of this entry »

JVM memory management and performance tuning

June 22nd, 2012 by
(http://blog.trifork.com/2012/06/22/jvm-memory-management-and-performance-tuning/)

Performance and scalability are important topics for most enterprise applications that we build. Oftentimes we mostly consider connections to external systems (databases, message brokers, external web services) when we think about performance bottlenecks or performance and scalability optimization. This makes sense, as most enterprise applications spend a lot of their time performing (typically blocking) I/O. However, when you start to dig deeper and use tools like profilers and do thread dump and garbage collection analysis you’ll find that the actual code running in the Java VM can be the primary cause of severe performance and/or scalability issues under load.
Memory allocation can play a big role here: although garbage collection performance has increased a lot with every new version of the JVM, applications — esp. those with large heap sizes — can still suffer from full GC sweeps that always seem to happen at the worst possible moment and stop the world for several seconds.

To ensure that your application makes the most efficient use of the VM it’s running on it’s important to understand the interaction between your app and the VM, and also to understand the various tuning options that are available in the JVM that you happen to be using.

Tech Meeting with Azul Systems

During our next tech meeting we will have a speaker from Azul Systems as our guest to deliver a workshop seminar on JVM performance. As you may know, Azul Systems is a company that provides a custom Hotspot compatible JVM called Zing for systems that require very high and predictable performance. However, this workshop will not be a product sales pitch: instead, it will help you to understand the way JVMs perform memory management in general and garbage collection in particular, and how you can inspect the runtime behavior of your VMs in this respect. You’ll learn about how to properly measure statistics and avoid common pitfalls, and how to use the results for tuning purposes.
Obviously you’ll see how Zing compares to the standard JVMs used by most people in this respect, but even when you are not planning on migrating to a new JVM you’ll leave this session with a deeper understanding of Java memory management, how it affects your applications and how you make it work to your advantage.

So, when you’re interested, make sure to sign up for the meeting! It takes place on Thursday 5th July at our office in Amsterdam and will start at 4 PM. Attending is free and we’ll make sure that there’s pizza and beer to accompany the workshop.

Orange11 announces MongoDB partnership with 10gen

May 21st, 2012 by
(http://blog.trifork.com/2012/05/21/orange11-announces-mongodb-partnership-with-10gen/)

At Orange11 we’re very happy to be able to inform you that we’ve recently become an official 10gen MongoDB Service Partner. MongoDB, the leading document-based database solution that allows high-performance and high-scalability data access without many of the limitations imposed by traditional relational database systems.

More on MongoDB
If you haven’t heard of MongoDB yet, then apparently you haven’t been keeping up with the whole NoSQL movement. NoSQL is the term used to label a whole new category of data storage and retrieval solutions that vary widely in their underlying models and offered functionality, but all have in common that they are not based on the relational database paradigm that’s still predominant as the storage solution for enterprise applications. Types of NoSQL databases include key-value stores, column databases, document-oriented databases and graph databases; roughly speaking this list is ordered ascending in terms of the richness of the supported data structures and descending in terms of raw performance and ease of horizontal scalability. In the document-based family of NoSQL solutions, MongoDB has managed to become the nr. 1 solution used by both cutting-edge start-ups as well as big established companies.

That means that MongoDB uses a document that’s part of a collection as the basic unit of storage, instead of a row in a table like with an RDBMS. This document is modeled in BSON, a binary format for representing JSON documents. Documents that you store in any one collection typically share a similar structure, but do not have to conform to an exact schema. That means that it becomes very easy to change or expand the exact contents of your documents. This is a great feature: not only does it enable rapid development in an agile setting, where data model requirements are still fluid, but it also allows working with semi-structured data easily; e.g for storing and querying the results of JSON-based RESTful APIs without having to pre-process your data. Documents can be nested, so you can easily group related data (like an order with its items) in a single document, avoiding the need for what would be a JOIN in a relational database schema.

On the other hand, many tried and proven techniques familiar from the relational database world are offered by MongoDB as well. You can add indices to improve query performance, write operations can be guaranteed to be durable (but don’t have to be), read operations can be fully consistent (but again, don’t have to be if your performance and scalability needs are better served by using eventual consistent reads instead), etc.

Changes to documents can be modeled as atomic unit-of-works and are supported by a find-and-modify style of operations that in many cases prevents the need for techniques like optimistic locking on a whole document level. For more advanced operations you can even write map-reduce jobs that execute directly at the database level.

Working with MongoDB

MongoDB can run on Unix, OS X and Windows. It’s developed as an open source project, which has led to a lot of support for it in many different languages and frameworks. 10gen provides drivers for the major programming languages like Java, but you can find community supported drivers for many more languages. These drivers allow for programming directly to the MongoDB API where JSON-like constructs are mapped to the natively supported data types of your language. If you prefer something more high-level, for example to map Java objects to documents, there are various projects and frameworks providing support for that as well.

If you are a regular reader of this blog, you might be familiar with the Axon CQRS framework that’s developed by Orange11. As part of the work that’s underway for Axon 2.0, MongoDB support is added directly to the framework. This means you will be able to use MongoDB as an event and saga store, and that events can be serialized directly to BSON instead of to XML using XStream to achieve the best possible performance. This work is almost complete: for more details, you can visit these blog entries:

Using the Spring data project and the MongoDB adapter specifically

A MongoDB based Axon framework event store

About the partnership

As a Service Partner, Orange11 provides various services around MongoDB. As a full service solution provider we can make MongoDB is a part of the custom applications we create for organizations through integration with the data access layer, but we can also advice you about the deployment topology to use for your MongoDB servers to ensure high availability and scalability that meets your data demands. Being an official 10gen partner we have direct access to the people developing and supporting the product, thereby providing our customers with the safety of knowing that they’re not only benefiting from Orange11’s expertise but that they’re backed by support straight from the source whenever necessary.

Finally, 10gen is a silver sponsor of this week’s GOTO Amsterdam conference (co-produced by Orange11) and Alvin Richards, Senior Technical Director for 10gen/MongoDB in Europe will be a speaker on Thursday’s program. We have a joint booth at the event, so make sure to drop by if you’re attending to find out more about MongoDB and how Orange11 can help you to get the best results with this innovative new technology!

We are excited about this partnership and look forward to sharing our insights within the community and our customers. Just contact us if you want to know more or take a look at our website.

P.S. Check out the recently created MongoDB User Group for the Netherlands and join us to find out more.