Trifork Blog

Basic Health check library for your application

April 19th, 2012 by
|

For a few years now I have been running multiple Hippo projects. All the projects use multiple servers and services. The biggest one has over 15 servers in production. A standard hippo solution consists of site servers and cms servers. We also had some other servers for things like rss and integration. The complete landscape consists of apache/tomcat combinations, proxy servers and load balancers. All the servers are monitored using a special tool. In the beginning we made the load balancer call the a webpage. Later on we created pingers. Calling the pinger was easy for a load balancer and the proxy servers. By using a generic interface for a pinger we could also implement pingers to check ftp connections or external systems. Using the generic interface made it easy to create a custom monitor that just showed all the results of the pingers.

For new projects we wanted to have a similar solution. Therefore I decided to make a library available to create and interact with these pingers. The library consists of a generic interface and some default implementations.

Another part of the library is a version servlet that reads a manifest file to show you information about the actual artifact that is deployed.

In this blogpost I am going to explain the library and show you how to use it.

The goal for the library

The library must define an abstract for Ping executors. Ping Executors provided the implementation for checking the status of a system. The library also provides in a front-end component access to the ping executor over the web.

Structure of the solution

The different modules

The solutions consist of three main components:

    • API – contains the interfaces of the ping executors. You need this component if you want to create a program to use the ping executors.
    • Servlet – contains the base servlet to expose a ping executor over the web. It transforms the result of the pinger in HTML status codes, gives the choice to return html or json and provides a way to execute the basic, extended or thorough pinger.
    • Ping executors – contain a few ping executors that you can use in your project. (At the moment we only have a hippo ping executor, but more will follow)

The code base and maven

All the code is made available as open source. You can find the code in github and the artifacts are made available through maven central.

https://github.com/orange11/JavaHealthCheck

And the maven artifacts

<dependency>
    <groupId>nl.orange11.healthcheck</groupId>
    <artifactId>orange11-healthcheck-api</artifactId>
    <version>0.3</version>
</dependency>
<dependency>
    <groupId>nl.orange11.healthcheck</groupId>
    <artifactId>orange11-healthcheck-servlet</artifactId>
    <version>0.3</version>
</dependency>
<dependency>
    <groupId>nl.orange11.healthcheck</groupId>
    <artifactId>orange11-ping-executors</artifactId>
    <version>0.3</version>
</dependency>

The artifacts are made available to maven central using http://oss.sonatype.org/

Maven is also used to build the library. If you want to build the project yourself checkout the sourcecode from github and run mvn clean install.

The ping executor api

The api of the PingExecutor is fairly easy. It consists of 1 interface, the PingExecutor. The interface specifies 2 method types. One to obtain the name of the Ping Executor and another one to specify multiple execute methods. The execute methods facilitate in calling the execute for a specific level (Basic, Extended and Thorough). A generic execute method that excepts the level is also available.

public interface PingExecutor {
    String getName();
    PingResult execute();
    PingResult executeExtended();
    ThoroughPingResult executeThorough();
    PingResult execute(PingLevel pingLevel);
}

The Levels available and Result objects are discussed in the next section.

Levels and results

The PingLevel enum describes the three levels we support for a ping executor:

  • Basic – This should be as light weight as possible for the underlying system. Just to see if it is up to it.
  • Extended – A more extensive ping that means a more extensive health check of the system is performed.
  • Thorough – Returns data about the system that can help with understanding the state of the system.

The result in a ping is a PingResult. Containing the status code and a message. If you are doing the thorough ping, you receive a ThoroughPingResult. This class adds a Map with key/value pairs. The implementation can store any key/value pair information in this result together with additional information.

The PingResult contains a SystemStatus. The SystemStatus is an enum with the following values: OK, ERROR, AUTHENTICATION_ERROR, TIMEOUT_ERROR, MAINTENANCE.

The servlet as a client of the api

The implementations of the PingExecutor interface are pure java implementations. If you want to call the PingExecutor over the web you’ll need a way to expose the ping. The library provides the base for a servlet. The only thing you need to do is configure the PingExecutor you want to expose. You’ll get the ServletConfig object to do it. If you get a problem creating your PingExecutor you can throw the ServletException. The PingExecutor is obtained during the initialization phase of the servlet.

The PingExecutor requires a level for the ping. There are a few ways that you can configure this.

  1. Configure the servlet in the web.xml. The init-param with the name pinglevel is read. The value can be a number from 1 till 3 or the string representation of the PingLevel enum. (BASIC, EXTENDED, THOROUGH) If you do not provide a value, the Basic level is used.
  2. Add a parameter to the url with the name ping level. Again you can provide a string with he names as mentioned before or a number from 1 to 3.

Single request to backend

At certain moments the backend can become slow. That moment, requests stack up and you could have multiple threads dealing with ping requests. We do not want the ping requests to overload the server. Therefore the base servlet has a mechanism to return the current status when multiple pings are received and only one request is actually being handled. One could argue that servers get notified later about problems. But when multiple requests need to wait for each other you also have no information.

Html or Json

The servlet uses http status codes to reflect what is happening. You’ll receive a 200 when it is fine and the range of 500 when things are not fine. The result can be a piece of html, easy when looking at the status on your screen. When you are creating an application to do something with the ping results you can request json. There are two ways to request json.

  1. Add the Content-Type to the request headers with the value application/json.
  2. Add a parameter to the url of the request with the name type and the value json.

The following three images show you the json responses using the example that is also provided in the project. They are created using the Advanced Rest Client plugin in Google Chrome.

http://localhost:8085/ping?type=json&pinglevel=1

Screenshot thorough ping

http://localhost:8085/ping?type=json&pinglevel=2

Screenshot extended ping

http://localhost:8085/ping?type=json&pinglevel=3

Screenshot thorough ping

Pinging Hippo

The only PingExecutor that we have available at the moment is an executor that checks the Hippo repository. The code is heavily based on the PingServlet from hippo. I am not going to explain the internals in this blog post. I only want to explain the different levels of the ping. If you want more information please check the sourcecode.

Following the api, there are three different levels for the ping:

  • Basic – Tries to connect to a repository by reading a node.
  • Extended – Tries to connect to a repository by reading from a node and writing to a node.
  • Thorough – The same as the extended check plus it adds information about memory consumption to the result.

The following image shows a screen dump of the result of a thorough ping with the hippo ping executor

Screenshot thorough ping hippo

Next steps

We are using this library in a few projects now. Based on these projects we want to create more ping executors. I am planning on making an http ping executor, an ldap ping executor and an ftp ping executor. If you have other ideas feel free to mention them or create them yourself of course.

References

That is about it. References to documentation other sources are outlined below.

One Response

  1. April 20, 2012 at 08:57 by Erik van Oosten

    Hi Jettro,

    Nice work! Did you also look at Yammer’s metrics? I see many people moving to that library for metrics. It also has Healthcheck support which would correspond to the function of your library.

    Regards,
    Erik.