Trifork Blog

Posts by Joris Kuipers

Twitter
« Older Entries

How to manage Database Migrations with Flyway?

December 5th, 2017 by
(https://blog.trifork.com/2017/12/05/how-to-do-database-migration-with-flyway/)

Joris Kuipers, CTO at Trifork, presented a webinar on some usage patterns for Flyway. You can find the recording on our Trifork YouTube channel.

Tools like Flyway address a common concern for many people, which quickly leads to questions on how to pick a tool and then apply it in the best manner for one’s particular situation.

In this blog post, Joris has summarized the Q&A session – he provides the readers with his answers and ideas for managing database migrations.

1. How does Flyway compare to Liquibase?

When I was choosing a DB schema migration tool 4 or 5 years ago, I’ve looked at both Liquibase and Flyway. In general, I’d say that Flyway is a bit more light-weight: Liquibase addresses some requirements that Flyway explicitly chooses not to support. These include support to define migrations declaratively (e.g. in XML) and then generate the correct SQL DDL statements for your particular DBMS and the support to generate migration rollbacks (countering operations).
Read the rest of this entry »

Tags: , , , , ,
Posted in Custom Development | No Comments »

How to send your Spring Batch Job log messages to a separate file

April 14th, 2017 by
(https://blog.trifork.com/2017/04/14/how-to-send-your-spring-batch-job-log-messages-to-a-separate-file/)

In one of my current projects we’re developing a web application which also has a couple of dozen batch jobs that perform all sort of tasks at particular times. These jobs produce quite a bit of logging output when they’re run, which is important to see what has happened during a job exactly. What we noticed however, is that the batch logging would make it hard to quickly spot the other logging performed by the application while also running a batch job. In addition to that, it wasn’t always clear in the context of what job a log statement was issued.
To address these issues I came up with a simple solution based on Logback Filters, which I’ll describe in this blog.

Logback Appenders

We’re using Logback as a logging framework. Logback defines the concept of appenders: appenders are responsible for handling the actual log messages emitted by the loggers in the application by writing them to the console, to a file, to a socket, etc.
Many applications define one or more appenders and them simply list them all as part of their root logger section in the logback.xml configuration file:

<configuration scan="true">

  <appender name="LOGSTASH" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
    <destination>logstash-server</destination>
    <encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
  </appender>

  <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>log/server.log</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <fileNamePattern>log/server.%d{yyyy-MM-dd}.log</fileNamePattern>
      <maxHistory>30</maxHistory>
    </rollingPolicy>
    <encoder>
      <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %mdc %-5level %logger{36} - %msg%n</pattern>
    </encoder>
  </appender>
  <root level="info">
    <appender-ref ref="LOGSTASH"/>
    <appender-ref ref="FILE"/>
  </root>

</configuration>

This setup will send all log messages to both of the configured appenders. Read the rest of this entry »

Posted in DevOps, From The Trenches, Java | 1 Comment »

Using Spring Session for concurrent session control in a clustered environment

April 8th, 2016 by
(https://blog.trifork.com/2016/04/08/spring-session-concurrent-session-control/)

For a long time, Spring Security has provided support to limit the number of sessions a single user can have concurrently. This prevents users from being logged in from many different devices at the same time, for example to ensure that they won’t share their credentials to a paid site with their friends and family.

My former colleague Quinten Krijger has blogged about this feature before. Note the last paragraph, which explains how this support is limited to single-node applications.

Although running on a single node may suffice for many applications, there are plenty applications running in a clustered environment that should be able to benefit from concurrent session control as well. As hinted in the aforementioned blog, this requires both implementing a custom SessionRegistry as well as ensuring that expiring a session is propagated to all nodes in the cluster.

This is exactly what I’ve done recently using Spring Session, a framework that allows you to take control over managing sessions using a shared external registry like Redis. In this post I’d like to walk you through the code, which can be found here: https://github.com/jkuipers/spring-session-concurrent-session-control

UPDATE:

Based on the code I wrote for this blog I’ve opened a pull request for Spring Session. That request is scheduled for inclusion in Spring Session 1.3, but the code works just fine with the upcoming 1.2 release and removes the limitation of not providing an expiry notification after exceeding the maximum number of sessions.

Read the rest of this entry »

Tags:
Posted in Custom Development, Spring | 5 Comments »

Spring-AMQP and payload validation: some notes from the trenches

February 29th, 2016 by
(https://blog.trifork.com/2016/02/29/spring-amqp-payload-validation/)

It’s been a while since I’ve written one of our from-the-trenches blogs: that’s mostly because I’ve been very busy in those trenches developing systems for our customers.

This week I completed a Spring Boot-based microservice which is responsible for interacting with some 3rd party SOAP service: its own clients communicate with it by sending request message over RabbitMQ, and the service then sends back a response to a response queue after handling the SOAP response message.

Of course I used Spring-AMQP to build this service. Spring-AMQP supports a nice annotation-based listener programming model, based on Spring’s generic Message support.
That allows writing listener methods like this:

@RabbitListener(queues = REQUEST_QUEUE) 
public DeclarationResponse submitDeclaration(DeclarationRequest request) { 
  // handle the request and return a response 
}

The request parameter here is the result of converting the incoming AMQP message using a Spring-AMQP MessageConverter, after which it is considered to be the payload of the message (even when headers are used in the conversion as well).

The request messages that the clients send have some required fields: without those fields, the service can’t make the SOAP calls. While reading the RabbitListener JavaDoc I noticed that Spring-AMQP allows you to apply validation to message payload parameters by annotating it. When using this, you also have to add the @Payload annotation (which is optional without validation if your method doesn’t have any other arguments), so the result looks like this:

@RabbitListener(queues = REQUEST_QUEUE)
public DeclarationResponse submitDeclaration(@Valid @Payload DeclarationRequest request) { … }

By the way, Spring’s own @Validated (even as a meta-annotation) and in fact every annotation whose name starts with “Valid” are supported for this purpose as well.

Now we can add some JSR-303 Bean Validation annotations to the fields in our DeclarationRequest, like @NotNull, to express our validation constraints.

Read the rest of this entry »

Tags: , , ,
Posted in Custom Development, From The Trenches, Spring | 2 Comments »

Booting your Microservices Architecture with Spring and Netflix: the aftermath

November 26th, 2015 by
(https://blog.trifork.com/2015/11/26/booting-your-microservices-architecture-with-spring-and-netflix-the-aftermatch/)

On 25 November Trifork hosted a webinar in which I gave a short overview of Spring Cloud and its support for the Netflix OSS stack, focusing on Spring Cloud Config and the support for Netflix’s Eureka, Ribbon and Hystrix.

We’ve been investigating this stack over the last couple of months and are using parts of it in production already: we have found that a lot of the common concerns that you need to tackle as you’re moving into a distributed systems architecture are nicely covered and, in many cases, even abstracted by the Spring Cloud platform. In a typical Spring fashion, this allows you as an application developer to focus more on your business logic while letting the framework handle the concerns related to things like accessing shared configuration, working with service registries, handling failing downstream services, etc.

This blog provides you with background info to accompany the webinar, which has been recorded and can be found on our YouTube channel.
The code has been published on GitHub, as well as the accompanying config repository, in case you’d like to code along with the video.

Read the rest of this entry »

Posted in Custom Development, Spring | Comments Off on Booting your Microservices Architecture with Spring and Netflix: the aftermath

Declarative multi-tenant security with Spring Security and Spring-MVC

September 5th, 2013 by
(https://blog.trifork.com/2013/09/05/multi-tenant-security/)

It’s been a while since our last ‘from the trenches’ entry, and as I’ve found I am better at authoring blogs than convincing colleagues to do the same I figured I’d write you another installment. This time I’d like to focus on an easy yet powerful approach that we used to secure a multi-tenant Spring-MVC application using Spring Security and its support for annotation-based declarative authorization.

Introduction

If you’re developing enterprise web applications, then you have certainly applied some form of security to your apps. In some cases it suffices to come up with a number of roles that you can assign to (groups of) users and to perform authorization based on that. However, in many cases that’s not enough and the concept of data access control comes into play: only users that are somehow related to the data they’re trying to work with should be allowed to access that data. This can be through direct ownership, access control lists, some temporary relation like a doctor-patient treatment relationship, etc. A common requirement in multi-tenant applications, where a single application instance is used by people from different organizations whose data should be strictly separated, is that data should only be accessible by people who work for the organization that that data belongs to.

In this blog we’ll show you one approach that we used to implement this with Spring Security.

Read the rest of this entry »

Tags: ,
Posted in Custom Development, From The Trenches | 2 Comments »

Adding user info to log entries in a multi-user app using Mapped Diagnostic Context

June 6th, 2013 by
(https://blog.trifork.com/2013/06/06/adding-user-info-to-log-entries-in-a-multi-user-app-using-mapped-diagnostic-context/)

Have you ever been in the situation where you were looking at one of your production log files and had a hard time seeing what log entries belonged to what requests? In a multi-user web application many requests are handled in parallel, so without additional context it becomes almost impossible to see how a single user is interacting with your application.

Similarly, when someone reports a problem with the application you can’t really tell what to look for: how do you know that some error in the log was actually caused by the user reporting the issue?

The good news is that there’s an easy solution to this problem, that has been around for a long time already and is very easy to add to your existing applications: use a Mapped Diagnostic Context

MDC explained

By now, you’re either saying “duh, I’ve been doing that for ages” or “what the hell is an MDC?”. In my experience as a consultant, a surprising number of people fall in the latter category; if you do as well, then this new “from the trenches” blog is for you.

Read the rest of this entry »

Posted in Custom Development, From The Trenches | Comments Off on Adding user info to log entries in a multi-user app using Mapped Diagnostic Context

From the trenches: Rendering Twitter Bootstrap form inputs for Spring-MVC with FreeMarker macros

May 27th, 2013 by
(https://blog.trifork.com/2013/05/27/bootstrap-spring-mvc-form-input-freemarker-macros/)

This is the first entry in a new series of what we hope will become a regular appearance on our Trifork blog: “from the trenches”, with blogs that explain in a no-nonsense just-the-tech fashion how we applied certain tools, technologies, libraries and frameworks in our own projects to solve real-world challenges that we faced while building solutions for our clients. We hope that by sharing the solutions we came up with, we can provide you with some examples and ideas that you can apply to your own projects — with the confidence of knowing that these techniques have proven themselves in real production systems.

We are also open to your suggestions on what technologies you might want us to cover. So don’t be shy and just drop us a note if you have any ideas.

In this first blog I’d like to cover how we used FreeMarker to build a small library of macros that make it very easy and effort-free to work with Twitter Bootstrap-based forms in a Spring-MVC application.

TL;DR: we’ve built a set of FreeMarker form macros for Spring-MVC apps with a Twitter Bootstrap-based frontend that allow one-liners in your form templates to render full Bootstrap-based HTML structures for various input types that include internationalized labels and rendering of binding and validation errors while allowing passing in custom attributes and nested contents. It’s awesome.

Read the rest of this entry »

Posted in Custom Development, From The Trenches, Frontend, Java | Comments Off on From the trenches: Rendering Twitter Bootstrap form inputs for Spring-MVC with FreeMarker macros

Putting the pedal to the Mongo metal

March 1st, 2013 by
(https://blog.trifork.com/2013/03/01/putting-the-pedal-to-the-mongo-metal/)

At Trifork we’re always looking to get the best performance out of our systems. As a 10gen partner that means that we also try to squeeze the most out of our MongoDB deployments in terms of read and write throughput. Experience has shown that it matters greatly whether those deployments are performed on dedicated hardware or on virtual machines: especially having enough and fast RAM and disk IO can make all the difference.

If you’re interested in reading up on this topic, make sure to check out this article on SoftLayer’s blog. It shows how hosted MongoDB deployments can perform when given the appropriate hardware, and how having a hosting provider that understands the technology running on their infrastructure can help you to achieve the performance that your applications require.

If you’re in The Netherlands and want to learn more about MongoDB, whether you’re a newbee or an experienced user already, you should sign up for  the upcoming Mongo User Group meetup that’s taking place coming Tuesday as well: SoftLayer will be there to tell more about their hosted MongoDB offering, 10gen’s Alvin Richards will talk about what’s planned for the upcoming releases of MongoDB (spoiler: it includes search!) and we’ll have Open Space sessions where you can decide for yourself what topics you’d like to see covered. Hope to see you there!

Posted in MongoDB, NoSQL | 2 Comments »

Authenticating Dutch organizations via eHerkenning

February 21st, 2013 by
(https://blog.trifork.com/2013/02/21/authenticating-dutch-organizations-via-eherkenning/)

Introduction

In The Netherlands, citizens can interact with digital government services using a central username and password through an authentication scheme called DigiD. This helps these services to hook into a central registry of users, thus providing them with a single identity corresponding to a single username and password. DigiD is a widely spread and well known authentication system that people use to file their taxes, interact with their local government etc.

The interesting challenge comes when one can offer digital services to organizations rather than individuals. From a business perspective, when people work for a certain organization they also interact with government services but do that on behalf of their organization, not on their own account. Also in time, people might switch jobs and therefore represent different organizations over time.
To deal with this issue, another national authentication scheme has been created that isn’t that well-known yet but is quickly gaining popularity: eHerkenning (meaning e-Recognition in Dutch).

eHerkenning overview

With eHerkenning, the idea is that organizations arrange accounts for users that represent them with one of the available eHerkenning brokers. Users can then authenticate with any system that offers eHerkenning integration. Those systems will receive a unique identifier for the user after a successful authentication attempt, as well as an organization ID that includes the registration number for the Dutch Chamber of Commerce. This allows government services to verify that users are truly acting on behalf of the organization they claim to represent. Authentication can be username/password based, but eHerkenning supports higher degrees of security as well by offering services with different security levels. That means that depending on the desired security level, something like 2-factor authorization with SMS or even based on PKI certificates handed out only face-to-face to the users involved can be required.

On the back-end, eHerkenning makes use of open security standards like SAML, on top of which it defines a custom profile. Initially the possibility to offer services that integrate with eHerkenning was restricted to government organizations, but this year the system is opened up for commercial services wishing to offer this ease of authentication through a central system as well.

eHerkenning for Ascert SMART 2.0

Trifork Amsterdam is delivering the new version of a system for Ascert (an organization in the asbestos removal branch), which in particular focuses on inventories of asbestos sources found on site at construction projects, called SMART 2.0. This application allows users from all SC-540 licensed organizations (i.e. organizations that are allowed to produce official asbestos inventory reports) to enter projects with one or more asbestos sources which are classified based on the user’s input. The input, classification result and working instructions for the removal company are then included in a report for the project’s asbestos sources. Other interested organizations, like city councils or asbestos removal organizations, can also enter sources but are not allowed to produce official reports.

The owner of the SMART 2.0 application is the Ascert foundation. Part of their requirements for this rebuild of their current application was authentication based on eHerkenning. Trifork has successfully added eHerkenning support to the SMART 2.0 application by integrating a Java adapter offered by the chosen eHerkenning broker with Spring Security, the open source framework used in most of our applications to provide authentication and authorization services. Since the information available after successfully authenticating with eHerkenning is limited to a meaningless user ID and the organization ID, users are required to complete their profile by entering their names and email addresses after logging in for the first time. The first user of an organization currently needs to update the organization profile with relevant details as well; if desired, future releases could easily automate this by integrating with a third-party web service that offers this data based on the Chamber of Commerce identifier that’s part of the organization ID.

Authorization, i.e. determining who is allowed to access what functionality and data, is still the job of the service implementation. Fortunately Spring Security enforces a very strict distinction between authentication and authorization, so adding an authentication mechanism like eHerkenning doesn’t affect the way that authorization is performed. This means that support for eHerkenning can be added to existing applications on demand with relatively little time and effort required.

Conclusion

While eHerkenning is not yet as widely adopted as something like DigiD, it’s expected that more and more government services will offer or even require it in the near future as the way to let users acting on behalf of other organizations authenticate themselves.

Trifork is now able to offer eHerkenning as one of the supported authentication mechanisms in our custom solutions, either exclusively or in addition to other mechanism like form-based login pages. Please contact us if you’d like more information about the options of using eHerkenning for your online services!

A Dutch version on this post Identificeren van bedrijven via eHerkenning is available on our website.

Tags:
Posted in Business | Comments Off on Authenticating Dutch organizations via eHerkenning

« Older Entries