Trifork Blog

Posts Tagged ‘Axon Framework’

The potential of Machine Learning with the Axon Framework

September 20th, 2018 by
(https://blog.trifork.com/2018/09/20/the-potential-of-machine-learning-with-the-axon-framework/)

Machine Learning creates lots of value in the business processes, if and when applied properly and the right data is available. These days a lot of data is already held within organisations. It is common to see this data unused, with possibilities and insights unseen. While it can be due to limitations of modelling and pattern recognition, we often see practical problems with data quality and availability setting the actual limit. In the recent years, development in this area has increased drastically, and with it, the possibilities!

Axon is a popular lightweight framework used by a growing number of companies which enables you to apply sound architectural principles like DDD, event sourcing and CQRS, and helps you to build less complex and better maintainable applications.

Using these principles, the Axon Framework promises scalability, agility and smooth integration with external systems. As a side effect, they also provide a reliable trail of events leading to a system state. By storing these in an event store, it enables perfect auditing capabilities for the past. This same detailed data also enables you to apply Machine Learning techniques to look forward and get precise predictions of the future

Organisations with all kinds of challenges and optimisation issues now have the opportunity to Forecast user interest and prevent churn, detect fraud right when it happens, and to cluster information to significantly improve recommendations.

Three main principles of Axon is aggregates, repositories and events.

An aggregate is an entity or group of entities that are always kept in a consistent state. For example, a “Contact” aggregate could contain two entities: Name and Address.

A repository is the mechanism that provides access to aggregates. There are two kinds of repositories, ones that save the current state of the aggregate and those that store the events of an aggregate (the Event Store).

An event is anything that has happened in the sense of business. For example, the action of creating a contact in an address book, would result in an event. Importantly, updating this contact, would also result in an event

When developing scalable applications, Axon helps you to focus on business logic. When the business logic changes over time, the original events are not touched but an upcaster maps the new understanding of the world on events before they are pushed to event listeners.

Imagine now, we can use all the events that have happened and stored in the Event Store, for different purposes.

Time series forecasting
Time series forecasting is an important area of Machine Learning which can solve many prediction problems that involve a time component. The dataset for the time series forecasting is a sequence of observations taken sequentially in time, and the goal is to eventually forecast the future values of a series based on this historical dataset. Considering the fact that all the events stored in the Event Store have the time component, we can leverage from the core structure of the Event Store in the Axon Framework and forecast the next trend or predict the next action.

A concrete use-case could be tracking the behaviour of a user on a web-shop, and finding out what actions leads to the purchase an item

Anomaly detection
Another example can be anomaly detection which is a technique used to identify unusual patterns that do not conform to the expected behaviour. It has many applications in business, such as intrusion detection in network traffic, spotting a tumor in an MRI scan, fraud detection in bank transactions and fault detection in operating environments.

Thanks to the Axon Framework, all the events that have been stored in the Event Store collectively help us in detecting what is normal, and hence what is an anomaly. An example use case could be analysing the events from an authentication system, and flagging when someone fails to login too many times, or has attempts from many different IPs.

Better ML Models
We can benefit from the rich and structured data stored in the payload of each event. High quality, structured data, plays a major role in the validity and accuracy of the Machine Learning models generated to solve a business problem.

In the Axon Framework, each event contains a payload which is the data being used in the event. By querying the collection of data related to events, we are able to create a rich dataset of the event’s data which enables us to make a relevant Machine Learning models.

For instance, we could query all the prescriptions which have been given to the patients with dementia. With that information, we were able to generate the next prescription for the patients with similar issues. Basically, we found that it was straightforward to start applying Machine Learning models for recommendation, prediction and recognition on the data stored in terms of events.

Integration
Since our models are based directly on the events that the system generates, it is easy to integrate them and start providing real business value. Examples of integrations would be providing live recommendations, alerting and in general helping decision making. With Axons focus on easy integration, we can even enable the models to take actual decisions.

Conclusion
These examples lead to the conclusion that events from a system built around the Axon Framework are a perfect starting point for Machine Learning. Of course we have a precondition for business value: Understanding the business challenges and being able to integrate the ML solution within the Business processes is paramount to success. If these preconditions are met, applying ML techniques to Axon based systems provide real valuable insights and optimisation possibilities, helping you to solve advanced business challenges.

 

More information about Machine Learning on Axon can be found here: https://trifork.com/ml4axon/

 

Exposing asynchronous communication through a synchronous REST API with Spring 5

April 13th, 2018 by
(https://blog.trifork.com/2018/04/13/exposing-asynchronous-communication-through-a-synchronous-rest-api-with-spring-5/)

On my current project, we opted not to use REST for the communication between our services. Instead we make use of AxonIQ’s AxonHub, which acts as a specialized message broker. Messages can be of three types:

  • Command – You want to change something
  • Event – You want to inform others of something that happened
  • Query – You want to know something

The communication is asynchronous and we also have to deal with eventual consistency. If we would create an order by sending a CreateOrderCommand, this order would result in various events which update the state of the Order. We then need to send a query, of which we also receive the result asynchronous.

Our web and mobile frontend communicate with a microservices backend through a REST API. When the user creates an order (clicks the ‘buy’ button), they expect a result immediately. For both frontends, an API which models this user experience closely makes the most sense. This means that we want a more synchronous API, where we send a ‘create order’ request and immediately receive a response.

We implemented this using a small REST facade which translates our asynchronous communication in the backend to the synchronous communication for the frontend. This is a good use case for Spring 5’s reactive Webflux and Project Reactor. Using Project Reactor’s reactive API makes it possible to combine multiple asynchronous calls and operate on their result. Webflux handles the conversion of the reactive types (Mono, Flux) to REST responses. It optimizes the use of threads; by writing non-blocking code, we can reuse threads between asynchronous calls for handling other requests.

Diagram 1 gives us an overview of this approach.

Diagram 1

 

Implementation

Let’s have a more detailed look at the code for the create order example. Listing 1 shows the (slightly) simplified implementation of our REST controller method.

@PostMapping
public Mono<ResponseEntity<OrderResponse>> createOrder(CreateOrderRequest request) 
{
 CreateOrderCommand command = CreateOrderCommand.fromRequest(request); //1

 return this.commandGateway.send(command) //2
  .flatMap(id -> queryGateway.send(new FindOrderSummaryQuery(id))) //3
  .retryWhen(errors -> errors.delayElements(Duration.of(100, MILLIS)) //4
  .take(10)).concatWith(Mono.error(new RuntimeException())).next() //5
  .onErrorReturn(new OrderResponse(orderID, OrderStatus.CREATED)) //6
  .map(orderResponse -> ResponseEntity.ok().body(orderResponse)); //7
}

Listing 1

We first create a command out of the REST request (line 1). A command is a message with the specific intent to change something in our domain. In this specific case, we want to create a new order.

After creating the command, the two asynchronous calls we make are:

Mono<String> id = this.commandGateway.send(command);

and:

Mono<OrderResponse> orderResponse = queryGateway.send(new FindOrderSummaryQuery(id));

Both calls return a single value by using a Mono. A Mono is a reactive type, comparable to the Java’s CompletableFuture. It has zero or one element and can represent an error. As with all reactive types, the value (or error) is delivered over time.

The second call takes the result of the first call as its input. We need the returned id of the command to query for the order. We use the flatMap operator to achieve this (line 3). The flatMap takes the asynchronous result of call 1 and passes this as a parameter to the lambda of call 2. The callback version can be seen in Listing 2: notice the nested lambda, which makes the code complex and less readable.

this.commandGateway.send(command, id -> {
  queryGateway.send(new FindOrderSummaryQuery(id));
});

Listing 2

There is a delay between sending the command and being able to query the result. When the created order cannot be found (e.g. it isn’t create yet or something has gone wrong), an exception is thrown. In this chain, this is represented as a Mono.error(throwable). We use the retryWhen method to retry the query (line 4). We do this 10 times with a delay of 100 ms. When we still don’t get a result, we throw an error (line 5). We don’t expose the error to the client, but pass an OrderResponse with the id and status CREATED (line 6). The client can then query the status of the order later by using this id. This is a form of graceful degradation.

Finally we map the order response from the query to a response entity which can be returned by Spring. Spring actually subscribes to this whole chain and sends out the REST response for us.

Conclusion

Spring 5 and Project Reactor allow us to handle asynchronous communication with concise and readable code. We can do retries, error handling and the combination of multiple asynchronous calls in just a few lines.
The integration of Webflux with Project Reactor allows the use of reactive paradigms in a REST controller. Webflux uses an asynchronous approach. While we wait for a backend query to return, we don’t block the main thread. this allows it to be used for other requests.
Our specific use case is a good example of one of the applications of Spring 5 Webflux and Project Reactor.

Using Axon with PostgreSQL without TOAST

October 9th, 2017 by
(https://blog.trifork.com/2017/10/09/axon-postgresql-without-toast/)

The client I work for at this time is leveraging Axon 3. The events are stored in a PostgreSQL database. PostgreSQL uses a thing called TOAST (The Oversized-Attribute Storage Technique) to store large values.

From the PostgreSQL documentation:

“PostgreSQL uses a fixed page size (commonly 8 kB), and does not allow tuples to span multiple pages. Therefore, it is not possible to store very large field values directly. To overcome this limitation, large field values are compressed and/or broken up into multiple physical rows”

As it happens, in our setup using JPA (Hibernate) to store events, the DomainEventEntry entity has a @Lob annotation on the payload and the metaData fields (via extension of the AbstractEventEntry class):

For PostgreSQL this will result in events that are not easily readable:

SELECT payload FROM domainevententry;

| payload |
| 24153   |

The data type of the payload column of the domainevententry table is OID.

The PostgreSQL JDBC driver obviously knows how to deal with this. The real content is deTOASTed lazily. Using PL/pgSQL it is possible to store a value in a file. But this needs to be done value by value. But when you are debugging your application and want a quick look at the events of your application, this is not a fun route to take.

So we wanted to change the data type in our database to something more human readable. BYTEA for example. Able to store store large values in, yet still readable. As it turned out, a couple changes are needed to get it working.

It took me a while to get all the pieces I needed. Although the solution I present here works for us, perhaps this could not be the most elegant of even the best solution for everyone.
Read the rest of this entry »

Axon from the trenches: how to keep your code compatible with legacy events and Sagas

June 8th, 2015 by
(https://blog.trifork.com/2015/06/08/axon-from-the-trenches-how-to-keep-your-code-compatible-with-legacy-events-and-sagas/)

Imagine you’re using Axon to run an event sourcing application. Your production event store might contain millions of events, created in various versions of the application. Wouldn’t it be nice to know for sure that the latest version of your application plays nicely with all your production events and Sagas, including those from previous versions? Well, you can check for that, and it is fairly easy. Read the rest of this entry »

CQRS as a Junior Developer

April 21st, 2015 by
(https://blog.trifork.com/2015/04/21/cqrs-as-a-junior-developer/)

Fresh from university, searching for a development job, having one within two weeks. And at an interesting company at that. That is what happened to me about a year ago. So, you could (and probably should) call me a junior developer. After a month of trainings and new experiences I was put on my first big project. Together with a senior colleague we were assigned to build the new roadside assistance application at the ANWB. The app was planned to work with Axon at the core and through this I came in touch with CQRS

Since I didn’t know squat about the ‘Command-Query Responsibility Segregation’ concept and soon had to work with it, I dove into a multitude of sites, blogs and wikis about the topic for self-study. That it separates the responsibility of commands and queries is quite obvious from the definition itself. And that this leads to scalability options since the command or the query side can be optimized to the system, made sense. Also, that it simplifies the creation of the domain model since there can be a focus on either the command or query side was quite clear through the information I read.

Read the rest of this entry »

New features in Axon Framework 2.1

February 13th, 2014 by
(https://blog.trifork.com/2014/02/13/new-in-axon-2-1/)

Recently, Axon Framework 2.1 has been released. It comes packed with improvements and some exciting new features. In this post, I’ll briefly iterate of what’s new in this version.

Furthermore, we have also scheduled a few workshops and trainings.

Read the rest of this entry »

AFAS’ CIO Rolf de Jong hosts Panel Discussions at Axon Seminar

February 12th, 2013 by
(https://blog.trifork.com/2013/02/12/afas-cio-rolf-de-jong-hosts-panel-discussions-at-axon-seminar/)

axon2_banner

On February 28th 2013, Trifork will organize the Axon 2 launch seminar. During this seminar, visitors will be introduced to CQRS and Axon Framework, of which version 2 was released just a few weeks ago. The seminar will be an afternoon packed of technical insight, case studies and panel discussions, whereby we look forward to a number of informative and interactive sessions. Rolf de Jong, CIO of AFAS ERP Software, has accepted our invitation to host the panel discussions where a team of experts will share their thoughts on CQRS, Axon Framework and software development in general.  Read the rest of this entry »

Axon Framework 2.0 Released!

January 22nd, 2013 by
(https://blog.trifork.com/2013/01/22/axon-framework-2-0-released/)

After laying the ground work for this release about a year ago, we now proudly announce the next major release of Axon Framework! The 2.0 release is a big step forward compared to the previous version, as it contains a thorough restructuring of components, making it even easier to build scalable and extensible applications.
Read the rest of this entry »

Axon Framework 2.0-rc1 released

November 30th, 2012 by
(https://blog.trifork.com/2012/11/30/axon-rc1-released/)

Axon 2 has hit an important milestone today, as we have published the first Release Candidate. This release marks a major milestone towards the final 2.0 release, expected early January. All features on the roadmap have been implemented in this release. The coming weeks, these features will be refined and fine-tuned for optimal performance and reliability to ensure Axon 2.0 helps meet the demands applications face today.

Read the rest of this entry »

Basic Axon Framework sample using vert.x and angular.js

November 27th, 2012 by
(https://blog.trifork.com/2012/11/27/basic-axon-framework-sample-using-vert-x-and-angular-js/)

Some people on the Axon Framework mailing list were asking for more basic samples, preferably using something like vert.x. Since I am familiar with Axon as well as vert.x I thought I could help out. Therefore I have created a very basic application based on vert.x using the Axon Framework. The application is all about maintaining a list of TODO items. These items are shared between all connected web browsers. You have to be able to create a new TODO item and you have to be able to mark them as completed.

During my visit of Devoxx I learned about a front-end framework called AngularJS. This is a very nice addition to the vert.x technology stack. Since I always use Bootstrap for my front-end work, the front-end should look familiar to a lot of you. The following images shows the end result of the application. If you want to know how I created the sample, read on.

ScreenShotAxonTodoList Read the rest of this entry »