Trifork Blog

Category ‘Microservices’

Using Axon with PostgreSQL without TOAST

October 9th, 2017 by
(https://blog.trifork.com/2017/10/09/axon-postgresql-without-toast/)

The client I work for at this time is leveraging Axon 3. The events are stored in a PostgreSQL database. PostgreSQL uses a thing called TOAST (The Oversized-Attribute Storage Technique) to store large values.

From the PostgreSQL documentation:

“PostgreSQL uses a fixed page size (commonly 8 kB), and does not allow tuples to span multiple pages. Therefore, it is not possible to store very large field values directly. To overcome this limitation, large field values are compressed and/or broken up into multiple physical rows”

As it happens, in our setup using JPA (Hibernate) to store events, the DomainEventEntry entity has a @Lob annotation on the payload and the metaData fields (via extension of the AbstractEventEntry class):

For PostgreSQL this will result in events that are not easily readable:

SELECT payload FROM domainevententry;

| payload |
| 24153   |

The data type of the payload column of the domainevententry table is OID.

The PostgreSQL JDBC driver obviously knows how to deal with this. The real content is deTOASTed lazily. Using PL/pgSQL it is possible to store a value in a file. But this needs to be done value by value. But when you are debugging your application and want a quick look at the events of your application, this is not a fun route to take.

So we wanted to change the data type in our database to something more human readable. BYTEA for example. Able to store store large values in, yet still readable. As it turned out, a couple changes are needed to get it working.

It took me a while to get all the pieces I needed. Although the solution I present here works for us, perhaps this could not be the most elegant of even the best solution for everyone.
Read the rest of this entry »

Heterogeneous microservices

July 13th, 2017 by
(https://blog.trifork.com/2017/07/13/heterogeneous-microservices/)

Heterogeneous microservices

Microservices architecture is increasingly popular nowadays. One of the promises is flexibility and easier working in larger organizations by reducing the amount of communication and coordination between teams. The thinking is, teams have their own service(s) and don’t depend on other teams, meaning they can work independently, thereby reducing coordination efforts.

Especially with multiple teams and multiple services per team, this can mean there are quite a few services with quite different usage. Different teams can have different technology preferences, for example because they are more familiar with the one or the other. Similar different usage can mean quite different requirements, which might be easier to fulfill with the one or the other technology.

The question I’m going to discuss in this blog post, how free or constrained should technology choices be in such an environment?

Spaghetti freedom

If you’ve ever worked in a hectic startup environment where quickly building features is more important than clean sustainable code, you probably know how that ends. People just build things and it all becomes a big spaghetti mess. There is lots of technical debt..

The combination of freedom and high pressure means choices are short-term focussed and there is little attention to keeping things tidy.

Over-standardization

Large organizations can have the opposite policy, any project or change need to be approved by multiple boards, architects and committees. There are strict rules about exactly what technologies should be used.

This can mean there might be some shoe-horning to implement things with a sub-optimal technology, or over engineering because complex technology is used for simple problems.

This can also be de-motivating to engineers, as they’re not allowed to pick their favorite technology.

Inspire!

I think the right approach is a middle ground. Some standardization and preferred languages, frameworks and solutions are very helpfull to promote software being built the same way. This helps engineers working on new services, for example when switching teams or when onboarding new people.

I think this preferred way of working should evolve also and be open for discussion. New insights or new ways of building can be integrated if they prove beneficial.

In my opinion such a preferred approach should serve as a starting point for teams building new services and be open for discussion. Teams should try and build software in a common way, but when needed it should be fine to diverge and do things differently.

Concluding

I’ve shared my opinion – how can technology be standardized in larger microservice environments. How does this work in your company? What’s your insight on this topic?