Joris Kuipers, CTO at Trifork, presented a webinar on some usage patterns for Flyway. You can find the recording on our Trifork YouTube channel. Tools like Flyway address a common concern for many people, which quickly leads to questions on how to pick a tool and then apply it in the best manner for one’s particular […]
Using Axon with PostgreSQL without TOAST
The client I work for at this time is leveraging Axon 3. The events are stored in a PostgreSQL database. PostgreSQL uses a thing called TOAST (The Oversized-Attribute Storage Technique) to store large values. From the PostgreSQL documentation: “PostgreSQL uses a fixed page size (commonly 8 kB), and does not allow tuples to span multiple […]
Kibana Histogram on Day of Week
I keep track of my daily commutes to and from the office. One thing I want to know is how the different days of the week are affecting my travel duration. But when indexing all my commutes into Elasticsearch, I can not (out-of-the-box) create a histogram on the day of the week. My first visualization […]
Smart energy consumption insights with Elasticsearch and Machine Learning
At home we have a Youless device which can be used to measure energy consumption. You have to mount it to your energy meter so it can monitor energy consumption. The device then provides energy consumption data via a RESTful api. We can use this api to index energy consumption data into Elasticsearch every minute […]
Heterogeneous microservices
Heterogeneous microservices Microservices architecture is increasingly popular nowadays. One of the promises is flexibility and easier working in larger organizations by reducing the amount of communication and coordination between teams. The thinking is, teams have their own service(s) and don’t depend on other teams, meaning they can work independently, thereby reducing coordination efforts. Especially with […]
How to send your Spring Batch Job log messages to a separate file
In one of my current projects we’re developing a web application which also has a couple of dozen batch jobs that perform all sort of tasks at particular times. These jobs produce quite a bit of logging output when they’re run, which is important to see what has happened during a job exactly. What we […]
Machine Learning: Predicting house prices
Recently I have followed an online course on machine learning to understand the current hype better. As with any subject though, only practice makes perfect, so i was looking to apply this new knowledge. While looking to sell my house I found that would be a nice opportunity: Check if the prices a real estate […]
Simulating an Elasticsearch Ingest Node pipeline
Indexing document into your cluster can be done in a couple of ways: using Logstash to read your source and send documents to your cluster; using Filebeat to read a log file, send documents to Kafka, let Logstash connect to Kafka and transform the log event and then send those documents to your cluster; using […]
Public Elasticsearch clusters are being held ransom
Last week several news sites and researchers reported that Elasticsearch clusters that are connected to the internet without proper security are being held ransom. You can use shodan.io to search for Elasticsearch clusters: https://www.shodan.io/search?query=port%3A9200+json&language=en. The first hit is actually a cluster that is ‘infected’: There are some secured clusters as well: But the default ‘root’ account with […]
Handling a massive amount of product variations with Elasticsearch
In this blog we will review different techniques for modelling data structures in Elasticsearch. A project case is used to describe our approach on handling a small sized product data set with a large sized related product variations data set. Furthermore we will show how certain modelling decisions resulted in a 1000 factor query performance gain! […]