Skip to main content

You are currently viewing the Trifork Blog, to view our full website please go to Trifork.com

Refactoring from Elasticsearch version 1 with Java Transport client to version 6 with High Level REST client

Every long running project accrues technical debt. It may be that the requirements today have evolved in a different direction from what was foreseen when the project was designed, or it may be that difficult infrastructure tasks have been put off in favor of new functionality. From time to time, you need to refactor your […]

Read More

Smart energy consumption insights with Elasticsearch and Machine Learning

Smart energy consumption insights with Elasticsearch and Machine Learning

At home we have a Youless device which can be used to measure energy consumption. You have to mount it to your energy meter so it can monitor energy consumption. The device then provides energy consumption data via a RESTful api. We can use this api to index energy consumption data into Elasticsearch every minute […]

Read More

Simulating an Elasticsearch Ingest Node pipeline

Indexing document into your cluster can be done in a couple of ways: using Logstash to read your source and send documents to your cluster; using Filebeat to read a log file, send documents to Kafka, let Logstash connect to Kafka and transform the log event and then send those documents to your cluster; using […]

Read More

Public Elasticsearch clusters are being held ransom

Public Elasticsearch clusters are being held ransom

Last week several news sites and researchers reported that Elasticsearch clusters that are connected to the internet without proper security are being held ransom. You can use shodan.io to search for Elasticsearch clusters: https://www.shodan.io/search?query=port%3A9200+json&language=en. The first hit is actually a cluster that is ‘infected’: There are some secured clusters as well: But the default ‘root’ account with […]

Read More

Handling a massive amount of product variations with Elasticsearch

Handling a massive amount of product variations with Elasticsearch

In this blog we will review different techniques for modelling data structures in Elasticsearch. A project case is used to describe our approach on handling a small sized product data set with a large sized related product variations data set. Furthermore we will show how certain modelling decisions resulted in a 1000 factor query performance gain! […]

Read More

Collecting data from a private LoRaWAN sensor network into Elastic

Collecting data from a private LoRaWAN sensor network into Elastic

Introduction to LoRaWAN and ELK Why LoRaWAN, and what makes it different from other types of low power consumption, high range wireless protocols like ZigBee, Z-Wave, etc … ? LoRa is a wireless modulation for long-range, low-power, low-data-rate applications developed by Semtech. The main features of this technology are the big amount of devices that […]

Read More

City-wide crowd management in Amsterdam

City-wide crowd management in Amsterdam

As most residents and visitors of Amsterdam know, every year more people are visiting Amsterdam, city wide events like GayPride, Koningsdag and MuseumNacht are getting bigger and more frequent, putting more strain on the city’s infrastructure and all people living in the city center. That’s why this November 7th, Amsterdam Marketing organized the Museumn8 hackathon […]

Read More

Dealing with NodeNotAvailableExceptions in Elasticsearch

Dealing with NodeNotAvailableExceptions in Elasticsearch

tl;dr Elasticsearch provides distributed search with minimal setup and configuration. Now the nice thing about it is that, most of the time, you don’t need to be particularly concerned about how it does what it does. You give it some parameters – “I want 3 nodes”, “I want 3 shards”, “I want every shard to […]

Read More