At the ANWB people are constantly trying to improve the services they provide. One of these services is to provide traffic information. In the Netherlands the National Data Warehouse for Traffic Information (NDW) provides an enormous database of both real-time and historic traffic data. This data comes from many different sources and is available as open […]
Creating an advanced Kibana dashboard using a script
Some time ago, Kibana joined the elasticsearch family. A lot of good things have come out of it. These days Kibana is becoming more advanced. But with more users also come more demands. One of those demands is more advanced dashboards than can be clicked together in the very nice GUI. We want to be […]
Elasticsearch, Spring MVC & Sencha Touch 2 in the Cloud – Part 2
This is the second part of my blog on how to develop an application using Elasticsearch, Spring MVC and Sencha Touch 2. In my previous blog post part 1 I showed and explained which technologies I used to accomplish the connection between the frontend and backend. In addition I presented the steps to connect a […]
Elasticsearch & Spring MVC & Sencha Touch 2 in the Cloud – Part 1
Introduction Welcome to my third blog entry. In this one, I want to show how to connect three different technologies (database, REST service, and a mobile framework) and deploy them into the cloud. Those three technologies are: Elasticsearch (Database) Spring MVC (REST service) Sencha Touch 2 (Client side framework) First, I want to give […]
Server-side clustering of geo-points on a map using Elasticsearch – continued
In a previous post I described a problem of data visualization and a possible solution provided by a plugin of elasticsearch. I noticed that elasticsearch might one day evolve to make the plugin unnecessary. That day seems to have come: starting from version 1.0.0, elasticsearch includes Aggregations, a new API for data mining. In this post I’ll […]
Evaluating elasticsearch and marvel on the raspberry pi
The past years I have been working with search solutions, mostly elasticsearch. During this time a bought myself a raspberry pi and installed java and elasticsearch on it. Then I put it in the closet and it did not come out anymore. Than a few weeks a go the guys from elasticsearch released marvel. Marvel […]
Using logstash, elasticsearch and Kibana to monitor your video card – a tutorial
A few weeks ago my colleague Jettro wrote a blog post about an interesting real-life use case for Kibana: using it to graph meta-data of the photos you took. Given that photography is not a hobby of mine I decided to find a use-case for Kibana using something closer to my heart: gaming. This Christmas […]
elasticsearch – how many shards?
We’ve all been there – you’re provisioning for an elasticsearch index and one of the first questions that comes to mind is “how many shards should I create my index with?”. In my previous posts on the subject, I wrote about how to find the maximum shard size for elasticsearch. Although informative, the results of […]
Use Kibana to analyze your images
If you are reading some technical blogs, maybe about search or data analysis, chances are big you have read about Kibana. You have seen stories about how easy it is to use. Most of the blogging effort deals with getting data into kibana using logstash for instance. Maybe some of you have installed Kibana […]
Maximum shard size in elasticsearch – revisited
In my last blog post on the subject, I tried to find the maximum shard size in elasticsearch. But in the end all I could say is that elasticsearch can index the whole English Wikipedia dump in one shard without any problem but that queries are painfully slow. I couldn’t find any hard limit because […]