Last week I attended GOTO Copenhagen. Everybody should know the GOTO conferences by now. This was a new conference in Denmark. I had a good time, attended very good keynotes and a number of very nice presentations. I have also given a talk myself called: Visualising real time traffic data using elasticsearch and c3js. In this blogpost I am giving a summary of some of the talks I attended.
Posts by Jettro Coenradie
For one of our projects we wanted to create some nice charts. Feels like something you often want but do not do because it takes to much time. This time we really needed it. I had prior experience with Highcharts, which is also a nice library. But this time we wanted more control. We had a look at D3.js library, a very nice library but so many options and a lot to do yourself. Than we found c3.js, check the blog post by Roberto: Creating charts with C3.js. Since I do a lot with AngularJS, I wanted to integrate these c3.js charts with AngularJS. I wrote another blogpost on that: Using C3js with AngularJS. Through twitter Abdullah Diaa mentioned that it would be great to have directives for these charts. That is were we start in this blogpost. I am going to describe the first version of a directive I have created to make it even easier to create these chart using AngularJS.
Last week was the week of the Berlin Buzzwords conference. For those of you that do not know it, Buzzwords is one of the best conferences about search, scale and store (read: big data). Next to having good speakers and nice content there is also a very important social aspect during the conference. There is a good reason the first session starts at 10 o’clock.
Some time ago, Kibana joined the elasticsearch family. A lot of good things have come out of it. These days Kibana is becoming more advanced. But with more users also come more demands. One of those demands is more advanced dashboards than can be clicked together in the very nice GUI. We want to be able to customize dashboards, prepare dashboards to be used by others.
In this blogpost I am going to show you some of the options you have to create a more advanced dashboard. I use an index I have created based on my iTunes library. We are going to create a dashboard showing information about artists, albums and we show how to use parameters through the url.
The past years I have been working with search solutions, mostly elasticsearch. During this time a bought myself a raspberry pi and installed java and elasticsearch on it. Then I put it in the closet and it did not come out anymore. Than a few weeks a go the guys from elasticsearch released marvel. Marvel is a monitoring tool for your elasticsearch cluster. Suddenly I realized what the problem with the raspberry pi is. It is not fun to have just one. Therefore I decided to buy two more and create an elasticsearch cluster. With this cluster I can do experiments. The first experiment is evaluating marvel.
In this blog post I will show some of the concepts of marvel. To make this possible I will also explain the steps I had to take to install elasticsearch on my raspberry pi cluster.
If you are reading some technical blogs, maybe about search or data analysis, chances are big you have read about Kibana. You have seen stories about how easy it is to use. Most of the blogging effort deals with getting data into kibana using logstash for instance. Maybe some of you have installed Kibana and are using it in combination with logstash. But what if you want to analyze other data. With the most recent release M4, Kibana is better than ever in analyzing other sort of data. In this blog I am going to show you how to create your own dashboard in Kibana. In order to do something useful with Kibana we have to have data. Peter Meijer had a very nice idea to index metadata from all of your images to learn about the type of photo’s that you take. I decided to put this in practice. I used Node.js and the exiftool to obtain metadata from images and store it in elasticsearch.
We have done multiple big Hippo projects. A regular Hippo project consists of multiple components like the website, the content management system and a repository for the documents. In most of the projects we also introduce the integration component. This component is used to pull other data sources into Hippo, but we also use it to expose data to third parties.
By default, the Hippo Site Toolkit delegates searches to the Hippo Repository that in turn delegates the search to Jackrabbit. This is the repository that is used by Hippo to store the documents. JackRabbit has integrated Lucene which can be used for search. This is a domain specific Lucene implementation targeting to be compatible with the Java Content Repository specification. This however comes at a price, for example more expensive and less customizable searches than search engines like Solr or Elasticsearch provides. for typical Solr/Elasticsearch features like highlighting, suggestions, boosting, full control over indexing or searching external content, Hippo Repository search is limited.
This search problem can be overcome using a specialized search solution like Elasticsearch. For multiple customers we have realized this solution, some based on Solr and others on Elasticsearch.
In this blogpost I am describing the solution we have created. I’ll discuss the requirements for creating (near) real time search using Hippo workflow events as well as the integration component that reads the documents from Hippo and pushes them to Elasticsearch.
A few weeks a go I attended the GOTO conference in Amsterdam. One of the reasons I like to go to conferences is to meet people I haven’t seen for a while, talk about technology and other stuff. The other reason to go to conferences is to get inspired. After the conference I like to write down what I got inspired by. But it is always hard to capture the impressions I got in a blog entry…
For this conference I could have written about Linda Rising and her keynote talking about incentives. Why should you use them, or why not. Than I would have talked about managers that still think you are doing nothing as a programmer when you are not working on your computer. I probably would have told about the effect of giving rewards to people. That it is usually counter productive when people get rewards. People can think they need to be given a reward because they need to do something that is not fun to do. I probably also would have told you the story about a company that paid new hires 3000 dollar to leave after their initial 2 week training program.
If you do want to know what I will write about, read on.
In the first part of this series of blogs on Introducing a Query as an Elasticsearch plugin, I described the functionality of a query tool I also discussed the functionality of the tool as the structure of the project. In this part I want to take a deeper dive in interacting with Elasticsearch.
The post consists of four parts:
- Getting information about the Elasticsearch instance,
- Executing the query,
- Using the results.
If you need more background around AngularJS, which is used on the client side you can have a look at the following two posts:
At Devoxx 2012 I attended the AngularJS presentation by Igor Minar and Misko Hevery. I was very enthusiastic about the capabilities of this front-end framework. Therefore I started experimenting with it. I created a sample for the Axon Framework, read more about it here. After my experiments I felt confident enough to start using it in real projects. One of them was adding management reporting using the HighCharts library.
In this blog post I’ll give you some lessons learned with respect to AngularJS.