Trifork Blog

The Keynotes of GOTO Amsterdam

No Comments

Did you already have a look at our keynotes? I want to say something about it.

Let’s start with one of the hottest topics these days: self-driving cars. There is, depending on the country you live in, a lot of discussion right now if self-driving and electric cars will solve many of our transportation problems or not. Other discussions are: are they coming or not? Are they coming big-bang or rather slow (over the years, cars take over more and more responsibility). Ethical questions. Do these systems work in heavy rain or snow in the dark? How do you insure (or sew) a self-driving car? And, for us developers, how does it actually work? We, the GOTO program committee, also wonder about these questions and therefore we invited Jonathan Sprinkle to give a keynote on self-driving cars. He heads the cyber-physical systems lab at the University of Arizona and many, if not all of his students joined Google or Uber to work on self-driving cars.

Ever wondered why Microsoft and others ship a web-based IDE? It makes a few things possible which weren’t possible before like executing Unit-tests or expensive code and architecture analysis as you go in the Cloud. But I can also see challenges: how do I deploy to my web-container and debug stuff? We invited Erich Gamma to explain the upcoming world of IDEs to us. As a side-effect, we’ll learn how to ship a large-scale JavaScript application and how to architect a 350KLOC TypeScript-based system including more than 100+ Open Source components.


Remember the good old days in ancient Rome where people already encrypted messages with the Ceasar encryption 🙂 Encrypted messages are more than 2000 years old and are everywhere in our daily life. In fact, if someone would be able to crack our current encryption standards, the world would stand still (or, in case of some secret services, in high dudgeon). The best book out there on the history of cryptography is The Code Book from Simon Singh. Simon is one of the most famous science authors today and he will come to GOTO Amsterdam to talk first about his 10.000 GBP prized “Cipher Challenge” and then about the history of cryptography and why encryption is more important today than ever before. He is also bringing along an original Enigma from WWII.

Keynotes should be interesting and inspiring for every conference attendee. What could be more interesting and inspiring than an astronaut who spent more than 200 days in space on the ISS? We invited Andre Kuipers to talk about his experience working in an outstanding multi-national endeavour realised by the United States of America, Europe, Russia, Canada and Japan. As he says in his abstract: “The ISS is the largest, most complex, international technological structure ever made by mankind”. Much of the insights engineers and nations gained from the construction and operation of the ISS will make it into our life not too far in the future, just like the moon landing did it for the US. If we want to have a glimpse into that future, we need to attend his keynote on “Technology & Innovation”.

That’s it. I hope all of you are looking forward as much as I do to these four outstanding talks!

Posted in: Conference

Collecting data from a private LoRaWAN sensor network into Elastic

No Comments

Introduction to LoRaWAN and ELK

Why LoRaWAN, and what makes it different from other types of low power consumption, high range wireless protocols like ZigBee, Z-Wave, etc … ?

LoRa is a wireless modulation for long-range, low-power, low-data-rate applications developed by Semtech. The main features of this technology are the big amount of devices that can connect to one network and the relatively big range that can be covered with one LoRa router. One gateway can coordinate around 20’000 nodes in a range of 10–30km. It’s a very flexible protocol and allows the developers build various types of network architectures according to the demand of the client. The general description of the LoRaWAN protocol together with a small tutorial are available in my previous post.

What is the ELK stack, and why use it with LoRaWAN?

NwusuJa2imewHTFDr_Xt-Tl72eJkfbmt4t8yenImKBVvK0kTmF0xjctABnaLJIm9

In the figure above, you can see a simplified model of what a typical LoRaWAN network looks like.
As you can see, the data from the LoRa endpoints, has to go through several devices before it reaches the back end application. Nowadays there are a lot of tools that would allow us to gather and manipulate the data. A very good solution is the ELK stack which consists of Elasticsearch, Logstash and Kibana; these three tools allow to gather, store and analyze big amounts of data. More information and details can be found on the official website: https://www.elastic.co/.

Read the rest of this entry »

Posted in: Elasticsearch | From The Trenches | Internet of Things

Server side applications in Apple’s Swift

No Comments

In 2014, Apple announced the release of Swift, a new programming language for all their platforms. Their programming language of choice on iOS and OSX has always been Objective-c, a language which is a bit dated (it predates C++) and as it has had new features (and syntaxes) bolted onto it every few years, it carries quite a bit of baggage. It seems I wasn’t the only one with this opinion, as the release of swift was greeted with great enthusiasm, and has been adopted very rapidly.

Swift combines all the features that are fashionable in a general purpose language today, without the feeling that they were bolted on after the fact. While building an iOS client for our customer Gerimedica in swift, I found myself wishing I could use this language on the server side as well as in the client. At WWDC 2015, Apple announced the intention to open source the language, and release a Linux version, so it looked like it could become a reality. Since december 2015, the sources have been available on github, and builds for OSX and Ubuntu are made available roughly twice per month.

PerfectLib

A number of groups and companies saw an opportunity to be among the first with something that was obviously going to be big. One of the first was PerfectSoft, a startup that aims to be the one big framework for all your server side development in swift. They started building their framework as soon as the open source release of swift was announced, and have been advertising their product everywhere. Because they started development before anyone outside Cupertino had a good idea what the release would look like, it only worked on OSX at first, and it didn’t use the Swift Package Manager, the intended default build and dependency management tool for swift. At the time, the framework compiled to one big binary, that you had to include in your build manually. They have a beautiful website and good documentation, but it just wasn’t working when I tried it. I intend to try this framework out again at a later date.

IBM

The biggest player (other than Apple) to openly jump on the swift bandwagon is IBM. As soon as the open source release of Swift was announced, IBM announced the Swift Sandbox, their Swift based version of google’s golang playground. It is a web based repl that can be shared online by sharing a URL. Cool, but not extremely useful, as unlike go, swift already comes with a repl. The real significance of this is not the swift sandbox itself, but the message that IBM is interested in this technology and intends to be involved. IBM isn’t the kind of company to back technologies just because they like them, so they either see an opportunity, or a potential strategic interest. At the moment, IBM’s swift related activities seem to be associated with their PaaS solution BlueMix, so they are likely working on the Swift / IBM version of google’s app engine for go. IBM offers its own web framework for swift: kitura. Kitura turns out to be less than trivial to install and for now somewhat bare bones, but as this is IBM, it is worth dedicating another blog post to it at a later date. Also check out their overview of the most popular, most active and most essential open source projects on github for swift.

Read the rest of this entry »

Posted in: Custom Development | Swift

Using Spring Session for concurrent session control in a clustered environment

3 Comments

For a long time, Spring Security has provided support to limit the number of sessions a single user can have concurrently. This prevents users from being logged in from many different devices at the same time, for example to ensure that they won’t share their credentials to a paid site with their friends and family.

My former colleague Quinten Krijger has blogged about this feature beforeNote the last paragraph, which explains how this support is limited to single-node applications.

Although running on a single node may suffice for many applications, there are plenty applications running in a clustered environment that should be able to benefit from concurrent session control as well. As hinted in the aforementioned blog, this requires both implementing a custom SessionRegistry as well as ensuring that expiring a session is propagated to all nodes in the cluster.

This is exactly what I’ve done recently using Spring Session, a framework that allows you to take control over managing sessions using a shared external registry like Redis. In this post I’d like to walk you through the code, which can be found here: https://github.com/jkuipers/spring-session-concurrent-session-control

UPDATE:

Based on the code I wrote for this blog I’ve opened a pull request for Spring Session. That request is scheduled for inclusion in Spring Session 1.3, but the code works just fine with the upcoming 1.2 release and removes the limitation of not providing an expiry notification after exceeding the maximum number of sessions.

Read the rest of this entry »

Posted in: Custom Development | Spring

GOTO Amsterdam – The ideas behind the program

3 Comments

We recently had our program release party with Jim Webber, where we published our program for GOTO Amsterdam 2016. Over the last weeks we filled every missing spot and the exciting program is now complete. But we haven’t shared our thoughts on the program decisions itself.

Read the rest of this entry »

Posted in: Custom Development

GOTO Nights Amsterdam, Join & Learn!

No Comments

As an introduction to the GOTO Conference in June, we organize a monthly meetup to introduce you to the topics of the conference. These meetups are called GOTO Nights and are totally free to attend and food and drinks are included.

At the start of the GOTO season we gather together to decide which topics are interesting for developers to learn more about. For these topics we try to find the best speakers for the conference and for the GOTO Nights.

Read the rest of this entry »

Posted in: Agile | Conference | Education | General | Internet of Things | Knowledge | Spring

From The Trenches: LoRa, LoRaWAN tutorial with the LoRaBee

1 Comment

Once in a while you stumble across an interesting technology which is an enabler of new solutions, and new business cases. At Trifork Eindhoven we believe LoRa / LoRaWan might be such a technology. We are currently working on several Internet of Things solutions which could directly benefit from long distance communication, with a low energy footprint.

In this document I will describe the things that I learnt while working with the LoRaBee development kit. I will also provide some basic tutorials on how to set it up, how to start a connection and how to connect it with different platforms.
The hardware that I am using:

  • SODAQ Mbili with Atmel 1284p Microcontroller x 2
  • SODAQ LoRaBee which contains the Microchip RN2483 Module x 2
  • A set of peripherals for data acquisition ( sensors, buttons, LEDs, serial interfaces… )

Quick intro into LoRa

According to Wikipedia:

LoRaWAN (Long Range Wide Area Network) is a low power wireless networking protocol designed for low-cost secure two-way communication in the Internet of Things (IoT). LoRaWANs use of sub-GHz ISM bands also means the network can penetrate the core of large structures and subsurface deployments within a range of 2 km.[1]

The technology utilized in a LoRaWAN network is designed to connect low-cost, battery-operated sensors over long distances in harsh environments that were previously too challenging or cost prohibitive to connect. With its penetration capability, a LoRa gateway deployed on a building or tower can connect to sensors more than 10 miles away or to water meters deployed underground or in basements. The LoRaWAN protocol offers unique and unequaled benefits in terms of bi-directionality, security, mobility and accurate localization that are not addressed by other LPWAN technologies. These benefits will enable the diverse use cases and business models in deployments of LPWAN IoT networks globally.”

The LoRa protocol is based on three types of devices:

  • Class A – LoRa end device, for example the RN2483 that I am using.
  • Class B – LoRa Gateway, which acts like a wireless modem, and is in control of the network and data flow
  • Class C – Network Server. Any web platform which is able to collect and process data.

In the following picture, you can see the typical LoRaWAN network:

NwusuJa2imewHTFDr_Xt-Tl72eJkfbmt4t8yenImKBVvK0kTmF0xjctABnaLJIm9

As you can see, all three classes are required for a fully working LoRaWAN network. In the tutorial, we used two Class A – LoRa devices for a P2P communication; however that is a simple radio transmit/receive between the LoRa modules and not the LoRaWAN protocol.

Read the rest of this entry »

Posted in: Custom Development | From The Trenches | Internet of Things

Spring-AMQP and payload validation: some notes from the trenches

2 Comments

It’s been a while since I’ve written one of our from-the-trenches blogs: that’s mostly because I’ve been very busy in those trenches developing systems for our customers.

This week I completed a Spring Boot-based microservice which is responsible for interacting with some 3rd party SOAP service: its own clients communicate with it by sending request message over RabbitMQ, and the service then sends back a response to a response queue after handling the SOAP response message.

Of course I used Spring-AMQP to build this service. Spring-AMQP supports a nice annotation-based listener programming model, based on Spring’s generic Message support.
That allows writing listener methods like this:

@RabbitListener(queues = REQUEST_QUEUE) 
public DeclarationResponse submitDeclaration(DeclarationRequest request) { 
  // handle the request and return a response 
}

The request parameter here is the result of converting the incoming AMQP message using a Spring-AMQP MessageConverter, after which it is considered to be the payload of the message (even when headers are used in the conversion as well).

The request messages that the clients send have some required fields: without those fields, the service can’t make the SOAP calls. While reading the RabbitListener JavaDoc I noticed that Spring-AMQP allows you to apply validation to message payload parameters by annotating it. When using this, you also have to add the @Payload annotation (which is optional without validation if your method doesn’t have any other arguments), so the result looks like this:

@RabbitListener(queues = REQUEST_QUEUE)
public DeclarationResponse submitDeclaration(@Valid @Payload DeclarationRequest request) { … }

By the way, Spring’s own @Validated (even as a meta-annotation) and in fact every annotation whose name starts with “Valid” are supported for this purpose as well.

Now we can add some JSR-303 Bean Validation annotations to the fields in our DeclarationRequest, like @NotNull, to express our validation constraints.

Read the rest of this entry »

Posted in: Custom Development | From The Trenches | Spring

Elastic{ON} 2016

No Comments

Elastic{ON} 2016 - ViewLast week a colleague and I attended Elastic{ON} in San Francisco. The venue at Pier 48 gave a nice view on (among others) the Oakland Bay Bridge. Almost 2000 Elastic fanatics converged to listen to and talk about everything in the Elastic Stack.

I have been to a lot of sessions. I think the two most important things that I will take home are “5.0” and “graphs”.

5.0

The next version of the Elastic Stack will be 5.0. This means that all main Elastic products (Elasticsearch, Logstash, Kibana and Beats) are having the same version number in all following release bonanzas. This will be easier for all customers and clients.

I mentioned the Elastic Stack. This is a little rebranding of the ELK Stack plus Beats. More rebranding is the renaming of the Elastic as a Service solution Found to Elastic Cloud. I think those are simple but good changes.

Also Elastic created the concept of packs to combine extensions. Most notably the X-Pack will all the monitoring, alerting and security (and more) goodies wrapped together.

More about 5.0 on the Elastic blog.

Graphs

Elastic{ON} 2016 - GraphThe other main take-away are the graph capabilities (Graph API) that will be added to Elasticsearch (through the X-Pack). It is still in an early phase but it looks awesome! It looks very easy to use and it is very fast. The UI is written as a Kibana plugin.

Actually there will be some more Kibana plugins. Managing users and roles via the Security API, for example.

Talks

Off course there were a lot of talks. Common subjects were security and recommendation. Graphs could play an important role there!

Some talks were cool user stories of companies that implemented (parts of) the Elastic Stack. Other talks dove deep into the different Elastic products. Some of those turned out to be a little out of my league. For example the math behind the new default BM25 scoring algorithm.

The talks will be put online in the next couple of weeks. So be sure to check them out! Maybe I will see you next year!

Posted in: Conference | Elasticsearch

Personalised city trip itinerary using integer linear programming

2 Comments

Case

As a research project I have developed an itinerary service. The idea started when I was doing a hackathon with colleagues for the city of Amsterdam (see earlier post). I wanted to recommend an itinerary to a tourist visiting the city of Amsterdam. Furthermore, I wanted to make the itinerary based on the user’s interests to recommend interesting places and activities for him in the city. If the user is interested in modern art for example, the recommendation scores for modern art museums will increase for that user.

The tool divides the duration of the tourist’s stay into separate time slots. For example, a single day could be divided in 3 time slots: morning, afternoon, evening. POIs get a different recommendation score for each time slot they can be visited. The Vondelpark for example could be less recommended on Monday morning because of expected rain or because of expected crowds. On Tuesday morning the Vondelpark could be recommended because of an interesting event or nice weather.

The itinerary tool will try to limit travel time between the recommended POIs (point of interests). In this way the tourist will not waste time on travailing. The user can also set a budget for the entire itinerary.

By taking all these considerations into account the tool should be able to aid the user in making good decisions about which places to visit and when to visit them.

Here is a screenshot of the user interface of the itinerary tool that was created as part of the research project:

image01

Read the rest of this entry »

Posted in: Java