MQTT DZone Refcard

DZone recently released my refcard „Getting started with MQTT“. I’m very happy with the result and I hope you’ll find it useful.

You can download the refcard here: https://dzone.com/refcardz/getting-started-with-mqtt

The refcard covers the following topics:

  • Why MQTT?
  • What Is MQTT?
  • Publish / Subscribe
  • MQTT Message Types
  • Topics
  • QoS Levels
  • Last Will and Testament
  • Retained Messages
  • Clean / Persistent Sessions
  • MQTT Clients
  • Trying MQTT on the Command Line: Linux and Mac OS X
  • Pub / Sub With Paho
  • MQTT Over Websockets
  • Scaling MQTT
  • MQTT and Security
  • New Features in the MQTT 3.1.1 Standard

All you need to know about MQTT Security

The HiveMQ blog recently ran a blog post series about securing MQTT. It featured 10 blog posts packed with content about security for the Internet of Things and MQTT. 

For convenience, here is a list with all blog posts.

If you plan to run MQTT in production, make sure to read the blog posts!

MQTT presentation at the Global IoT Day 2015 in Vienna

I recently gave a talk about „Pub / Sub for the masses – An introduction to MQTT“ in Vienna at the Global Iot Day 2015 Event. It was a very short talk and I only covered the most important aspects of MQTT.

It’s available on Youtube and you can see it in full length below. Unfortunately the microphone had a loose contact, so I apologize for the bad sound. The audience had some great questions afterwards, so make sure to watch the whole video. Enjoy!

The slides are also available on Slideshare, you can see it below:

All you need to know about MQTT

If you are interested in IoT and particularly in MQTT, you should check out the MQTT Essentials blog post series on the HiveMQ blog. For convenience, here is the full list of all MQTT Essentials parts:

Every Monday is MQTT Monday at the HiveMQ blog, so a fresh new blog post about MQTT is available every Monday. I’m excited to see what’s coming next!

Eclipse Democamp Mini-Tour 2014

I’m finally back home from my Eclipse Luna Democamp Mini-Tour this year and it was awesome. I met many new and old friends and learned a lot. While I am not interested that much in topics like modelling, SWT and RCP for day-to-day business (remember, we do M2M/IoT software like the MQTT Server HiveMQ), I enjoyed learning about new things in these spaces and I was impressed again about the diversity of the Eclipse Ecosystem.

Democamp Munich

My first stop was the Eclipse Democamp in Munich. The location, catering and organization was fantastic and the room was packed. There was a great variety of topics and all the talks had a great quality.
The most notable highlight was for sure the talk by Torkild Ulvøy Reshdeim from Itema AS (Norway). After introducing the new Eclipse Science Working Group, Torkild presented their Eclipse based simulation engine for oil platforms. He showed a live simulation of nautical environment changes which caused catastrophic accidents in the past.
Another interesting talk for me was the Microsoft Team Foundation Integration for Eclipse, I was surprised how well TFS integrates with Eclipse and that Microsoft now really seems to go more open routes, including encouraging companies to use Git.

Angelika Wittek and I showed a proof-of-concept of the redesign of the Eclipse Events page. The awesome thing about it is, that it uses MQTT and the Eclipse Paho Javascript library for bringing the event data to the webpage. The HiveMQ enterprise MQTT broker is used so it is trivial to use MQTT over Websockets thanks to its native MQTT Websocket support. It was great to see to get so many people interested in Eclipse IoT in general and MQTT in particular and I really enjoyed the discussions in the break and after the event.

For those interested, these are the slides we had for our presentation:

Democamp Stuttgart

Next stop was the Democamp in Stuttgart. What I like about the Stuttgart Democamp is, that it’s very practical and the speakers always tend to do much live coding, which is always fun. I was pleasantly surprised that there were 3 (!) out of 6 talks related to IoT and all of them covered MQTT, including the talk from Angelika Wittek and me.
My favourite demo was the „MQTT-Robot-Arm“ Demo by the Eclipse Franca Team which controlled a robot arm via a web page and MQTT. They generated two different backends and one frontend with Franca for the communication. If they would have used HiveMQ together with Paho.JS, the overall design would have been much simpler, though.

Democamp Vienna – Vienna Calling!

The last stop was the Democamp in Vienna. I was surprised that I got 40 minutes for the talk and so I decided to present something different than „just“ MQTT over websockets. The idea was to present the open source plugin system of HiveMQ. When I was sitting in the ICE train from Landshut to Vienna, I suddenly thought about the song „Vienna Calling“ by Falco and it just didn’t get out of my head. So I threw away my initial demo I prepared for the democamp and started hacking right away.

Honestly, this is my favourite demo I ever did: I demonstrated a very simple HiveMQ plugin which was able to actually call people after a MQTT message was sent to a specific topic. To make things more interesting, the called guy wouldn’t just get called by a robot, no, Falco himself would call and play his hit „Vienna Calling“. (Ok, actually Falco didn’t call but at least the song was played by the caller).

The democamp in Vienna was awesome, well organized and there was plenty of beer ;-). I had very interesting discussions and I enjoyed all the talks, especially the talk about Eclipse Oomph by Eike Stepper and Ed Merks, you should definitely check it out if you are a Eclipse User.

If you’ve never an Eclipse Democamp before, you should definitely consider visiting one near you, people and speaker are usually awesome at Eclipse Events and you are definitely going to learn new and cool things.

Bringing MQTT authentication and REST together

For most non-trivial server software which run in production, authentication is a very important. Of course this also applies to MQTT brokers. Luckily the MQTT v3.1 specification includes a username and password authentication mechanism which most MQTT brokers implement.

While username/password files for authentication are sufficient for playing around with MQTT brokers, they are not sufficient for enterprise-grade production systems. When integrating a MQTT broker in existing software landscapes, typically there are existing databases and services.

To demonstrate how dead simple it is to integrate an existing HTTP REST API for MQTT authentication, I created a simple HiveMQ MQTT broker plugin which delegates the authentication mechanism to the REST API (in this case a mock REST API). This API returns a JSON response which we parse in the HiveMQ plugin. It uses the excellent Apache HTTPClient from the HTTP Components project to integrate the authentication mechanism. You can find the project on Github here: https://github.com/dobermai/hivemq-rest-auth-plugin.

HiveMQ + REST API

This shows the whole implementation of the authentication mechanism. Feels like 90% exception handling 😉

The exact same mechanism can be used if you want to integrate MQTT authentication with some SOAP webservices, NoSQL databases, SQL databases, OAuth and anything you can imagine.

Make sure your MQTT broker of choice supports plugins. If you want to give the HiveMQ plugin system a shot, start here.

Installing a HiveMQ MQTT Server on AWS EC2 with enabled Websockets

To enable communication between MQTT devices, it’s necessary to use a MQTT broker as the central server for your M2M communication. Although there are some public brokers available like mqtt-dashboard.com, it’s a good idea to set up an own server for playing around. This post shows how to set up a HiveMQ MQTT server instance on Amazon Web Services Elastic Compute Cloud (EC2). As an additional goodie, we want to enable MQTT over websockets, so every browser can be a full-featured MQTT client!

Step 1: Create a new EC2 instance

General

The first step is to launch a new EC2 instance. In general it does not matter which OS you choose for HiveMQ as it runs perfectly on every major OS. Any Linux distribution should be fine, I will use a Ubuntu 12.04 LTS. To get started, a Micro Instance will be sufficient, if you need real power and throughput, you should start with more RAM and more vCPUs.

Security groups

Security Groups are very important to configure correctly, otherwise we won’t be able to connect to our server correctly.

AWS MQTT Security Group Settings

AWS MQTT Security Group Settings

Open the following ports to the outside world for maximum MQTT pleasure:

  • 22: Needed for SSH. You probably lock yourself out if your don’t have this port open. Consider restricting this port to your IP adress(es) only.
  • 1883: The MQTT standard port
  • 8883: The MQTT standard port for MQTT over TLS.
  • 8000: The port we want to use for MQTT over websockets

Step 2: Download and install HiveMQ

After launching EC2 instance, we should SSH into it to install Java and HiveMQ. Depending on your OS, these commands might be a bit different.

Install Java + Utils

First we want to install Java and needed utilities. Execute the following commands:

sudo apt-get update
sudo apt-get install openjdk-7-jre-headless unzip

Now you can run

java -version

and the output should look like this:

java version "1.7.0_25"
OpenJDK Runtime Environment (IcedTea 2.3.10) (7u25-2.3.10-1ubuntu0.12.04.2)
OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)

Install HiveMQ

Now we just have do download HiveMQ and unzip it. Execute the following:

wget --content-disposition http://www.hivemq.com/downloads/releases/latest
unzip hivemq-1.x.x
cd hivemq-1.x.x

Configure HiveMQ

HiveMQ comes with sensible defaults and we could get started without modifying the configuration if we don’t need websockets support. But since MQTT over websockets is just awesome, we’ll enable it:

Edit the conf/configuration.properties file and change the following values:

websockets.enabled=true
websockets.port=8000

Step 3: Start HiveMQ

Now just run bin/run.sh and HiveMQ should start up. Verify to see an output like this:

2013-11-30 23:04:46,872 INFO  - HiveMQ home directory: /home/ubuntu/hivemq-1.4.2
2013-11-30 23:04:46,876 INFO  - Starting HiveMQ Server
2013-11-30 23:04:50,225 WARN  - No license file found. Using free personal licensing with restrictions to 25 connections.
2013-11-30 23:04:50,832 INFO  - Activating statistics callbacks with an interval of 60 seconds
2013-11-30 23:04:50,833 INFO  - Activating $SYS topics with an interval of 60 seconds
2013-11-30 23:04:52,053 INFO  - Starting on all interfaces and port 1883
2013-11-30 23:04:52,069 INFO  - Starting with Websockets support on all interfaces and port 8000
2013-11-30 23:04:52,076 INFO  - Started HiveMQ 1.4.2 in 5207ms

That was all. Now you have a high performance MQTT server up and running in the cloud and you can start writing your MQTT applications on devices AND in the browser.

P.S. You can test the MQTT over websockets support with the nifty Websocket Browser Client here

EclipseCon 2013

I’m back from the EclipseCon Europe in Ludwigsburg and it was a great conference, I met many new and old friends and had lots of fun. The conference was very well organized and as far as I heard it was the biggest EclipseCon Europe ever.

Although I didn’t visit all the sessions I wanted to attend (to many good conversations in the hallways!), I at least managed to see the keynotes. :-) The keynote „Single Points of Failure: The Human Element of Software Engineering“ by Brian Fitzpatrick was brilliant. Very entertaining and non-technical keynote.

The second keynote, „Scaling Pinterest“ by Marty Weiner was very insightful and I was amazed and shocked how fast some startups like Pinterest grow and which problems they have to solve. I think Weiner had every NoSQL database you can think of on his slides.

When visiting the conference, it was clear to everyone, that M2M was one of the most important topics this year. In fact, most people were surprised how fast the M2M ecosystem of Eclipse has grown the last two years. Many people (jokingly?) speculated when the number of M2M projects will surpass the „classic“ Eclipse projects.

I talked with many people about MQTT, Eclipse Paho and HiveMQ and it was great to see that most people didn’t know much about MQTT before and most of them got very excited about this neat little technology. I personally think the conference was a big success for Eclipse Paho, there was a workshop (Christian Götz and me) about Java Paho and Javascript Paho, there was a regular Session about Paho and many people were also talking about it on the hallways.

Also, the other M2M projects on Eclipse were parts of many discussions on the hallways. I personally think there is some confusion about the focus of all these projects, but luckily Benjamin Cabé did a session about all these M2M projects and hopefully cleared the confusion for some people.

Although there are many people aware of the M2M projects in the Eclipse universe and these projects gained attention and traction recently, I think we have a long way to go until the M2M projects of Eclipse get „mainstream enough“ to be an integral part of every (Java) developers toolbox. But if Eclipse continues to follow the path they chose with M2M, I’m confident there is an even brighter future for the M2M projects and Eclipse as Foundation in General.

The slides for my sessions are available here:

Bringing M2M to the web with Paho – Connecting Java devices and online Dashboards with MQTT
M2M for Java developers – MQTT with Eclipse Paho

Review of „Getting started with Google Guava“

I recently read the book Getting Started with Google Guava by Bill Bejeck, published by PACKT Publishing. I have been working with Google Guava a long time and I was surprised how many new things I learned from the book, because I thought I knew this library pretty well.

The book itself is very practical and there are tons of code examples. I especially liked the fact that most code examples are written as JUnit tests, so you immediately get a feel of how to use the utility class under discussion and in which context they can be used. I want to discuss each chapter in detail now:

Basic Guava Utilities

This chapter discusses utils from Guava which you will find in many projects. The author explains the advantages of the Guava classes for String manipulations like Splitter, Joiner, Strings and Charmatcher. This chapter is pretty straightforward and if you don’t know these utilities yet, you’ll immediately see how these utils can simplify your life. Guavas Preconditions class and different builders for toString(), hashCode() and equals() methods are also discussed in this chapter.

Functional Programming

One of the most amazing features of the Guava library are the utilities to enable functional-style programming in Java. The author discusses in-depth when to use this functional paradigm and when it’s better to stick with the imperative classic Java approach. You’ll find many examples for the use of the Function and Predicate classes of Guava. Although Suppliers are introduced in this chapter, too, I personally think the Supplier classes were not discussed extensive enough because in my opinion Guavas Suppliers are way too underrated and I use them very frequently.

Collections

The Guava originated from the google-collections library and so it’s no surprise that a big part of Guava are utils for working with Collections or completely new collection classes. The book demonstrates the use of all relevant collection utils and covers additional collection classes like Immutable Collections, Table, BiMap, etc. This chapter also introduces the Ordering class as a complement to classic Comparators.

Concurrency

Although very short, this is one of the chapters I enjoyed most in the book. It covers the most important classes in Guava for dealing with concurrency and introduces Monitor, ListenableFuture, FutureCallback, AsyncFunction and RateLimiter. I learned a lot here and especially Monitor and ListenableFuture are classes which I intend to use more. Especially ListenableFuture is interesting as it can completely replace classic Java Futures and are more powerful.

Cache

Guavas Cache implementation is very useful and is a lightweight alternative to libraries like EHCache. It’s widely adopted and so it’s no surprise that the Cache gets an extra chapter. For people who are looking for a lightweight, configurable and powerful local cache implementation, this chapter covers everything you will ever need to know about Guavas Cache.

Eventbus

I was excited to see that Guavas Eventbus is covered in a whole chapter because I use the EventBus regularly and it’s an easy and lightweight way to decouple your application. The chapter discusses how to use the event bus and and what advantage you get in your application using it. The code examples are great and show how easy it is to decouple your application components.
Finally it’s discussed how to integrate the Eventbus with your Dependency Injection Framework for convenient usage. The code examples are for spring which is a bit irritating, I personally think Google Guice would be a better fit. The concepts for integrating are the same, though.

Files

The eight chapter discusses the shortcomings of classic Java File handling and how Guava fills this hole. It introduces many utility classes for manipulating files, file contents and folders. I personally get bored quickly when reading about file handling, but this is nothing I can blame the author for.

Other utilities

The last chapter covers some other utilities form Guava which are worth mentioning. It introduces Hashing functionality, Bloom Filters, and the super handy Optional class. I am using the later heavily and I think this class should have deserved more than just a very short introduction.

Conclusion

I was very excited from the book, it covers all relevant utils from Guava thoroughly and I learned a lot. The author decided to let code speak and included many code snippets for every topic discussed in the book. I can absolutely recommend this practical book for anyone who is not familiar with Guava (yet) and even long time users of the library will learn some new and exciting stuff.

MQTT Table Football with Arduino, Raspberry Pi and Websockets

One of our main motivation refreshers in our office is our football table and we use it heavily every day. Someone came up with the idea: „Hey, why don’t we add some freaking MQTT support to the football table?“. Of course there was no argument against it and so we added the MQTT support and used a Arduino One for the job. To make things more interesting we decided against a mechanical goal trigger and used infrared sensors for detecting goals. To raise the motivation even more we used a Raspberry Pi to play goal celebration sounds on actual goals, notified via MQTT, of course.

To raise the nerd factor a bit more, we also decided to remove the built-in goal counter (which needed human interaction count up) and built a very basic small web application which acted as goal counter by using websockets to get MQTT messages when a goal was shot. This web application also implemented the logic when a player has won and published messages to the MQTT broker.

MyMQTT for Android was used as a remote control to reset the game with. With MQTT, of course.

The HiveMQ MQTT broker was the heart of the communication. All communication was done via MQTT with the HiveMQ broker.

Architectural Overview

MQTT Table Football Architecture

MQTT Table Football Architecture

In Action

We brought the football table to an event a few days ago and it was the absolute highlight of the event. Everyone had great fun.