Simplifying IoT Architecture

I’ve been working with IoT projects for the last couple of years. A very common pattern is illustrated in the figure below:

Your devices are sending in data. The Azure IoT hub is used as cloud gateway or ingestion point. You persist all the incoming messages so you can retrieve them later and at the same time you forward them to the Event Hub for (near) real-time processing. This is not illustrated, but you can have a consumer picking off messages from the Event Hub.

Until recently you had to create two consumer groups on your IoT hub and have Azure Stream Analytics do the forwarding. I’ve shown two jobs here, but depending on the load, you might have been able to do with just one, including two select statements.

With the introduction of Endpoints and Routes in the IoT hub and the Archive functionality in th Event Hub this pattern can be simplified quite a lot, cutting out components, hence making the architecture simpler, more manageable and more robust.

The new pattern is illustrated below:

We now use endpoints and routes to forward the messages to the Event Hub. It is possible to use filtering, so if this was done in Azure Stream Analytics it is not a problem.

Messages are archived to blob storage directly from the Event Hub. Note, that archived data is written in Apache Avro format.