by Georg Held

Moving IoT data with MQTT

feature
Dec 20, 20226 mins
Cloud ComputingSoftware DevelopmentTechnology Industry

Billions of connected devices need to communicate and move data effectively across networks and cloud platforms. Here’s why MQTT is the standard protocol.

UK | United Kingdom  >  London  >  skyline with virtual connections

The internet of things (IoT) continues to expand. Transforma Insights projects the number of connected devices to exceed 13.1 billion worldwide this year. With that vast number of devices comes the need for them to communicate and move data across networks and cloud platforms, which is where IoT protocols come into play. 

The MQTT protocol was invented in 1999 by Andy Stanford-Clark and Arlen Nipper to monitor oil and gas pipelines over satellite networks. Over the past several years it has become the de facto standard for IoT messaging. This article will discuss the basics of MQTT, its benefits for IoT data movement, and how to get started using MQTT.

MQTT publish/subscribe architecture

Standardized by OASIS and ISO, the MQTT protocol provides a scalable and reliable way to connect devices over the internet. MQTT is an extremely lightweight publish/subscribe messaging protocol ideal for connecting remote devices with a small code footprint and minimal network bandwidth. MQTT’s decoupled architecture reduces the complexity of integrating and connecting various applications and equipment. Many industries employ MQTT, including automotive, manufacturing, telecommunications, and oil and gas. 

mqtt publish subscribe HiveMQ

The MQTT publish/subscribe architecture.

The publish/subscribe architecture shown in the figure above decouples the publisher (the client that sends the message) from the subscribers (clients that receive the messages). They never contact each other directly; instead an MQTT broker handles the connections, correctly filtering and distributing all of the incoming messages to subscribers. Each MQTT message includes a topic. MQTT clients subscribe to the topics they want to receive, and the MQTT broker dispatches the messages to the appropriate clients.

An MQTT broker can buffer messages that can’t be dispatched to MQTT clients when they are disconnected. This becomes very useful for situations where network connections are unreliable. In support of reliable message delivery, the protocol supports three different types of quality of service (QoS) messages:

0 – at most once 1 – at least once 2 – exactly once

QoS gives the client the power to choose a level of service that matches its network reliability and application logic.

MQTT allows persistent sessions between the client and the broker, which enables sessions to continue even if the network is disconnected. Because MQTT holds messaging information during non-network connectivity, the system remembers the client-to-broker session once reconnected. This level of reliability is one of the key features that makes the MQTT protocol more efficient than HTTP for use over unreliable cellular networks.

Moving large amounts of data to and from tens of thousands of connected devices and applications requires a messaging protocol that fits this purpose. The inventors explicitly created MQTT to answer the need to monitor remote equipment with unreliable connectivity. Its architecture is ideally suited for moving IoT data.

Technical and business benefits of MQTT

There are several reasons technical architects and software developers choose to work with MQTT and why IoT platforms support and promote the protocol as the ideal way to get data into their platforms. The technical benefits:

  • Lightweight and efficient: MQTT reports by exception and message headers are very small, minimizing the resources required for the client and network bandwidth.
  • Bi-directional: MQTT allows messaging from devices to the cloud and from the cloud to devices, which enables broadcasting messages to groups of things.
  • Supports unreliable networks: MQTT’s support for persistent sessions reduces the reconnection time required over unreliable networks.
  • Scalable: MQTT can scale to connect millions of IoT devices simultaneously. 
  • Security enabled: MQTT makes it easy to encrypt messages using TLS (transport layer security) and authenticate clients using modern authentication protocols, such as OAuth.
  • Reliable: The publish/subscribe functionality and the ability to queue messages means no data loss through MQTT. MQTT also supports reliable message delivery through the three defined quality of service levels.

The technical benefits of MQTT drive multiple business benefits, making MQTT a good choice for any company looking to digitally transform data collection and communication to make better business decisions. The business benefits:

  • Reduced network costs: Protocols such as HTTP and OPC UA have larger message sizes and also use poll/response to collect data at a set interval, even if the data hasn’t changed. The small footprint of MQTT messages plus reports by exception saves network bandwidth costs.
  • Improved customer experience: MQTT’s focus on reliability and support for persistent sessions enhances the customer experience. In the case of BMW’s car-sharing service, when using SMM/HTTP to transmit messages, a customer could wait up to 30 seconds to unlock a car door. Moving to MQTT solved the issue.
  • Reduced costs from data loss: Lost data can negatively impact businesses in multiple ways such as missing key events, inaccurate data collection, and even loss of revenue when data loss impacts the customer. MQTT message queueing reduces costs associated with data loss because it sends data when the network reconnects. 
  • Faster time-to-market: The client/broker relationship in the MQTT architecture ensures easy rapid iteration, which means products can get to market faster. Once an organization sets up an MQTT broker as the hub of the system, any number of new applications or clients can be added or updated quickly with minimal effort at any time.

Getting started with MQTT

The OASIS MQTT Technical Committee manages two versions of the specification: MQTT 3.1.1 and MQTT 5. Most commercial MQTT brokers now support MQTT 5, but many IoT-managed cloud services only support MQTT 3.1.1. New IoT deployments should use version 5 due to the new features focusing on more robust systems and cloud-native scalability. 

MQTT 5 is a significant update of the MQTT protocol. In response to the feedback from MQTT users, MQTT 5 adds the features that modern IoT applications need. These new features cater to applications deployed to the cloud, applications that require robustness and reliable error handling to implement mission-critical messaging, and application developers who seek easier integration of MQTT messages into their existing computing infrastructure.

Over the next few years, we expect to see massive growth in MQTT adoption across all industries, including manufacturing, automotive, critical infrastructure, logistics, and smart cities. MQTT is on the verge of becoming the standard for all IoT.

Many open source MQTT clients and brokers are available in various programming languages. You can find a detailed list on mqtt.org. Once the basics are understood, the best way to get started with MQTT is to study real-world example deployments. You can find several examples in our GitHub repository.

For a more in-depth description of the protocol, we recommend that you read the MQTT Essentials series of articles or review the companion video series. The MQTT 5 Essentials series also provides an in-depth introduction to the specific features of MQTT 5.

Georg Held is the head of product development at HiveMQ. He is responsible for the development of the HiveMQ Enterprise MQTT Platform and HiveMQ ecosystem.

New Tech Forum provides a venue to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to newtechforum@infoworld.com.