Data Protocols for Smart City
Comparison of data protocols for Smart City — MQTT vs. CoAP
In the architecture context, Smart Cities are composed of heterogeneous technology elements with different power, communication and computation characteristics. Smart City includes sensor nodes and gateways that have limited battery and computational resources. Hence, Smart City systems require protocols for data transmission that are, bandwidth and energy efficient and need small memory footprint in the implementation. These requirements dictate that protocols for Smart City applications must be very different from those that facilitate the data exchange in conventional Internet applications.
Most of the Internet traffic is made out of utilization of HTTP over TCP medium. However, HTTP is verbose, complex and not suitable for deployment on the constrained devices found in Smart City applications. HTTP has human-readable format which was very helpful for the communication between people, however the communication between machines demands a different approach.
Current trends in data protocols for Smart City applications tend to be so called “messaging” protocols where series of messages are exchanged between the device and cloud. Among available data protocols, two of them became very popular for Smart City, namely, Message Queue Telemetry Transport — MQTT and Constrained Application protocol — CoAP. Both protocols were invented to accommodate ultra-constrained Internet connected devices requirements such as energy efficiency, lightweight, bandwidth friendly and reliability in environments dominated with intermittent connectivity.
MQTT was first proposed by the engineers at IBM and Arcom and was standardized in 2013 at OASIS. MQTT has a client — server model where IoT nodes act as a client that connects to a server known as broker. MQTT was designed to be a lightweight protocol for devices with limited computational abilities to send data over constrained bandwidth networks. Possible scenario for the adaptation of MQTT was intended to be interconnected sensor networks that operate on the scale of thousands of nodes deployed in harsh physical environments.
The architecture of MQTT protocol consists of a message broker, a client subscriber, and a client publisher. The broker maintains topic subscription state and information about the clients. A client can publish messages on one or more topics, can subscribe to one or more topics and can both simultaneously publish and subscribe. MQTT is implemented on top of the TCP IP and has a small overhead with minimized protocol exchanges to reduce network traffic. It also uses the publish/subscribe message mechanism so that clients can broadcast messages to many different channels. The protocol has three levels of quality of service (QoS);
- QoS 0 — “at-most-once” message delivery. This quality of service is suitable to be used in situations where loss of messages is acceptable. In the Smart City context this QoS can be used with ambient monitoring sensors where loosing data once in a while is not critical for understanding of the variables being sensed.
- QoS 1 — “at-least-once” message delivery. This quality of service guaranties the delivery of sent messages; however messages can be delivered more that once.
- QoS 2 — “exactly once” message delivery. This quality of service is used when the duplicate of messages is not permitted and message delivery must be guaranteed. In the Smart City context, this QoS is suitable for the more mission critical application (health, traffic, etc).
MQTT has a small overhead and protocol exchanges are minimized to reduce network traffic. Figure 1a. illustrates the publish subscribe process that the protocol is based on and Figure 2b. shows the MQTT message format. The message format is as follows; Message Type specifies the type of message including connect, connack (notification about disconnection), publish, subscribe, etc.; DUP stands for message duplication; quality of service is identified in the QoS field; Retain field stand for message retention, any message can be retained and publish as the first message to a new subscriber; and Remaining Length field indicates the remaining length of the message.
Constrained Application protocol (CoAP) is a protocol designed for the “constrained” environments in relation to the energy, computational power and bandwidth of communication of devices that employ this protocol. CoAP was designed specifically to keep the message overhead small. The protocol was created by the IETF Constrained RESTful Environments (CoRE) working group and is officially defined as the application layer protocol for IoT, and as such, is very popular data protocol for Smart City applications as well. The CoAP is based on REpresentational State Transfer (REST) which is a simple method for exchange of the data between HTTP clients and HTTP servers. However, CoAP differs from the REST because it is based on UDP and not TCP which makes CoAP more suitable for constrained Smart City applications. Furthermore, CoAP also modifies HTTP to meet the low power and resource constrained operations (IoT applications). Additionally, since the CoAP was designed to resemble REST, two protocols can easily be translated using REST-CoAP proxies.
The protocolconsists of two layers; messaging and request/response layer. Due to the implementation of CoAP over UDP, transport layer the error recovery mechanism doesn’t exist. The error recovery is handled on the messaging level where duplicates of the messages are detected and certain level of reliability is implemented. The request/response layer on the other hand handles modified REST like communication. CoAP utilizes four types of messages: confirmable, non-confirmable, reset and acknowledgement. Confirmable and non-confirmable messages are implemented to increase the reliability of the CoAP protocol. It can operate in four different modes as depicted in Figure 2.
The CoAP message format is shown in Figure 3. Messages are encoded in a simple and small format. Each message consists of following fields; four bytes header; a token value (up to 8 bytes), options and payload which are all optional fields.
The main advantages of CoAP protocol are;
- the ability to subscribe to the resource on demand through publish/subscribe mechanism;
- ability to reduce communication overhead by exchanging data between client and server in blocks (partial update of data);
- resource discovery is easy through URI paths based on the web link fields in CoRE link format;
- easy interaction with HTTP through a simple proxy implementation.
Two discussed protocols can be compared both qualitatively and quantitatively. Qualitatively, the comparison can be made regarding the communication pattern, transport protocol, reliability, design orientation, message detainment (caching), fragmentation and discovery features. It is carried out based on official protocol documents.
- MQTT is a pub/sub protocol and CoAP is a REST protocol. Additionally, CoAP has a built-in support of a pub/sub mechanisms using Observe mechanism. Having both options makes CoAP a more flexible solution.
- MQTT is built on top of TCP which is a connected-oriented protocol, whereas CoAP is built on top of UDP, a connectionless internet protocol. As such MQTT is expected to be more reliable than CoAP while having higher overhead.
- MQTT has three QoS levels and CoAP is based on the NON/CON messages. These quality levels can be matched in a sense that MQTT’s QoS0 corresponds to CoAP’s NON, MQTT’s QoS1 corresponds to CoAP’s CON and MQTT QoS2 has no match with CoAP which makes MQTT more favorable when it comes to reliability.
- MQTT can be preferred for Smart City scenarios where devices need to report small updates in short periods of time (e.g. mobility tracking). Whereas CoAP is a document centric protocol that includes short identifiers of media types allowing for exchange of different content types with a minimal overhead.
- Implementation of CoAP is such that information about the validity and “freshness” of the messages is contained inside the CoAP responses. It is able to cache responses accordingly. On the other hand, MQTT doesn’t have such feature, but it has complementary features that nearly correspond to caching; retained messages.
- Smart city applications that involve picture and video transmission usually deal with transfer of substantial amounts of data. MQTT does not have inherit capability of message fragmentation and therefore large data transmission becomes difficult and unreliable. On the other hand, CoAP integrates the mechanism called Blockwise where large data sets can be transmitted in blocks of data.
- Classic MQTT does not support the discovery mechanism while the later version of MQTT namely MQTT-SN developed exclusively for sensor networks has this feature. CoAP on the other hand has discovery mechanism as a part of the standard specification.
Experimental Case Studies
Due to the popularity of MQTT and CoAP protocols many researchers investigated how do these two protocol compared quantitively in terms of performance criteria. Thangavel et.al. studied two protocols performance using a common middleware. Aggregated sensor data was transported from the gateway node to the server/broker. The performance of two protocols were compared experimentally and following conclusion were drawn;
- MQTT messages have lower delays than CoAP for lower packet loss and higher delays than CoAP for higher packet loss;
- in relation to the message size, when message size is small and loss rate is equal to or less than 25%, CoAP generates less extra traffic than MQTT.
The main conclusion from these authors was that MQTT is better in terms of reliability but generates more traffic.
In a separate study DeCaro et.al. compared these two protocols on the experimental basis as well. They also achieved a similar result where CoAP gives better results both in terms of bandwidth usage and round trip time. In terms of reliability, MQTT performs better due to the additional control mechanisms. However, MQTT’s reliability is only dominant when the data needs to be exchanged frequently.
The main limitation of both protocols is that the communication security is not explicitly addressed in the protocol implementation. One of the main reasons for this is due to their design goal of being lightweight with small overheads. Nevertheless the extensive security implementation is becoming more of a necessity than luxury in Smart City applications. Both MQTT and CoAP benefit from the security related to their underlying medium of communication, TCP and UDP respectively. MQTT can be implemented over the Secure Sockets Layer (SSL) and CoAP uses Datagram Transport Layer Security (DTLS). Still, other layers of security such as user protection, sensitive content protection and exchange of enhanced encryption keys are still missing. Possibly, MQTT and CoAP could embed application level security algorithms in the near future.
MQTT and CoAP, two most popular protocols for data exchange in constrained environments such as Smart City were analyzed. The thorough analysis uncovered main underlying features of these two protocols, their main advantages and disadvantages, their comparative analysis and their limitations. MQTT is identified as reliable pub/sub protocol for situations where tiny amounts of data need to be exchanged between client and server frequently. The integration of MQTT with the existing Internet services is somewhat complex and MQTT does not rate well when it comes to resource exposure. On the other hand, CoAP is based on the REST architectural style and easily integrates with the existing Internet services. However, CoAP is based on the UDP and inherently less reliable than MQTT . CoAP produces smaller amounts of traffic and is often more preferred protocol for constrained applications. Main limitation of both protocols is the lack of security at the transport level. They both rely on the security schemes of the underlying TCP or UDP, but fail to fulfill basic needs of user protection, sensitive content protection and exchange of enhanced encryption keys.