Implementation and Analysis of QUIC for MQTT · Implementation and Analysis of QUIC for MQTT Puneet...

1

Implementation and Analysis of QUIC for MQTTPuneet Kumar and Behnam Dezfouli

Internet of Things Research Lab Department of Computer Engineering Santa Clara University USApkumarbdezfouliscuedu

AbstractmdashTransport and security protocols are essential toensure reliable and secure communication between two partiesFor IoT applications these protocols must be lightweight sinceIoT devices are usually resource constrained Unfortunately theexisting transport and security protocols ndash namely TCPTLS andUDPDTLS ndash fall short in terms of connection overhead latencyand connection migration when used in IoT applications Inthis paper after studying the root causes of these shortcomingswe show how utilizing QUIC in IoT scenarios results in ahigher performance Based on these observations and given thepopularity of MQTT as an IoT application layer protocol weintegrate MQTT with QUIC By presenting the main APIs andfunctions developed we explain how connection establishmentand message exchange functionalities work We evaluate theperformance of MQTTwQUIC versus MQTTwTCP using wiredwireless and long-distance testbeds Our results show thatMQTTwQUIC reduces connection overhead in terms of thenumber of packets exchanged with the broker by up to 56 Inaddition by eliminating half-open connections MQTTwQUICreduces processor and memory usage by up to 83 and 50respectively Furthermore by removing the head-of-line blockingproblem delivery latency is reduced by up to 55 We also showthat the throughput drops experienced by MQTTwQUIC whena connection migration happens is considerably lower than thatof MQTTwTCP

Index TermsmdashInternet of Things (IoT) Transport layer Ap-plication layer Latency Security

I INTRODUCTION

The Internet of Things is the enabler of many applica-tions such as the smart home smart cities remote medicalmonitoring and industrial control by connecting a largenumber of sensors and actuators to the Internet Existingstudies predict that the number of connected devices willsurpass 50 billion by 2020 [1] To facilitate interconnectionand software development the communication between IoTdevices usually employs a protocol stack similar to that ofregular Internet-connected devices such as smartphones andlaptops Specifically IP (or 6LowPAN [2]) and transport layerprotocols are provided by various protocol stacks (eg microIP[3] LwIP [4]) to enable interconnectivity

A TCP and UDP

The primary responsibility of the transport layer is to sup-port exchanging segments between the two end-to-end com-municating applications Among the transport layer protocolsTCP (Transport Layer Protocol) and UDP (User DatagramProtocol) are the most widely used depending on the appli-cation at hand TCP provides a reliable end-to-end connectionand implements congestion control mechanisms to avoid bufferoverflow at the receiver During the past couple of decades

several improved versions of TCP have been proposed to ad-dress the increasing demand for throughput [5] [6] Howeverthese features impose high overhead in terms of connectionestablishment and resource (ie processor memory energy)utilization UDP on the other hand does not provide any ofthe above-mentioned features and therefore its overhead issignificantly lower than that of TCP

While throughput is the main performance metric for usertraffic such as voice and video the prevalent communicationtype of IoT which is machine-to-machine (M2M) is character-ized by short-lived bursts of exchanging small data chunks [7][8] In addition compared to user devices such as smartphonesand laptops IoT devices are usually resource constrained interms of processing memory and energy [9] [10] Using TCPin IoT domains to satisfy reliability and security requirementstherefore imposes high overhead Specifically the shortcom-ings of TCP when used in IoT applications are as followsndash Connection startup latency is highly affected by the TCP

handshake This handshake requires 1 Round-Trip Time(RTT) for TCP and 2 or 3 RTTs when TLS (TransportLayer Security) is added to this protocol [11] The overheadimpact is even higher in IoT scenarios where unreliablewireless links cause frequent connection drops [12] Inthese scenarios imposing a high connection establishmentoverhead for the exchange of a small amount of data resultsin wasting the resources of devices TCP Fast Open [13]seeks to address this problem by piggybacking data in SYNsegments in repeated connections to the same server Thissolution is not scalable since the TCP SYN segment canonly fit a limited amount of data [14]

ndash IoT devices are often mobile and as such supportingconnection migration is an essential requirement [15]ndash[18]However any change in network parameters (such as IPaddress or port) breaks the connection In this case eitherthe connection must be re-established or a gateway isrequired to reroute the data flow Unfortunately these so-lutions increase communication delay and overhead whichmight not be acceptable in mission-critical applications suchas medical monitoring [8]

ndash To preserve energy resources IoT devices usually transitionbetween sleep and awake states [19] [20] In this case aTCP connection cannot be kept open without employingkeep-alive packets These keep-alive mechanisms howeverincrease resource utilization and bandwidth consumptionWithout an external keep-alive mechanism IoT devices areobliged to re-establish connections every time they wakefrom the sleep mode

ndash In disastrous events such as unexpected reboots or a devicecrash TCP connections between client and server might

arX

iv1

810

0773

0v2

[cs

NI]

3 J

an 2

019

2

end up out of state This undefined state is referred to asTCP half-open connections [21] A half-open connectionconsumes resources such as memory and processor time Inaddition it can impose serious threats such as SYN flooding[22] [23]

ndash If packets are dropped infrequently during a data flowthe receiver has to wait for dropped packets to be re-transmitted in order to complete the packet re-ordering Thisphenomena which impedes packet delivery performance iscalled the head-of-line blocking [24]ndash[26]Despite the aforementioned shortcomings several IoT ap-

plication layer protocols rely on TCP and some of themoffer mechanisms to remedy these shortcomings For exampleMQTT [27] employs application layer keep-alive messagesto keep the connection alive This mechanism also enablesMQTT to detect connection breakdown and release the re-sources

Another transport layer protocol used in IoT networksis UDP Generally UDP is suitable for applications whereconnection reliability is not essential Although this is notacceptable in many IoT scenarios several IoT application layerprotocols rely on UDP due to its lower overhead compared toTCP These protocols usually include mechanisms to supportreliability and block transmission (eg CoAP [28])

B TLS and DTLS

In addition to reliability it is essential for IoT applicationsto employ cryptographic protocols to secure end-to-end dataexchange over transport layer TLS [29] is the most commonconnection-oriented and stateful client-server cryptographicprotocol Symmetric encryption in TLS enables authenticatedconfidential and integrity-preserved communication betweentwo devices The key for this symmetric encryption is gener-ated during the TLS handshake and is unique per connectionIn order to establish a connection the TLS handshake canrequire up to two round-trips between the server and clientHowever since connections might be dropped due to phe-nomenons such as sleep phases connection migration andpacket loss the overhead of establishing secure connectionsimposes high overhead In order to address this concern alighter version of TLS for datagrams named DTLS (DatagramTransport Layer Security) [30] has been introduced UnlikeTLS DTLS does not require a reliable transport protocol asit can encrypt or decrypt out-of-order packets Therefore itcan be used with UDP Although the security level offeredby DTLS is equivalent to TLS some of the technical dif-ferences include the adoption of stream ciphers is prohib-ited and an explicit sequence number is included in everyDTLS message Compared to TLS DTLS is more suitablefor resource-constrained devices communicating through anunreliable channel However similar to TLS the underlyingmechanism in DTLS has been primarily designed for point-to-point communication This creates a challenge to secure one-to-many connections such as broadcasting and multicasting Inaddition since DTLS identifies connections based on sourceIP and port number it does not support connection migration[31] Furthermore DTLS handshake packets are large and may

fragment each datagram into several DTLS records where eachrecord is fit into an IP datagram This can potentially causerecord overhead [32]

C Contributions

Given the shortcomings of UDP and TCP we argue that theenhancement of transport layer protocols is a necessary step inthe performance improvement of IoT applications In order toaddress this concern this paper presents the implementationand studies the integration of QUIC [33] with application layerto address these concerns QUIC is a user space UDP-basedstream-based and multiplexed transport protocol developedby Google According to [14] around 7 of the world-wideInternet traffic employs QUIC This protocol offers all thefunctionalities required to be considered a connection-orientedtransport protocol In addition QUIC solves the numerousproblems faced by other connection-oriented protocols suchas TCP and SCTP [34] Specifically the addressed problemsare reducing the connection setup overhead supporting mul-tiplexing removing the head-of-line blocking supporting con-nection migration and eliminating TCP half-open connectionsQUIC executes a cryptographic handshake that reduces theoverhead of connection establishment by employing knownserver credentials learned from past connections In additionQUIC reduces transport layer overhead by multiplexing severalconnections into a single connection pipeline Furthermoresince QUIC uses UDP it does not maintain connection statusinformation in the transport layer This protocol also eradicatesthe head-of-line blocking delays by applying a lightweightdata-structure abstraction called streams

At present there is no open source or licensed version ofMQTT using QUIC Current MQTT implementations (suchas Paho [35]) rely on TCPTLS to offer reliable and securedelivery of packets Given the potentials of QUIC and itssuitability in IoT scenarios in this paper we implement andstudy the integration of MQTT with QUIC First since the datastructures and message mechanisms of MQTT are intertwinedwith the built-in TCP and TLS APIs it was necessary toredesign these data structures The second challenge was toestablish IPC (Inter-Process Communication) between QUICand MQTT MQTTwTCP utilizes the available APIs foruser space and kernel communication However there is noavailable API for QUIC and MQTT to communicate as theyare both user space processes To address these challengeswe have developed new APIs which are referred to as agentsSpecifically we implemented two types of agents server-agent and client-agent where the former handles the broker-specific operations and the latter handles the functionalities ofpublisher and subscriber This paper presents all the functionsdeveloped and explains the connection establishment andmessage exchange functionalities by presenting their respec-tive algorithms The third challenge was to strip QUIC ofmechanisms not necessary for IoT scenarios QUIC is a webtraffic protocol composed of a heavy code footprint (15GB[36]) We have significantly reduced the code footprint (toaround 22MB) by eliminating non-IoT related code segmentssuch as loop network and proxy backend support

3

Client ServerServer

Inchoate CHLO

REJ

Complete CHLO

Encrypted request

SHLO

Encrypted response

Initial 1-RTT Handshake(a)

Encrypted request

Complete CHLO

SHLO

Encrypted response

Successful 0-RTT Handshake(b)

Client

1-RTT

DataExchange

DataExchange

Fig 1 The messages exchanged by QUIC during (a) 1-RTT and (b) 0-RTTconnections

Three types of testbeds were used to evaluate the perfor-mance of MQTTwQUIC versus MQTTwTCP wired wire-less and long-distance Our results show that in terms of thenumber of packets exchanged during the connection estab-lishment phase MQTTwQUIC offers a 562 improvementover MQTTwTCP By eliminating half-open connectionsMQTTwQUIC reduces processor and memory usage by upto 832 and 503 respectively compared to MQTTwTCPFurthermore by addressing the head-of-line blocking problemMQTTwQUIC reduces message delivery latency by 556compared to MQTTwTCP In terms of connection migrationthe throughput drop experienced by MQTTwQUIC is signif-icantly lower than that of MQTTwTCP

The rest of this paper is organized as follows Section IIexplains the QUIC protocol along with its potential bene-fits in IoT applications The implementation of QUIC forMQTT is explained in Section III Performance evaluationand experimentation results are given in Section IV SectionV overviews the existing studies on QUIC and IoT applicationlayer protocols The paper is concluded in Section VI

II QUIC

QUIC employs some of the basic mechanisms of TCP andTLS while keeping UDP as its underlying transport layerprotocol QUIC is in fact a combination of transport andsecurity protocols by performing tasks including encryptionpacket re-ordering and retransmission This section overviewsthe main functionalities of this protocol and justifies theimportance of its adoption in the context of IoT

A Connection Establishment

QUIC combines transport and secure layer handshakes tominimize the overhead and latency of connection establish-ment To this end a dedicated reliable stream is provided forthe cryptographic handshake Figure 1(a) and (b) show thepackets exchanged during the 1-RTT and 0-RTT connectionestablishment phases respectively Connection establishmentworks as followsndash First handshake In order to retrieve the serverrsquos config-

uration the client sends an inchoate client hello (CHLO)message Since the server is an alien to the client theserver must send a REJ packet This packet carries the serverconfiguration including the long-term Diffie-Hellman value

Flags (8)

Connection identifier (64) (optional)

QUICHeader

Version (32) (Client-only optional)

Diversification Nonce (256)

Packet Number (8-48)

Type (8) Stream ID (8-32) Offset (0-64)

Data length (0 or 16)

Stream Data (data length)

Frame 1 Frame 2 Frame N

Fig 2 The solid and dashed lines show the clear-text and encrypted partsof a QUIC packet respectively The non-encrypted part is used for routingand decrypting the encrypted part of the packet

connection ID (cid) port numbers key agreement and ini-tial data exchange After receiving the serverrsquos configurationthe client authenticates the server by verifying the certificatechain and the signature received in the REJ message At thispoint the client sends a complete CHLO packet to the serverThis message contains the clientrsquos ephemeral Diffie-Helmanpublic value This concludes the first handshake

ndash Final and repeat handshake After receiving the completeCHLO packet the client has the initial keys for the connectionand starts sending application data to the server For 0-RTT the client must initiate sending encrypted data withits initial keys before waiting for a reply from the serverIf the handshake was successful the server sends a serverhello (SHLO) message This concludes the final and repeathandshake

Except some handshake and reset packets QUIC packetsare fully authenticated and partially encrypted Figure 2 showsthe non-encrypted and encrypted parts of the packet usingsolid and dotted lines respectively The non-encrypted packetheader is used for routing and decrypting the packet contentThe flags encode the presence of cid and the length ofthe Packet Number (PN) field which are visible to read thesubsequent fields

B Connection Migration

The QUIC connections are identified by a randomly gen-erated 64-bit Connection Identifier (cid) A cid is allocatedper connection and allows the clients to roam between net-works without being affected by the changes in the networkor transport layer parameters As shown in Figure 2 cidresides in the header (non-encrypted part) and makes theclients independent of network address translation (NAT) andrestoration of connections The cid plays an important rolein routing specifically for connection identification purposesFurthermore using cids enables multipath by probing a newpath for connection This process is called path validation [37]During a connection migration the end point assumes thatthe peer is willing to accept packets at its current addressTherefore an end point can migrate to a new IP addresswithout first validating the peerrsquos IP address It is possiblethat the new path does not support the current sending rate ofthe endpoint In this case the end point needs to reconstituteits congestion controller [38] On the other hand receivingnon-probe packets [39] from a new peer address confirms thatthe peer has migrated to the new IP address

4

C Security

For transport layer encryption MQTTwTCP usually relieson TLSSSL The primary reasons why TLSSSL cannot beused in QUIC were described in [40] In short the TLSsecurity model uses one session key while QUIC uses twosession keys This difference in particular enables QUIC tooffer 0-RTT because data can be encrypted before the final keyis set Thus the model has to deal with data exchange undermultiple session keys [41] Therefore MQTTwQUIC usesits own encryption algorithm named QUIC Crypto [42] Thisalgorithm decrypts packets independently to avoid serializeddecoding dependencies The signature algorithms supported byCrypto are ECDSA-SHA256 and RSA-PSS-SHA256

D Multiplexing

Unlike TCP QUIC is adept in transport layer header com-pression by using multiplexing Instead of opening multipleconnections from the same client QUIC opens several streamsmultiplexed over a single connection Each connection isidentified by a unique cid The odd cids are for client-initiated streams and even cids are for server-initiated streamsA stream is a lightweight abstraction that provides a reliablebidirectional byte-stream A QUIC stream can form an appli-cation message up to 264 bytes Furthermore in the case ofpacket loss the application is not prevented from processingsubsequent packets Multiplexing is useful in IoT applicationswhere a large amount of data transfer is required per trans-action For example this feature enhances performance forremote updates and industrial automation [43]

E Flow and Congestion Control

Similar to TCP QUIC implements a flow control mecha-nism to prevent the receiverrsquos buffer from being inundated withdata [14] A slow TCP draining stream can consume the entirereceiver buffer This can eventually block the sender fromsending any data through the other streams QUIC eliminatesthis problem by applying two levels of flow control (i)Connection level flow control limits the aggregate buffer thata sender can consume across all the streams on a receiver (ii)Stream level flow control limits the buffer per stream level AQUIC receiver communicates the capability of receiving databy periodically advertising the absolute byte offset per streamin window update frames for sent received and deliveredpackets

QUIC incorporates a pluggable congestion control algorithmand provides a richer set of information than TCP [44] Forexample each packet (original or re-transmitted) carries a newPacket Number (PN) This enables the sender to distinguishbetween the re-transmitted and original ACKs hence removingTCPrsquos re-transmission ambiguity problem QUIC utilizes aNACK based mechanism where two types of packets arereported the largest observed packet number and the unseenpackets with a packet number lesser than that of the largestobserved packet A receive timestamp is also included in everynewly-acked ACK frame QUICrsquos ACKs can also provide thedelay between the receipt of a packet and its acknowledge-ment which helps in calculating RTT QUICrsquos ACK frames

QUIC Server

MQTT Broker

Server-agentApplication

Layer

Transport Layer

QUIC Client

Client-AgentMQTT Publisher

Broker

Publisher

Application Layer

Transport Layer

QUIC Client

Client-AgentMQTT Subscriber

Subscriber

Application Layer

Transport Layer

UDP

IP

UDP

IP IP

UDP

Network Layer

Network Layer

Network Layer

MQTT Broker

Server-Agent

QUIC Server

MQTT Publisher

Client-Agent

QUIC Client

MQTT Subscriber

Client-Agent

QUIC Client

Fig 3 The high-level architecture of the proposed implementation

support up to 256 NACK ranges in opposed to the TCPrsquos 3NACK range [45] This makes QUIC more resilient to packetreordering than TCP (with SACK) The congestion controlalgorithm of QUIC is based on TCP Reno to determine thepacing rate and congestion window size [45] In additionQUIC supports two congestion control algorithms (i) PacingBased Congestion Control Algorithm (PBCCA) [46] and (ii)TCP CUBIC [47] The superior performance of QUICrsquos flowcontrol over TCP for HTTP traffic has been demonstrated inthe literature [48]

III INTEGRATION AND IMPLEMENTATIONOF MQTT WITH QUIC

This section presents the integration of MQTT with QUICand is divided into six sub-sections The first sub-sectionoverviews the system architecture The second sub-sectiondescribes the definitions methods and assumptions The thirdand fourth sub-sections explain the operations of the APIs andfunctions developed for the broker and clients respectivelyWe present the common APIs and functions which are used bythe broker and client in the fifth sub-section A short discussionabout code reduction is presented in the sixth sub-section Inorder to simplify the discussions we refer to the publisher andthe subscriber as client The presented implementation for theclient and broker are based on the open-source Eclipse Pahoand Mosquitto [35] respectively

A Architecture

Both MQTT and QUIC belong to the application layer Tostreamline their integration either the QUIC library (ngtcp2[49]) must be imported into MQTT or new interfaces mustbe created However the former approach is not suitablefor resource-constrained IoT devices as QUIC libraries aredeveloped mainly for HTTPHTTPS traffic and thereforeimpose a heavy code footprint In our implementation wechose the latter approach and built a customized broker andclient interfaces for MQTT and QUIC We refer to theseinterfaces as agents which enable IPC between the MQTTand QUIC Figure 3 shows a high level view of the protocolstack architecture

5

initialize_rx_tx_msg_queue

process_rx_packets

quic_input_message initialize_rx_tx_msg_queue

mqtt_message_initializer

mqtt_client_connect initialize_quic

populate_quic_header

start_connect

quic_stream_gen

do_handshake

do_handshake_once

protocol_connectconnect1-RTT

Resume0-RTT

crypt_quic_message

quic_dispatcher decrypt encryptmqtt_parse_args

Server-Agent Client-Agent

Common APIs

If session files found

If session files not found If session files not found


Fig 4 Server-agent and client-agent are two entities between QUIC andMQTT This figure represents the packet processing flow by the server andclient agents

Since QUIC uses UDP connection-oriented features suchas reliability congestion control and forward error correctionare implemented in QUIC In addition QUIC incorporatescyptographic shields such as IP spoofing protection and packetreordering [50] To keep the QUIC implementation lightweightand abstracted we segregated the implementation into twoparts (i) the QUIC client and server only deal with UDPsockets and streams (ii) the agents deal with reliability andsecurity

Figure 4 shows the high-level view of the server and clientagent implementations The implemented entities are as fol-lows (i) Server-Agent APIs and functions They handle serverspecific roles such as accepting incoming UDP connectionssetting clientsrsquo state and storing and forwarding packets tothe subscribers based on topics (ii) Client-Agent APIs andfunctions They perform client-specific tasks such as openinga UDP connection and constructing QUIC header and streams(iii) Common APIs and functions They are utilized by boththe server and client However their tasks differ based on theirroles

After initializing the transmit and receive messagequeues by initialize_rx_tx_msg_queue() the serverand client agents process the messages differently Inthe server-agent the received message queue is fed toquic_input_message() This API begins the handshakeprocess to negotiate the session keys However if the client hascontacted the server in the past then quic_input_message()is directly available for the crypt_quic_message() to parsethe incoming MQTT message On the other hand the client-agent first processes the MQTT message and then sends thatmessage to the server by using start_connect() Based onthe communication history between the server and client thestart_connect() API either enters or skips the handshakeprocess with the server The common APIs handle the hand-shake control messages

B Definitions Methods and AssumptionsThis section presents the primitives used to present and

explain the implementations Table I shows the main acronymsand notations used in the rest of this paper We chose most ofthese notations based on the RFCs published relevant to thiswork

Notation 0 1lowast represents the set of all finite-length binarystrings If a and b are two uniquely decodable strings then(a b) simply represents the concatenation of both If κ isin Nthen 1κ represents the κ consecutive strings of 1 bits Thenotation s

$minusrarr S represents the uniform random selectionof s from a finite set S The set of integers [1 n] isrepresented by [n] where n isin N We also assume that a publickey infrastructure (PKI) is available This means that publickeys are user-identity bounded valid and publicly knownTherefore certificates and their verification are excluded fromthe implementation

A Digital Signature Scheme (SS) with message space Msgis used by the broker during a connection establishment toauthenticate certain data The scheme is defined as follows

SS = (Kg Sign Ver) (1)

where Kg Sign and Ver are the randomization key genera-tion algorithm signing algorithm and verification algorithmrespectively The input of Kg is the security parameter λ andits output is a public and secret key pair as follows

Kg(λ) $minusrarr (pk sk) (2)

The signing algorithm returns a signature

Sign(skm)$minusrarr σ (3)

where sk is the secret key and m isin Msg The verificationalgorithm is denoted as follows

Ver(pkm σ)rarr p (4)

where pk is the public key and p isin 0 1 The output value pwhich is a bit shows whether the signature is valid or invalid

The requirement for correctness of SS is Ver(pkmSign(skm)) = 1 for every m isin Msg and Kg(λ) Correctness isdefined by the requirement that the input of one partyrsquos msend

be equal to the output of the other partyrsquos mgetSecure channel implementation is based on an authenti-

cated encryption with associative data schemes (AEAD) [52]AEAD consists of two algorithms E and D First E is adeterministic encryption algorithm defined as follows

E(k noncmH)rarr c (5)

where c is ciphertext nonc isin 0 1n message m isin 0 1lowastH isin 0 1lowast is an additional authenticated data and key k isdefined as

0 1λ $minusrarr κ (6)

Second D is a deterministic decryption algorithm defined as

D(κ noncH E)rarr pl or perp (7)

where pl is plaintextThe correctness requirement of AEAD is

D(κnoncH E(κnoncH m)) equiv m for all κ isin 0 1λnonc isin 0 1n H and m isin 0 1lowast

6

TABLE IKEY NOTATIONS AND ABBREVIATIONS

Variable Descriptionperp Represents a rejected messageλ Protocol-associated security parameter used to derive the session keys

κ Input key 0 1κ $minusrarr κE Deterministic algorithm used in AEADH SHA-256 functionσ Signature used in digital signature scheme

(pk sk) Key pair representing public key and secret key

a Derived by primes of size λ $minusrarr aaction_flag Flag used to determine if packet is subject to transmission or processing

b Derived by generators of Za$minusrarr b

C Client (ie publisher or subscriber)CHLO Client hello message in QUIC aka inchoate hello (c_i_hello)cid Connection identifier

client_info Struct used in storing client info such as cid socket etcclient_initial Clientrsquos initial state which is the state immediately after receiving REJ message from server

D Deterministic algorithm used in AEADDH Diffie-Helman public valuesdup Flag in MQTT (value 1 indicates the packet is a retransmission)HMAC Key hash message authentication used in expansion of keys [51]ik Initial key variable set in the initial phase of QUICrsquos connection establishment

init bit initializer init isin 0 1iv Initialization vectork Key to derive data in final_data() phaseKg Key generation algorithm takes λ as security parameter and generates (pksk) as key pair in QUICkstk Derived same as session key (for simplicity in our implementation we treat kstk as random string to replicate unpredictable input)M Message to be sent or received by client or server in QUICm The message part of Msg

msgid Message ID for MQTT queuing

Msg Message space used in QUIC(Msg consists of bitstrings starting with a 1 while key exchange messages to be encrypted start with a 0)

nonc Noncepub Public DH Valuespk Public keyREJ Reject message (message from server after CHLO in QUIC)

retained Flag used in MQTT to prevent loosing subscribed topics when a connection loss happensRTT Round-trip timeS Server

scfg Variable (output by scfg_gen()) to represent the global state of server in QUICscid Serverrsquos cidSHLO Server Hello packet (s_hello) in QUICsk Secret keysqn Sequence number (used in signalling for every segment in QUIC)stk Source address token to guard IP-spoofing in QUIC

strikerng Strike variable used by QUIC

C Server-agent

Algorithm 1 shows the implementation of these APIs andfunctions Our server-agent implementation is event-basedWhen an event packet_process occurs the server-agent de-termines whether to transmit the packet or insert it intothe receiving queue Server-agent APIs and functions aredescribed as follows

1) main() If the packet is subject to transmission thenaction_flag is set otherwise the packet will be processedby process_rx_packets() This function also handles thedisconnect events If the server receives a disconnect eventthen particular clients will be disconnected However theserver might receive a disconnect event for all the clients Thishappens for example when a reboot event occurs

2) process_rx_packets() It processes the packets inthe receive queue by quic_input_message()

3) quic_input_message() This API first checkswhether the handshake process with the client has been

completed or not In order to determine this it checksthe handshake_completed flag If it has been set then itassumes that the connection is alive and handshake has beencompleted In this case the API skips the handshake processand enters the do_handshake_once() API to set the flag forencryption or decryption If the handshake_completed flag isnot set then the API runs the handshake process by executingdo_handshake() where the client and server either followthe 1-RTT or 0-RTT implementation This API also sets theclientrsquos state according to its operation For instance if theclient is contacting the server for the first time then theclientrsquos state is set to client_initial If the handshake failsthen the clientrsquos state is set to client_handshake_failedIf the handshake has to be skipped then it sets the clientrsquosstate to client_post_handshake

7

Algorithm 1 Server-agent APIs and functions1 function main()2 Initialize tx and rx queues 3 initialize_rx_tx_msg_queue()4 is_client = false5 while true do6 if event = process_packet then7 if packet must be transmitted then8 action_flag = send9 else

10 int sock =recvfrom()11 client_infosock = sock12 process_rx_packets()

13 else if event = client_disconnect socket_timeout then14 disconnect socket15 break

16 function process_rx_packets()17 while rx_message_queue do18 quic_input_message()

19 return

20 function quic_input_message()21 if (handshake_completed) then22 if (do_handshake()) then23 client_state = client_handshake_failed24 return error25 else26 client_state = client_initial

27 else28 client_state = client_post_handshake 29 do_handshake_once()

30 return

D Client-agent

This section deals with the APIs and functions specificallyused by the client-agent Algorithm 2 shows the client agentimplementation

1) main() Similar to the server-agent the main() func-tion initializes both the transmit and receive queues by callinginitialize_rx_tx_queue() The client-agent is also event-driven and processes both incoming and outgoing packetsbased on the packet_process event The main() functionchecks action_flag to determine if the packet is meant tobe sent or processed by process_rx_packets()

2) mqtt_message_initializer() This API creates anMQTT client instance via MQTTClient_create() and setsthe client ID the persistence parameter (retained) and theserver IP address When a subscriber connects to a brokerit creates subscriptions for all the topics it is interested inWhen a reboot or reconnect event occurs the client needsto subscribe again This is perfectly normal if the client hasnot created any persistent sessions The persistence parametercreates a persistent session if the client intends to regain allthe subscribed topics after a reconnection or reboot event

Immediately after the creation of aclient instance a message is initialized byMQTTClient_connectOptions_initializer()This API sets the msgid dup retainedpayload and version fields of this packetMQTTClient_connectOptions_initializer() createsa directory to store all the topics and includes this in

Algorithm 2 Client-agent APIs and functions1 function main()2 initialize_tx_rx_msg_queue()3 is_client = true4 while true do5 if event = process_packet then6 if packet must be transmitted then7 int sock =create_udp_socket()8 action_flag = send9 mqtt_message_initializer()

10 else11 insert(rx_msg_queue)12 process rx_msg_queue for received packets


16 function mqtt_message_initializer()17 if (MQTTClient_create()) then18 return error19 if (MQTTClient_connectOptions_initializer()) then20 return error21 if (mqtt_client_connect()) then22 return error23 return

24 function mqtt_client_connect()25 if (sanity(thisrarr msg)) then26 return error27 initialize_quic()28 return

29 function initialize_quic()30 filling quic header 31 if (clientrarr cid) then32 clientrarr cid = get_cid() Assign cid to client 33 populate_quic_header()34 start_connect()35 return

36 function start_connect()37 quic_stream_gen()38 if (handshake_completed) then39 do_handshake()40 else41 do_handshake_once()

42 return

43 function quic_stream_gen()44 Check stream presence 45 if stream_present then46 if if_stream_closed then47 clear_entry()48 return error49 else50 Find stream 51 stream =find_stream()

52 else53 Create a stream and associate it with the cid54 streamrarr cid = cid55 stream =create_stream_buf()

56 return

client_info to be easily retrieved later

3) mqtt_client_connect() At this point all the mes-sage construction functionalities related to MQTT are com-pleted and finally the client instance is ready for QUIC relatedoperations Sanity checking (eg header size data lengthetc) is performed before entering the main QUIC API (ieinitialize_quic())

8

Algorithm 3 Common APIs and functions1 function do_handshake()2 if (protocol_connect) then3 return error4 handshake_completed = true5 if (do_handshake_once()) then6 return error7 return

8 function do_handshake_once()9 if action_flag = send then

10 flag = encrypt 11 else12 flag = decrypt

13 if (crypt_quic_message()) then14 return error15 return

16 function crypt_quic_message()17 if flag = decrypt then18 thisrarr msg = decrypt_message(thisrarr msg)19 quic_dispatcher()20 else if flag = encrypt then21 thisrarr msg = encrypt_message(thisrarr msg)22 insert(thisrarr msg tx_msg_queue)23 return

24 function protocol_connect()25 if session files not found then26 connect() 1-RTT Implementation 27 else28 resume() 0-RTT Implementation

29 return

30 function quic_dispatcher()31 if is_client then32 If the client is calling the API 33 process rx_msg_queue for mqtt application34 return35 else36 MQTT parsing for received messages 37 if (valid_mqtt_header(thisrarr msg)) then38 mqtt_parse_args (thisrarr msg)39 return40 else41 return error

42 return

4) initialize_quic() This API first determines if theclient has an assigned cid and if not then a new cid isgenerated by 0 164 $minusrarr cid The QUIC header is initializedafter cid assignment The non-encrypted part of the QUICheader consists of cid diversification nonce packet number orsequence number pointer to payload size of the payload sizeof the total packet QUIC version flags and encryption levelThe encrypted part of the QUIC header is made of frameswhere each frame has a stream ID and a final offset of astream The final offset is calculated as a number of octets ina transmitted stream Generally it is the offset to the end ofthe data marked as the FIN flag and carried in a stream frameHowever for the reset stream it is carried in a RST_STREAMframe [37] Function populate_quic_header() fills boththe encrypted and non-encrypted parts of the QUIC headerand enters the API start_connect() to connect the client

5) start_connect() This API is the entering point tothe common APIs (described in Section III-E) Its first task isto find an appropriate stream for the connection Based on the

Algorithm 3 Common APIs and functions (continued)1 function encrypt_message()2 if (handshake_completed) then3 Initial session key is used 4 get_iv(H ik)rarr iv5 if iv was used then6 return perp7 else8 For client 9 if is_client then

10 return (H E(ikc iv H m)11 For server 12 else13 return (H E(iks iv H m)

14 else15 The stored established key is used 16 get_iv(H k)rarr iv17 if iv is used then18 return perp19 For client 20 if is_client then21 return (H E(kc iv H m)22 For server 23 else24 return (H E(ks iv H m)

25 function decrypt_message()26 Extracting ciphertext27 mrarr ci28 if (handshake_completed) then29 initial data phase key is used 30 get_iv(H ik)rarr iv31 For client 32 if is_client then33 if D(ikc iv H ci) 6= perp then34 return plain_text35 else36 return perp37 else38 For Server 39 if D(iks iv H ci) 6= perp then40 return plain_text

41 else42 Final data phase key is used 43 get_iv(H k)rarr iv44 For client 45 if is_client then46 if D(kc iv H ci) 6= perp then47 return plain_text48 else49 return perp50 else51 For server 52 if D(ks iv H ci) 6= perp then53 return plain_text

handshake_completed flag start_connect() API deter-mines if the client and server have completed the handshake Ifthe handshake has been completed then it means that QUICconnection is alive and data can be encrypted or decryptedIf not then it enters the handshake process by executing thedo_handshake() API

6) quic_stream_gen() The only purpose of this API isto detect and create streams As mentioned earlier QUIC iscapable of multiplexing several streams into one socket ThisAPI first detects whether there is already an open stream for

9

Algorithm 4 1-RTT Implementation1 function connect()2 if initial_key_exchange() then3 if initial_data() then4 if key_settlement() then5 if final_data() then6 return success7 return error

8 function initial_key_exchange()9 message m

10 For client 11 if is_client then12 switch phase do13 case initial do14 pk is public key 15 c_i_hello(pk)rarr m116 break

17 case received_reject do18 m2 received packet from server (REJ) 19 c_hello(m2)rarr m320 break

21 case complete_connection_establishment do22 ik is initial key variable set during initial phase 23 get_i_key_c(m3)rarr ik24 break

25 For server 26 else27 switch phase do28 case received_chlo do29 s_reject(m1) rarr m230 break

31 case received_complete_chlo do32 get_i_key_s(m3) rarr ik33 break

34 return

35 function initial_data()36 For client 37 if is_client then38 for each α isin [i]39 α + 2 rarrsqnc40 sqnc = Client sequence number 41 Mα

c = Client constructed message 42 pak(ik sqnc Mα

c ) rarr mα4 43 process_packets(ikm5)44 For server 45 else46 for each β isin [j]47 β + 1 rarr sqns48 sqns = Server sequence number49 Mα

s = Server constructed message 50 pak(ik sqnsM

βs ) rarr m

51 process_packets(ikm4)

52 return

a particular client and if not then it creates a new stream byrunning create_stream_buf()

E Common APIs and functions

Several APIs and functions such as encryption decryptionand processing transmission packets are mandated on both theserver and client sides Although the packets handled by thesefunctions are different in the client-agent and server-agent theunderlying mechanisms are almost similar Algorithm 3 showsthe implementation

Algorithm 4 1-RTT Implementation (continued)1 function key_settlement()2 if is_client then3 get_key_c(m6 sqns) rarr k4 else5 2 + j rarr sqns6 s_hello(m ik sqns) rarr m67 get_key_s(m6) rarr k

8 return

9 function final_data()10 if is_client then11 for each α isin i + 1u12 α + 2 rarrsqnc13 pak(k sqnc Mα

c )rarr mα7

14 (mi+17 mu)rarr m7

15 process_packets(km7)16 else17 for each β isin j + 1w18 β + 2 rarrsqns19 pak(k sqns Mβ

s ) rarr mβ8

20 (mj+18 mw)rarr m8

21 process_packets(k m8)

22 return

1) do_handshake() This API is the starting point of theQUIC connection establishment and key exchange processIt behaves as an abstraction of the handshake process Firstit calls protocol_connect() for protocol communicationIf protocol_connect() is executed successfully then thehandshake_completed flag is set Based on this flag the clientdetermines whether to execute the handshake process or skipit Lastly this API calls the do_handshake_once() API toset the encrypt or decrypt flag for further processing

2) do_handshake_once() The primary responsibility ofthis API is to set the encryption or decryption flags Thisdecision is made based on the type of the next operationwhich is transmission or processing If action_flag is set thenthe packet must be encrypted and sent Finally it enters thecrypt_quic_message() API

3) crypt_quic_message() This API is thestarting point in establishing a secure connection Itchecks the flag to determine if the packet shouldundergo decryption (decrypt_message()) or encryption(encrypt_message())

4) protocol_connect() In this API the client (C)checks the existence of a session file to determine whetherit has communicated with the server (S) during the last τtseconds If C and S are interacting for the first time thenthe connect() API completes the 1-RTT scenario in fourphases as shown in Figure 5 and Algorithm 4 If C and Shave communicated before then the resume() API followsthe 0-RTT scenario shown in Figure 6 and Algorithm 7

1-RTT Connection The 1-RTT implementation is dividedinto four phases The first phase exchanges the initial keysto encrypt the handshake packets until the final key is setThe second phase starts exchanging encrypted initial data Thethird phase sets the final key Last the fourth phase startsexchanging the final data We explain the details of thesephases as follows

[1-RTT] Phase 1 This phase is handled by

10

Clientmc pkj

Serverms

Send m1 = c_i_hello(pkj)

Send m2 = s_reject(m1)

Send m3 = c_hello(m2) initial_key_exchange

Send m4 = pak(ik sqnc mc)

Send m5 = pak(ik sqns ms) initial_data

Send m6 = s_hello(m3 ik sqns)

key_settlement

Send m7 = pak(k sqnc mc)

Send m8 = pak(ks sqns ms)

Server key ksget_key_s(m6)Client key

kcget_key_c(m6sqns) final_data

Fig 5 The four phases of the 1-RTT connection

Algorithm 5 Server Configuration State1 function get_scfg(sk τt λ)

2 Zaminus1$minusrarr xs

3 bxs minusrarr ys4 (b a ys)rarrpubs5 xs rarrsecs6 τt+1 rarrexpy7 H(pubs expy) rarrscid H is a SHA-256 function 8 QUIC Server Config Signature rarr str9 Sign(sk (str 0x00 scid pubs expy))rarrprof

10 (scid pubs expy) rarrscfgtpub11 return scfg

initial_key_exchange() and consists of three messagesm1m2 and m3 The client C runs c_i_hello(pk) whichreturns a packet m1 with sequence number 1 m1 is an initialconnection packet sent to S containing a randomly generatedcid In response to m1 S sends a REJ packet m2 generatedusing s_reject(m1) m2 contains a source-address tokenstk (similar to TLS session tickets [53]) which is later usedby C to prove its identity to S for the ongoing session andfuture sessions This is performed by checking if the sourceIP address equals the IP address in stk Fundamentally thestk consists of an encryption block of Crsquos IP address anda timestamp In order to generate stk S uses the same Edeterministic algorithm (ie AEAD) with kstk (derived by0 1128 $minusrarr kstk) The initialization vector for stk (ieivstk) is selected randomly and is used in s_reject Forsimplicity we implemented a validity range for stk whichis bounded by the time period during which it was eithergenerated or set up Another important parameter in m2 is theSprimes current state scfg_pub (refer to Algorithm 5) It containsSrsquos DH values with an expiration date and a signature profThis signature is signed by SS over all the public valuesunder the Sprimes secret key sk

Upon receiving m2 the client C checks scfgtpub for itsauthenticity and expiration The algorithms for this purposecan be found in [52] As we mentioned in Section III-Bour implementation assumes that a PKI is in place Afterpossessing the public key of S the client generates a noncand DH values by running c_hello(m2) and sends them

to the server in message m3 At this point both C and Sderive the initial key material ik using get_i_key_c(m3)and get_i_key_s(m3) respectively The server keeps trackof the used nonc values in order to make sure that it does notprocess the same connection twice This mechanism is referredas strike-register or strike The timestamp is included in thenonc by the client The server only maintains the state ofa connection for a limited duration of time Any connectionrequest from a client is rejected by the server if its nonc isalready included in its strike or contains a timestamp thatis outside the permitted range strikerng The initial key ik= (ikc iks iv) is made of two parts two 128-bit applicationkeys (ikc iks) and two 4-byte initialization vector prefixesiv = (ivc ivs) The client uses iks and ivs to encryptthe data and send it to S On the other hand ikc and ivcassist in decryption and encryption This phase happens onlyonce during the time period τt until scfgtpub and stk are notexpired

[1-RTT] Phase 2 This phase is handled byinitial_data() and consists of two messages m4

and m5 The client C and server S exchange the initial datamessage Mc and Ms which are encrypted and authenticatedwith ik in function pak(iksqnc M i

α) for every α isin [i]and pak(ik sqns M i

β) for every β isin [j] respectivelyHere sqnc and sqns represent the sequence number ofpackets sent by C and S respectively i and j representthe maximum number of message blocks that C and S canexchange prior to the first phase The initialization vector ivis generated based on the server or client role When S sendsa packet get_iv() outputs iv by concatenating ivc andsqns When C sends a packet get_iv() generates iv byconcatenating ivs and sqnc The total length of each iv is12 bytes since both the server and client initialization vectorprefixes (ie ivc and ivs) are 4 bytes in length and sequencenumbers (ie sqns and sqnc) are 8 bytes in length WhenC receives packets from S it uses the process_packets()function to decrypt those packets to extract their payloadsand concatenates them based on their sequence numberThe server S performs a similar mechanism for the packetsreceived from C

[1-RTT] Phase 3 This phase is handled bykey_settlement() and involves message m6 The serverS produces new DH values (authenticated and encryptedvia AEAD with ik) and transmits them to the client usings_hello(m3ik sqn) The client verifies the serverrsquos newDH public values with the help of ik At this point the serverand client both derive the session key by get_key_s(m6)and get_key_c(m6) and use extract_expand() for keyexpansion as defined in Algorithm 6

[1-RTT] Phase 4 This phase is handled by final_data()and consists of two messages m7 and m8 Instead of initialkey ik the established key k is used to encrypt and authen-ticate the remaining data (Msg) by both C and S Similar toik k is derived from kc ks and iv and consists of twoparts the two 128-bit application keys (kc ks) and the two4-bytes initialization vector prefixes iv = (ivcivs) In orderto encrypt the data C uses ks and ivs before sending to SFor decryption C uses ivc and kc to decrypt the data received

11

Algorithm 6 QUIC messages exchange APIs and func-tions

1 function c_i_hello(pk)

2 0 164 $minusrarrcid3 return IPc IPs portc ports cid 1

4 function s_reject(m)

5 0 196 $minusrarrivstk6 (ivstk E(kstk ivstk ε 0) (IPc current_times))rarr stk7 return IPsIPcportsportccid1scfgtpub prof stk8 prof is generated by get_scfg

9 function c_hello(m)10 if expy le τt then11 return12 str = rdquoQUIC server config signaturerdquo13 if Ver(pk (str 0X00scid pubs expy) prof)) 6= 1 then14 return

15 0 1160 $minusrarr r16 (current_timec r)rarr nonc

17 Zaminus1$minusrarr xc bxc rarr yc (b a yc)rarr pubc

18 (IPc IPs portc ports)rarr pkt_info19 return (pkt_info cid 2 stk scid nonc pubc)

20 function get_i_key_c(m)

21 yxcs$minusrarripm

22 return extract_expand(ipm nonc cid m 40 1)

23 function get_i_key_s(m)24 stkrarr(ivstk tk)25 D(kstk ivstk ε tk)rarr d26 if (d = perp ( first 4 bytes of d 6= 0) then27 if ( first 4 bytes of d) 6= IPc) then28 return29 return

30 if last 4 bytes corresponds to outside strikerng then31 return32 if r isin strike τt isin strikerng then33 return34 if scid is unknown then35 return36 if scid corresponds to expired scfgt

lsquo

pub then37 where tlsquo lt t 38 return39 if b a isin pubc 6= b a isin pubs then40 return

41 yxsc rarripm42 return extract_expand(ipmnonccidm401)

43 function extract_expand(ipm nonc cid m l init)44 HMAC(noncipm)rarr ms45 if init = 1 then46 rdquoQUIC key expansionrdquorarr str47 else48 rdquoQUIC forward secure key expansionrdquorarr str

49 (str 0X00 cid m scfgtpub) rarrinfo50 return first l bytes (octets) of T = (T(1) T(2))51 if all i isin N T(i) = HMAC(ms (T(i-1) info 0x0i)) and

T(0) = ε

from S0-RTT Connection Another scenario of connection es-

tablishment is 0-RTT as Figure 6 shows If C has alreadyestablished a connection with S in the past τt seconds then Cskips sending c_i_hello() and initiates another connectionrequest to the server by sending a c_hello() packet Thispacket contains the existing values of stk scid cid noncand pubc It is important to note that pubc requires new DHephemeral public values After receiving c_hello S verifies

Algorithm 6 QUIC messages exchange APIs and func-tions (continued)

1 function get_iv(Hκ)2 κc = Client key3 κs = Server key4 ivc = Client initialization vector5 ivs = Server initialization vector 6 κrarr (κc κs ivc ivs)7 if is_client then8 crarr src srarr dst9 else

10 srarr src crarr dst

11 sqn is packet sequence number 12 H rarr(cid sqn)13 return (ivdst sqn)

14 function pak(k sqn m)15 κrarr (kc ksivc ivs)16 if is_client then17 crarr src and srarr dst18 else19 srarr src and crarr dst

20 (IPsrc IPdst portsrc portdst) rarr pkt_info21 (cid sqn)rarr H22 get_iv(Hκ)rarr iv23 return (pkt_info E(kdstivH (1 m)))

24 function process_packets(κ p1 p2pv)25 κrarr (kc ks ivc ivs)26 if is_client then27 crarr src and srarr dst28 else29 srarr src and crarr dst

30 for each γ isin [v] 31 pγ rarr (Hγ cγ)32 get_iv(Hγ κ)rarr ivγ 33 D(ksrc ivγ Hγ cγ)rarr mγ34 if mγ isin Msg then35 return36 return (m1 m2 mv)

37 function s_hello(m3 ik sqn )38 ik rarr (ikc iks ivc ivs)

39 Zaminus1$minusrarr xs bxs rarr ys (b a ys)rarr pubs

40 (cid sqn)rarr H41 E(ikc (ivc sqn) H (0 (pubs stk)))rarr e42 return (ipsipcportsportc H e)

43 function get_key_s(m)44 yxs

c rarrpms45 return extract_expand(pms nonc cid m 40 0)

46 function get_key_c(m)47 mrarr(IPsIPcports portc cid sqn e)48 if D(ikc(ivc sqn)(cidsqn)e) = perp then49 return50 if first bit of the message 6= 0 then51 return52 yxcs rarr pms53 return extract_expand(pms nonc cid m 40 0)

if the nonc is fresh This is performed against strike-registerprovided that stk is valid and scid is not unknown or expiredIf these verification steps do not succeed then S returns to the1-RTT process by generating and sending out a s_rejectmessage (Algorithm 6) If these verification steps succeedthen the rest of the protocol remains the same Algorithm 7shows the implementation for 0-RTT

5) quic_dispatcher() This API is the starting pointwhere MQTT-related parsing starts After the QUIC header is

12

Algorithm 7 0-RTT Implementation1 function resume()2 if (pubc) then3 if stk then4 if scid then5 return c_hello(stk scfgtpub)6 Jumping back on 1-RTT 7 return connect()

8 function c_hello(stk scfgtpub)

9 0 164 $minusrarrcid

10 0 1160 $minusrarr r (current_timec r) rarr nonc


12 (IPc IPsportcports)rarrpkt_info13 return (pkt_info cid 1 stk scid nonc pubc)

Clientmc pkj

Serverms

Send m3 = c_hello(stk scfgpubt)

initial_dataSend m6 = s_hello(m3 ik sqns)

key_settlement



Server key ksget_key_s(m6)

Client key kcget_key_c(m6sqns)

final_data

Fig 6 In 0-RTT scenario client sends the first packet c_hello with previousserver global state scfg and strike This step itself sends initial data to server

stripped off and decrypted the packet is delivered to MQTTHere the broker finds the topic and delivers the message toall the subscribers that have subscribed to that topic

F Code Reduction

QUIC is a part of the Chromium Projects [33] Theseprojects have a heavy code size because they include sev-eral features such as the Chrome browser SPDY protocolChromecast Native Client and QUIC with their entwinedimplementations In order to reduce the code size we haveremoved all these features except those that are essentialto the functionality of QUIC Furthermore to reduce itsoverhead most of the logging features have been removedor disabled We have also eliminated alarm_factory whichgenerates platform-specific alarms The host resolver has beenremoved as well As most clients have an in-built DNS pre-fetching [54] feature they can resolve DNS queries withouthaving to communicate with a server Since MQTT routesdata flows based on topics instead of URLs this feature is notrequired We have also removed the backend proxy-relatedcode Backend proxy is used when the proxy server is behindthe firewall and load balances the requests from clients to theservers Since MQTT is based on the publishsubscribe modelit is futile to involve any proxy or load balancing Finally weremoved all the unit-testing code

(a) Wired (b) Wireless (c) Long Distance

Subscriber

Internet

1Gbps Ethernet Switch

Broker

80211n Wireless Access Point (24GHz)

Publisher Publisher Subscriber

Broker

BrokerLocated in

Washington DC

Subscriber


Publisher

Located in Santa Clara

California

Fig 7 The three testbeds used for the performance evaluations

IV EVALUATION

This section presents the performance evaluation ofMQTTwQUIC versus MQTTwTCP

In order to measure the impact of the physical layer anddistance on performance three different testbeds are usedwired wireless and long-distancendash Wired All the nodes are connected to a Netgear 1Gbps

L2-learning switchndash Wireless The nodes communicate through an 80211n

network The access point used is a Linksys AC1200 andoperates on channel 6 of the 24GHz band In additionwe placed two other similar access points 3 meters awayform the main access point In order to introduce latencyand packet drops these two access points operate on thesame channel and therefore interfere with the main accesspoint Each interfering access point continuously exchangesa 10Mbps UDP flow with a nearby user

ndash Long Distance Our objective in using the long-distancerouted network is to introduce longer and unpredictable end-to-end delays The subscriber and publisher are placed inSanta Clara California and the broker is situated acrossthe country in Washington DCRaspberry Pi 3 model B is the device used as the subscriber

publisher and broker These devices run Raspbian Stretch astheir operating system As mentioned in the previous sectionthe MQTT implementation is based on Eclipse Paho andMosquitto In addition the TLS version is 12 the sockettimeout for TCP and QUIC is 30 seconds and the keep-alivemechanism of MQTT has been disabled Figure 7 shows thesetestbeds

When presenting the results each point is the median ofvalues obtained in 10 experiments where each experimentincludes 10 iterations For example when measuring theoverhead of connection re-establishment the subscriber andpublisher are connected to the broker 10 times and the resultis counted as one experiment The iterations of an experimentare run consecutively and the minimum time interval betweenthe experiments is 5 minutes

The rest of this section studies the performance ofMQTTwQUIC versus MQTTwTCP in terms of the overheadof connection establishment head-of-line blocking half-openconnections resource utilization and connection migration

A Overhead of Connection EstablishmentThis section evaluates the number of packets exchanged be-

tween devices when using MQTTwTCP and MQTTwQUIC

13

Subscriber Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)

Connect Command (MQTT)ACK (TCP)

Connect ACK (MQTT)

ACK (TCP)

Subscriber Request (MQTT)Subscriber ACK (MQTT)

ACK (TCP)

Client Hello (TLS)

Server Hello (TLS)

Client Key Exchange (TLS)Server Key Exchange (TLS)

Client Finished (TLS)Server Finished (TLS)

Publish Message (MQTT)

ACK (TCP)Publish ACK (MQTT)

Data Exchange

Connection Establishment

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

Publisher Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)

Connect Command (MQTT)

ACK (TCP)

Connect ACK (MQTT)


Publish ACK (MQTT)

Client Hello (TLS)

Server Hello (TLS)

Client Key exchange (TLS)

Server Key exchange (TLS)

Client Finished (TLS)

Server Finished (TLS)

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

ACK (TCP)


Data Exchange

Publisher Broker

Inchoate Hello (QUIC)

REJ (QUIC)

Complete Hello (QUIC)

Encrypted Request (QUIC)

Server Hello-SHLO (QUIC)

Encrypted Response (QUIC)


Connect ACK (MQTT)


Publish ACK (MQTT)


Data Exchange

1

2

3

4

5

6

7

8

9

10

Subscriber Broker


REJ (QUIC)Complete Hello (QUIC)Encrypted Request (QUIC)


Encrypted Response (QUIC)Connect Command (MQTT)

Connect ack (MQTT)

Subscriber Request (MQTT)

Subscriber ack (MQTT)

Publish Message (MQTT)Publish ack (MQTT)


Data Exchange

1

2

3

4

5

6

7

8

9

10

11

12

Subscriber BrokerComplete Hello (QUIC)





Connect ack (MQTT)


Subscriber ACK (MQTT)


Publish ACK (MQTT)

Data Exchange

Publisher Broker

1

2

3

4

5

6

7

8

9

10






Connect ACK (MQTT)


Publish ACK (MQTT)

1

2

3

4

5

6

7

8

Data Exchange

(a) MQTTwTCP

(b) MQTTwQUIC 1-RTT

(c) MQTTwTCP 0-RTT

Fig 8 The connection establishment process using (a) MQTTwTCP (b)MQTTwQUIC 1-RTT and (c) MQTTwQIUC 0-RTT

during the connection establishment process Before present-ing the results we first study the sequence of packet exchangesbetween a client and a server Figures 8(a) (b) and (c)show this sequence in MQTTwTCP and MQTTwQUICrsquos 1-RTT and 0-RTT respectively Please note that these figuresdemonstrate ideal scenarios when there is no packet drop Inaddition MQTT ping packets are excluded to simplify theevaluations Figure 9 and Table II summarize the results from

0

5

10

15

20

25

30

35

Num

ber o

f Pac

kets

MQTTwTCPMQTTwQUIC 1-RTTMQTTwQUIC 0-RTTWiredLong DistanceWireless

Subscriber Publisher Broker

Fig 9 The number of packets exchanged during the connection establish-ment phase in the wired wireless and long-distance testbeds The error barsshow the lower quartile and higher quartile of the collected results

the subscriber publisher and broker point of views using thethree testbed types mentioned earlier

As the results show MQTTwQUIC reduces the numberof packets exchanged with the broker by up to 5625 Inaddition the wireless testbed shows the highest performanceimprovement compared to the other two testbeds which is dueto its higher packet loss rate Therefore since the number ofpackets required by MQTTwQUIC is lower in this scenariocompared to MQTTwTCP the total number of packet re-transmissions is lower Compared to the wired testbed thelong-distance testbed shows a higher performance improve-ment In the wired network all the three devices are in samebroadcast domain therefore no layer-3 routing is performedThis an ideal situation since all the devices are connecteddirectly through a LAN and the probability of packet loss isalmost zero However packets might be lost or significantlydelayed in the long-distance testbed due to events such ascongestion and multipath As IoT applications such as smarthomes usually include lossy links these results indicate thesignificant potential benefits of using QUIC in these scenarios

B Head-of-Line Blocking

The head-of-line blocking problem occurs when the receiveris waiting for the dropped packets to be re-transmitted in orderto complete the packet reordering before delivering them tothe application To simulate this problem in our testbeds aFreeBSD [55] router was introduced in between the publisherand broker

FreeBSD enables us to intercept packets based on the fire-wall rules and their network buffers During this experimentwhich ran for 300 seconds data packets are intercepted basedon their flow ID The flow ID is a unique tuple of sourceIP address and source port to identify the connection TheFreeBSDrsquos firewall function ipfw_chk is used to interceptthe packets and drop them In order to replicate a real-world scenario data packets were dropped randomly at fixedintervals For instance to simulate a 10 drop scenario every10th packet belonging to the same ID is dropped In theseexperiments latency is computed as the interval between theinstance a packet leaves the publisher until the reception ofthat packet by the subscriber To accurately measure latencywithout introducing extra traffic WiringPi [56] has beenintegrated into our testbed For each received message the

14

TABLE IIPERFORMANCE IMPROVEMENT OF MQTTWQUIC VERSUS MQTTWTCP IN TERMS OF THE NUMBER OF PACKETS EXCHANGED DURING THE

CONNECTION ESTABLISHMENT PHASE

Testbed Wired Wireless Long DistanceDevice Subscriber Publisher Broker Subscriber Publisher Broker Subscriber Publisher Broker

1-RTT 3684 3333 3529 4818 4285 4791 4583 3529 40Improvementvs MQTTwTCP 0-RTT 4736 4666 4705 5555 5238 5625 5416 4705 50

receiver uses the WiringPi library to notify the sender bygenerating a signal to toggle a pin connected to the sender

Table III summarizes the results for all the three testbedsDropping random packets intermittently forces the sender toretransmit the lost packets As the results show TCPrsquos latencyis higher than QUICrsquos in all the scenarios This is due to thepacket re-ordering delay in TCP which happens when a lineof packets is being held up by the receiver when prior packetsare lost In contrast QUIC is based on UDP which does nothold up the received packets while the lost packets are beingre-transmitted This enables the receiverrsquos application layer toperform the decryption operation as soon as packets arrive

In the wireless and long distance testbeds apart fromthe intentional drops introduced both MQTTwQUIC andMQTTwTCP show higher latencies as they suffer from theadditional packet drops compared to the wired testbed There-fore the improvement margins between MQTTwTCP andMQTTwQUIC in wireless and long distance testbeds arelower than the wired network For example for the 10 droprate scenario MQTTwQUIC reported 13833 ms and 71099ms latencies in the wired and wireless testbeds respectivelyTherefore the highest improvement achieved is for wirednetworks

C TCP Half-Open Connections

An act of receiving data in TCP is passive which meansthat a dropped connection can only be detected by the senderand not by the receiver In order to simulate the TCP half-open connection problem 100 connections were establishedby 10 publishers using different topics and client IDs Onemessage is transmitted per second over each connection Togenerate TCP half-open connections the keep-alive messagetransmission mechanism of MQTT has been disabled Thehalf-open connections are detected using Linux command ss-a In order to reveal the overhead of half-open connectionsmemory usage and processor utilization level were measuredusing the Linuxrsquos top utility To generate half-open connec-tions all the publishers are restarted in the middle of a messageflow In this scenario the broker is unaware of the connectiontear-down

Figures 10 and 11 show the resource utilization of thebroker Both figures present the results of a single experimentto reveal the variations of memory and processor utilizationversus time The duration of this experiment is long enough toinclude the process of client connection subscription publish-ing and restart of the clients After each of the aforementionedoperations we wait for the processor and memory utilizationto become stable and then start the next operation These

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

200

400

600

800

Mem

ory

Usa

ge (M

B)

MQTTwTCPMQTTwQUIC

TCPQUIC

SubscribingClients

Publishersstarted

receving

Publishersrestarted

ConnectingClients

Fig 10 The memory utilization of broker While the memory utilizationlevel of MQTTwQUIC is dropped after the publishers were restarted thememory utilization of MQTTwTCP remains high

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

20

40

60

80

100

Proc

esso

r Util

izat

ion

()

MQTTwTCPMQTTwQUIC

TCPQUIC

PublisherRestarted

ConnectingClients

SubscribingClients

Publisherstarted

Receiving

Fig 11 The processor utilization of broker Although the processorutilization of MQTTwQUIC is more than MQTTwTCP after the publisherswere restarted MQTTwQUIC shows a higher drop in processor usage asQUIC runs over UDP and does not require any connection state information

results shows that MQTTwQUIC releases the resources assoon as it detects the connection is broken Specifically theMQTTwQUICrsquos UDP socket times out under 1 minute andQUIC starts its draining period On the other hand TCPconnections still wait for incoming packets as there is nomechanism to detect whether the connections opened by thepublishers are still active or not In summary it is observed thatMQTTwQUIC reduces processor utilization between 7467to 8324 and lowers memory utilization between 4552to 5032 compared to MQTTwTCP It must be notedthat MQTTwQUIC achieves these improvements without in-troducing any communication overheads such as keep-alivepackets

15

TABLE IIIPERFORMANCE IMPROVEMENT OF MQTTWQUIC VERSUS MQTTWTCP IN TERMS OF PACKET DELIVERY LATENCY IN THE PRESENCE OF PACKET

DROPS THE DROP RATE IS CHANGED BETWEEN 10 TO 50

Testbed Wired Wireless Long DistanceDrop Rate 10 20 50 10 20 50 10 20 50Latency of

MQTTwTCP 3114 ms 5234 ms 106904 ms 112176 ms 157451 ms 214137 ms 443578 ms 549340 ms 756283 ms

Latency ofMQTTwQUIC 13833 ms 35680 ms 89389 ms 71099 ms 105313 ms 185313 ms 330088 ms 440462 ms 643635 ms

Improvementvs MQTTwTCP 5557 3183 1638 3661 3311 1346 2558 1981 1489

0 2 4 6 8 10 12 14 16Minutes

0

5

10

15

20

25

30

Thro

ughp

ut (K

bps)

MQTTwTCPMQTTwQUIC

Changed IP address

Changed IP address

Changed IP address

Fig 12 The effect of connection migration on throughput After each IPrenewal TCP re-establishes the connections In contrast QUIC connectionsare survived and only a slight throughput degradation can be observed due tothe change in interface IP address

D Connection Migration

As mentioned earlier connection reestablishment due toroaming imposes energy overhead This section evaluates theefficiency of the proposed system during connection migra-tions To this end 100 connections were established from 10publishers to a broker and the subscribers were subscribedto all the topics The nload utility was used to measurethroughput In order to emulate the change of network the IPaddress of the brokerrsquos interface connected to the subscriber ischanged every 5 minutes The total duration of the experimentis 16 minutes to include three connection reestablishmenttriggers and allow the publishers and broker to stabilize aftereach migration

Figure 12 shows the results Whenever a change in theconnection parameters occurs MQTTwTCP has to repeat thecumbersome process of connection re-establishment In thiscase its bandwidth drops to zero and picks up again afterthe connection re-establishment process completes In contrastMQTTwQUICrsquos throughput shows a slight reduction due tothe change in the interface state but connections were mi-grated to the new IP unscathed Specifically QUIC allows theend point devices to survive in such events requiring only tohave a stable IP address during the handshake process to obtainthe 1-RTT keys This feature is particularly useful in mobileIoT scenarios such as medical and industrial applications [8]Another useful case is to make the clients resilient againstNAT rebinding

V RELATED WORK

This section is composed of two sub-sections The first sub-section reviews the IoT application layer protocols and their

employed transport layer protocols The second sub-sectionreviews the literature relevant to QUIC and compares the fea-tures offered by the proposed MQTTwQUIC implementationagainst existing IoT application layer protocols

A IoT Application Layer Protocols

This section overviews the main IoT application layerprotocols and their adoption of transport layer protocols

1) MQTT (Message Queuing Telemetry Transport) MQTTis a lightweight IoT application protocol for machine-to-machine connectivity The protocol specification [27] man-dates the use of a connection-oriented transport protocol toensure packet ordering and reliable delivery Therefore mostimplementations [35] [57]ndash[60] use TCPIP with TLSSSL tosatify this requirement In some cases [61] the Stream ControlTransmission Protocol (SCTP) an alternative to TCP has beenused as well However the key problems with SCTP whenused in IoT domains are (i) 4-way handshake which increaseslatency compared to the TCPrsquos 3-way handshake (ii) SCTPis mainly used for multihoming and redundancy Whereas inIoT redundancy is not an essential aspect compared to latency(iii) SCTP is not suitable for heterogeneous networks [62] Theauthors in [63] studied TCP UDP and SCTP and justified whySCTP is not the best fit as an IoT protocol

2) XMPP (Extensible Messaging and Presence Protocol)XMPP offers both publishsubscribe (asynchronous) and re-questresponse (synchronous) messaging capabilities Thisprotocol uses TLSSSL for security and relies on TCP to en-sure reliability XMPP is most suited for real-time applicationswith small message footprints and low latency in messageexchange [64] Although XMPP is preferred over CoAP indata-centric IoT scenarios due to offering the publishsubscribemechanism the lack of QoS provisioning might not beacceptable in mission-ciritical applications Furthermore itsunderlying transport protocol TCP inherits the shortcomingswe discussed earlier Google has reported various XMPPincompatibilities with their new applications [65] and declaredit as an obsolete protocol

3) REST (Representational State Transfer) REST is anarchitectural paradigm REST uses the HTTP [66] methods(GET POST PUT and DELETE) to provide a resource-oriented messaging system All actions are performed bysimple requestresponse (synchronous) HTTP commands Ithas a built-in accept HTTP header to determine the format ofthe data which is usually JSON (JavaScript Object Notation)or XML (Extensible Markup Language) REST is easy to

16

implement and is supported by most M2M cloud platformsOther features supported by REST are caching authenticationand content type negotiation [67] This protocol howeverdoes not offer high energy efficiency since polling must beperformed in order to check the availability of data

REST provides a one-way connection between the client andserver Client only connects to the server when it has to eitherpush or pull the data On the other hand MQTT relies on atwo-way established connection between the server and clientThis enables the server to respond to clientrsquos request instantly[68] In contrast since REST is a requestreply protocol itdoes not need to establish a long-term connection However inlossy networks in particular establishing connections repeat-edly increases energy consumption [69] BevyWise Networksconducted experiments and estimated that their proprietaryMQTTRoute broker [70] consumes 20 less power than RESTTheir findings concluded that the primary reason behind morepower consumption in REST was due to the resources usedfor connecting re-connecting and cleaning up both the serverand client connection states As another example [68] showsthat MQTT is up to 25 times faster than REST in terms ofdata transfer rate

The availability of a REST server might be limited ifthe devices are behind a firewall A common scenario offirewall deployment is when all new incoming connections areblocked while outgoing connections are always allowed dueto zoning In this case establishing connection to IoT deviceswould not be possible MQTT solves this issue by relying ontwo-way connections

4) AMQP (Advanced Message Queuing Protocol) Al-though AMQP is agnostic to transport protocol it uses TCP(and TLS) by default AMPQ offers a publishsubscribe modelfor messaging This protocol provides a reliable connection byutilizing a store and forward mechanism [71] [72] therebyoffering reliability in the presence of network disruptions Theauthors in [73] show that AMQP can send a larger amountof messages per second compared to REST Studies showthat AMQP has a low success rate in low bandwidth but thesuccess rate increases as bandwidth increases [72] [74]

5) MQTT-SN (Message Queuing Telemetry Transport - Sen-sor Networks) MQTT-SN is a variant of MQTT whichis mainly designed for very resource-constrained devicesSpecifically this protocol aims to reduce the high energyconsumption and bandwidth usage of MQTT networks [75]Although MQTT-SN can use UDP as its underlying transportprotocol it is agnostic to the underlying transport servicesApart from introducing UDP MQTT-SN also mandates theinclusion of a gateway (GW) and forwarder The primaryfunction of the gateway is to transform UDP packets intostandard MQTT packets and transmit the packets to a brokerThere are two categories for this connection transparent andaggregating Using a transparent gateway every MQTT-SNnode owns a dedicated TCP connection to the broker Onthe other hand an aggregating gateway only has one TCPconnection to the broker which is then used in tunnelingThe forwarder receives MQTT-SN frames on the wireless sideand encapsulates and forwards them to the gateway For thereturn traffic frames are decapsulated and sent to the clients

without applying any modifications MQTT-SN avails DTLSand TLS to provide security for UDP and TCP respectivelyUsing a gateway is mandatory in MQTT-SN The authors in[76] proved MQTT-SN performs 30 faster than CoAP

6) CoAP (Constrained Application Protocol) CoAP elim-inates the overhead of TCP by relying on UDP [77] [78] andDTLS is used to secure the connections CoAP implementsreliability by defining two bits in each packet header Thesebits determine the type of message and the required Quality ofService (QoS) level The authors in [79] show that the delayof CoAP is higher than MQTT which is caused by packetlosses when UDP is replaced with TCP

B QUIC

Various studies have evaluated the performance of QUIC inthe context of Internet traffic Google has deployed this proto-col in its front-end servers that collectively handle billions ofrequests per day [14] Google claims that QUIC outperformsTCP in a variety of scenarios such as reducing latency by 8in Google search for desktop users and 36 for mobile usersIn video streaming latency in YouTube playbacks is reducedby 18 for desktop and 153 for mobile users CurrentlyQUIC is used for 30 of Googlersquos egress traffic

In [80] the authors evaluate the performance of multipleQUIC streams in LTE and WiFi networks Their experimentsshow that for mobile web page median and 95th percentilecompletion time can be improved by up to 591 and 723respectively compared to HTTP The authors in [48] show thatQUIC outperforms TCP with respect to Page Load Time (PLT)in different desktop user network conditions such as addeddelay and loss variable bandwidth and mobile environmentThe studies of [81] show that QUIC outperforms TLS whenused in unstable networks such as WiFi [82] Specifically theauthors performed experiments to measure PLT for YouTubetraffic with different delays (0ms 50ms 100ms and 200ms)These values were adopted from [83] for QUIC to simulatereal networks Assuming the PLT is x when the introduceddelay is 0 these observations have been made For the secondconnection the PLT is doubled and it is increased by 400msfor each consecutive connection However for HTTP-2 repeatconnections the PLT is multiplied by the connection numberfor each consecutive connection ie 2x 3x etc In anotherstudy [84] the authors showed that in more than 40 ofscenarios the PLT of QUIC is lower than SPDY [85] and TLScombined

The authors in [86] have evaluated the performance ofHTTP2 with QUIC for Multi-User Virtual World (MUVW)and 3D web MUVW networks mostly run on UDP [87] [88]and require the network administrator to open 50 or more UDPports on the firewall Since these ports carry UDP traffic anapplication protocol capable of offering connection-orientatedstreams and security is required QUIC is an ideal candidateto fill this void

The implementation and integration of QUIC with MQTTare complementary to the aforementioned studies since noneof them have considered the applicability of QUIC in IoTapplications Specifically this work reveals the importance

17

TABLE IVCOMPARISON OF IOT APPLICATION LAYER PROTOCOLS

Protocol CoAP MQTT MQTT-SN XMPP REST AMQP MQTTwQUICUDP Compatible

radicX

radicX X

radic radic

TCP Compatible Xradic radic radic radic radic

XMultiplexing Capability X X X X X X

radic

0-RTT Capable X X X X X Xradic

Fixing Head-of-Line Blocking NA X X X X Xradic

Fixing TCP Half-Open Problem(Adaptability to Lossy Networks) NA X X X X X

radic

Supporting Connection Migration X X X X X Xradic

and benefits of integrating MQTT with QUIC when used invarious types of IoT networks including local wireless andlong-distance networks The results presented in this paper inparticular confirm 56 lower packet exchange overhead 83lower processor utilization 50 lower memory utilization and55 shorter delivery delay of MQTTwQUIC compared toMQTTwTCP Table IV compares MQTTwQUIC versus theexisting application layer protocols

VI CONCLUSION

Enhancing transport layer protocols is essential to ensurecompatibility while addressing the particular demands of IoTnetworks Specifically shifting the implementation of pro-tocols to the user space brings substantial benefits such asoffloading the cost of modifying the kernel and enhancingprocessing speed In addition given the resource-constrainednature of IoT devices reducing communication overhead isessential Unfortunately these requirements are not satisfiedby TCPTLS and UDPDTLS

In this paper we justified the potential benefits of QUICcompared to TCPTLS and UDPDTLS in IoT scenarios andpresented its integration with MQTT protocol Specificallywe showed the software architecture proposed as well asthe agents developed to enable the communication betweenMQTT and QUIC Three different testbeds were used to eval-uate the performance of MQTTwQUIC versus MQTTwTCPThe results confirmed that MQTTwQUIC reduces connectionestablishment overhead lowers delivery latency reduces pro-cessor and memory utilization and shortens the level and du-ration of throughput degradation during connection migrationsignificantly compared to MQTTwTCP

Some of the potential areas of future work are as followsFirst the processor utilization of QUIC is higher than TCPThis is due to the cost of encryption and packet processingwhile maintaining QUICrsquos internal state To reduce the over-head of encryption for example the ChaCha20 optimizationtechnique could be used [89] Although packet processingcost was minimized by using asynchronous packet receptionin the kernel by applying a memory-mapped ring bufferaka PACKET_RX_RING the cost is still higher than TCP Tominimize this problem one solution is to employ kernel bypass[90]ndash[92] to bring packet processing into the user spaceSecond the core functionality of QUIC assumes a maximumMTU of 1392 bytes for handshake packets [93] This includes14 bytes for Ethernet header 20 bytes for IP header 8 bytes forUDP and 1350 bytes for QUIC At present the 1392 bytes is a

static value in the client side which is based on observationaltesting All handshake packets are required to be padded tothe full size in both directions This limitation prohibits IPfragmentation and as a result limits the path MTU discoveryThird as reviewed in this paper QUIC offers many featuresthat can be employed to further enhance the peroformance ofIoT networks For example since most IoT applications donot rely on high throughput we did not evaluate the effect ofQUICrsquos flow control However the study of this mechanism isleft as a future work which can be justified by IoT applicationssuch as motion detection and image classification

REFERENCES

[1] Cisco (2016) Cisco Visual Networking Index GlobalMobile Data Traffic Forecast Update [Online]Available httpswwwciscocomcdammen_ininnovationenterpriseassetsmobile-white-paper-c11-520862pdf

[2] R Silva and J M Silva (2009) An Adaptation Model forMobile IPv6 Support in lowPANs [Online] Available httpstoolsietforgiddraft-silva-6lowpan-mipv6-00html

[3] A Dunkels ldquoFull TCPIP for 8-bit Architecturesrdquo in Proceedings ofthe 1st international conference on Mobile systems applications andservices ACM 2003 pp 85ndash98

[4] A Dunkels ldquoDesign and Implementation of the lwIP TCPIP StackrdquoSwedish Institute of Computer Science vol 2 p 77 2001

[5] T Henderson S Floyd A Gurtov and Y Nishida (2012) RFC3782 The New Reno modification to TCPrsquos Fast Recovery Algorithm[Online] Available httpstoolsietforghtmlrfc3782

[6] L Xu K Harfoush and I Rhee ldquoBinary Increase Congestion Con-trol (BIC) for Fast Long-Distance Networksrdquo in Twenty-third AnnualJoint Conference of the IEEE Computer and Communications Societies(INFOCOM) vol 4 2004 pp 2514ndash2524

[7] A Betzler C Gomez I Demirkol and J Paradells ldquoCoAP congestioncontrol for the internet of thingsrdquo IEEE Communications Magazinevol 54 no 7 pp 154ndash160 2016

[8] B Dezfouli M Radi and O Chipara ldquoREWIMO A Real-Time andReliable Low-Power Wireless Mobile Networkrdquo ACM Transactions onSensor Networks (TOSN) vol 13 no 3 p 17 2017

[9] W Shang Y Yu R Droms and L Zhang (2016)Challenges in IoT networking via TCPIP Architecture[Online] Available httpsnamed-datanetwp-contentuploads201602ndn-0038-1-challenges-iotpdf

[10] B Dezfouli I Amirtharaj and C-C Li ldquoEMPIOT An Energy Mea-surement Platform for Wireless IoT Devicesrdquo Journal of Network andComputer Applications vol 121 pp 135ndash148 2018

[11] A Freier P Karlton and P Kocher (2011) RFC 6101 TheSecure Sockets Layer (SSL) Protocol Version 30 [Online] Availablehttptoolsietforghtmlrfc6101

[12] L Atzori A Iera and G Morabito ldquoThe internet of things A surveyrdquoJournal of Computer networks (Elsevier) vol 54 no 15 pp 2787ndash2805 2010

[13] S Radhakrishnan Y Cheng J Chu A Jain and B Raghavan ldquoTCPFast Openrdquo in Proceedings of the Seventh Conference on EmergingNetworking Experiments and Technologies 2011 p 21

[14] A Langley A Riddoch A Wilk A Vicente C Krasic D ZhangF Yang F Kouranov I Swett J Iyengar et al ldquoThe QUIC TransportProtocol Design and Internet-Scale Deploymentrdquo in ACM SIGCOMMComputer Communication Review 2017 pp 183ndash196

18

[15] J Soldatos M Serrano and M Hauswirth ldquoConvergence of UtilityComputing with the Internet-of-Thingsrdquo in Sixth International Con-ference on Innovative Mobile and Internet Services in UbiquitousComputing (IMIS) 2012 pp 874ndash879

[16] S Distefano G Merlino and A Puliafito ldquoSensing and Actuation as aService A New Development for Cloudsrdquo in 11th IEEE InternationalSymposium o Network Computing and Applications (NCA) 2012 pp272ndash275

[17] S R Hussain S Mehnaz S Nirjon and E Bertino ldquoSeamblue Seam-less Bluetooth Low Energy Connection Migration for Unmodified IoTDevicesrdquo in Proceedings of the International Conference on EmbeddedWireless Systems and Networks (EWSN) 2017 pp 927ndash944

[18] B Dezfouli V Esmaeelzadeh J Sheth and M Radi ldquoA review ofsoftware-defined wlans Architectures and central control mechanismsrdquoIEEE Communications Surveys amp Tutorials 2018

[19] B Dezfouli M Radi S A Razak T Hwee-Pink and K A BakarldquoModeling low-power wireless communicationsrdquo Journal of Networkand Computer Applications vol 51 pp 102ndash126 2015

[20] A Nikoukar S Raza A Poole M Guumlnes and B Dezfouli ldquoLow-power wireless for the internet of things Standards and applicationsrdquoIEEE Access vol 6 pp 67 893ndash67 926 2018

[21] S Saini and A Fehnker ldquoEvaluating the stream control transmissionprotocol using uppaalrdquo arXiv preprint arXiv170306568 2017

[22] S V Chimkode ldquodesign of an fpga based embedded system forprotecting the server from syn flood attackrdquo PhD dissertation GoaUniversity 2017

[23] Z Ahmed M Mahbub and S J Soheli ldquoDefense Against SYNFlood Attack Using LPTR-PSO A Three Phased Scheduling ApproachrdquoInternational Journal of Advanced Computer Science and Applicationsvol 8 no 9 pp 433ndash441 2017

[24] D S S R A R Nageswara K LakshmiNadh ldquoAnalysis of TCPIssues in Internet of Thingsrdquo International Journal of Pure and AppliedMathematics vol 118 no 14 2018

[25] W Bziuk C V Phung J Dizdarevic and A Jukan ldquoOn http per-formance in iot applications An analysis of latency and throughputrdquoin 41st International Convention on Information and CommunicationTechnology Electronics and Microelectronics (MIPRO) IEEE 2018pp 0350ndash0355

[26] M Scharf and S Kiesel ldquoHead-of-line Blocking in TCP and SCTPAnalysis and Measurementsrdquo in GLOBECOM vol 6 2006 pp 1ndash5

[27] A Banks and R Gupta (2014) Message Queuing Telemetry Transport(MQTT) v311 [Online] Available httpdocsoasis-openorgmqttmqttv311osmqtt-v311-ospdf

[28] Z Shelby K Hartke and C Bormann (2014) RFC 7252The Constrained Application Protocol (CoAP) [Online] Availablehttpwwwrfc-editororginforfc7252

[29] T Dierks and E Rescorla (2008) RFC 5246 The Transport LayerSecurity (TLS) Protocol [Online] Available httpstoolsietforghtmlrfc5246

[30] E Rescorla and N Modadugu (2012) RFC 6347 DatagramTransport Layer Security Version 12 [Online] Available httpstoolsietforghtmlrfc6347

[31] H T Eric Rescorla (2017) RFC 6347 The Datagram TransportLayer Security (DTLS) Connection Identifier [Online] Availablehttpstoolsietforghtmldraft-ietf-tls-dtls-connection-id-00page-3

[32] H T Eric Rescorla (2017) RFC 6347 Datagram Transport LayerSecurity Version 12 [Online] Available httpstoolsietforghtmlrfc6347

[33] Google (2016) QUIC Protocol [Online] Available httpswwwchromiumorgquic

[34] R Stewart (2007) RFC 4960 Stream Control Transport Protocol[Online] Available httpstoolsietforghtmlrfc4960

[35] Eclipse (2018) Paho Eclipse MQTT Project [Online] Availablehttpprojectseclipseorgprojectstechnologymosquitto

[36] Google (2018) The Chromium Project [Online] Available httpwwwchromiumorgdevelopersdesign-documentsdns-prefetching

[37] Google (2018) QUIC A UDP-based Multiplexed andSecure Transport [Online] Available httpsdatatrackerietforgdocdraft-ietf-quic-transport

[38] J Iyengar and M Thomson (2017) Loss Detection andCongestion Control [Online] Available httpswwwtoolsietforghtmldraft-ietf-quic-transport-11page-41

[39] J Iyengar and M Thomson (2017) Probing a New Path [Online] Avail-able httpstoolsietforghtmldraft-ietf-quic-transport-11page-38

[40] H Krawczyk K G Paterson and H Wee ldquoOn the Security of theTLS Protocol A Systematic Analysisrdquo in Advances in Cryptology(CRYPTO) Springer 2013 pp 429ndash448

[41] M Fischlin and F Guumlnther ldquoMulti-Stage Key Exchange and the Caseof Googlersquos QUIC Protocolrdquo in Proceedings of the ACM SIGSACConference on Computer and Communications Security 2014 pp1193ndash1204

[42] A Langley and W-T Chang (2016) QUIC Crypto[Online] Available httpsdocsgooglecomdocumentd1g5nIXAIkN_Y-7XJW5K45IblHd_L2f5LTaDUDwvZ5L6gedit

[43] S Gruumlner J Pfrommer and F Palm ldquoRESTful Industrial Communi-cation with OPC UArdquo IEEE Transactions on Industrial Informaticsvol 12 no 5 pp 1832ndash1841 2016

[44] I Swettn (2016) QUIC General [On-line] Available httpsdocsgooglecomdocumentd1gY9-YNDNAB1eip-RTPbqphgySwSNSDHLq9D5Bty4FSUedit

[45] J Iyengar and I Swett (2017) QUIC Loss Detection andCongestion Control [Online] Available httpstoolsietforghtmldraft-ietf-quic-recovery-08

[46] A Aggarwal S Savage and T Anderson ldquoUnderstanding the Perfor-mance of TCP Pacingrdquo in Proceedings of 9th Annual Joint Conference ofthe IEEE Computer and Communications Societies (INFOCOM) vol 32000 pp 1157ndash1165

[47] I Rhee L Xu S Ha A Zimmermann L Eggert and R Scheffenegger(2018) RFC 8312 CUBIC for Fast Long-Distance Networks [Online]Available httpstractoolsietforghtmlrfc8312

[48] A M Kakhki S Jero D Choffnes C Nita-Rotaru and A MisloveldquoArchitectural Considerations for a New Generation of Protocolsrdquo inProceedings of the Internet Measurement Conference ACM 2017 pp209ndash303

[49] ngtcp2 team (2018) ngtcp2 Project [Online] Available httpsgithubcomngtcp2ngtcp2

[50] Y Cui T Li C Liu X Wang and M Kuumlhlewind ldquoInnovatingTransport with QUIC Design Approaches and Research ChallengesrdquoIEEE Internet Computing vol 21 no 2 pp 72ndash76 2017

[51] H Krawczyk M Bellare and R Canetti (1997) RFC2104 HMACKeyed-Hashing for Message Authentication [Online] Availablehttpstoolsietforghtmlrfc2104

[52] P Rogaway ldquoAuthenticated-Encryption With Associated-Datardquo in Pro-ceedings of the 9th ACM conference on Computer and communicationssecurity (ACM) 2002 pp 98ndash107

[53] P E H T Joseph Salowey Hao Zhou (1999) RFC 5077 TransportLayer Security (TLS) Session Resumption Without Server-Side Statel[Online] Available httpstoolsietforghtmlrfc5077

[54] S Krishnan and F Monrose ldquoDNS Prefetching and its Privacy Im-plications When Good Things Go Badrdquo in Proceedings of the 3rdUSENIX conference on Large-scale exploits and emergent threatsbotnets spyware worms and more (USENIX Association) 2010 pp10ndash10

[55] M K McKusick G V Neville-Neil and R N Watson The Design andImplementation of the FreeBSD Operating System Pearson Education2014

[56] G Henderson (2018) Wiring Pi GPIO Interface Library for theRaspberry Pi [Online] Available httpwiringpicom

[57] S Obermaier Dominik (2014) QUIC Loss Detection andCongestion Control [Online] Available httpsgithubcomhivemqhivemq-mqtt-web-client

[58] A Y Feng Lee (2018) Cocoa MQTT [Online] Available httpsgithubcomemqttCocoaMQTT

[59] Ralight (2015) IOSphere Mosquitto [Online] Available httpsgithubcomiospheremosquitto

[60] L Petersen and S Clausen (2018) HaskellMQTT [Online] Availablehttpsgithubcomlpetersehaskell-mqtt

[61] N Tonpa (2017) HaskellMQTT [Online] Available httpsgithubcomsynrcmqttblobmasterindexhtml

[62] S Fu and M Atiquzzaman ldquoSCTP State of the Art in ResearchProducts and Technical Challengesrdquo IEEE Communications Magazinevol 42 no 4 pp 64ndash76 2004

[63] R Stewart and P Amer ldquoWhy is SCTP needed given TCP and UDPare widely availablerdquo Internet Society Member Briefing vol 17 2004

[64] S Bendel T Springer D Schuster A Schill R Ackermann andM Ameling ldquoA Service Infrastructure for the Internet of Things Basedon XMPPrdquo in IEEE International Conference on Pervasive Computingand Communications Workshops (PERCOM Workshops) 2013 pp 385ndash388

[65] S Vaughan-Nicholsn (2013) Google Gives upXMPP [Online] Available httpswwwzdnetcomarticlegoogle-moves-away-from-the-xmpp-open-messaging-standard

19

[66] R T Fielding and J Reschke (2014) RFC 7231 Hypertext TransferProtocolndashHTTP11 Semantics and Content [Online] Availablehttpstoolsietforghtmlrfc7231

[67] B Upadhyaya Y Zou H Xiao J Ng and A Lau ldquoMigration ofSOAP-based Services to RESTful Servicesrdquo in 13th IEEE InternationalSymposium on Web Systems Evolution (WSE) 2011 pp 105ndash114

[68] R K Diraviyam (2018) QUIC Loss Detection andCongestion Control [Online] Available httpswwwbevywisecomblogmqtt-vs-rest-iot-implementation

[69] T Savolainen N Javed and B Silverajan ldquoMeasuring Energy Con-sumption for RESTful Interactions in 3GPP IoT Nodesrdquo in 7th IFIPWireless and Mobile Networking Conference (WMNC) 2014 pp 1ndash8

[70] B Networks ldquoMQTTRoute Brokerrdquo httpswwwbevywisecommqtt-broker 2018

[71] P Bhimani and G Panchal ldquoMessage Delivery Guarantee and StatusUpdate of Clients Based on IOT-AMQPrdquo in Intelligent Communicationand Computational Technologies (Springer) 2018 pp 15ndash22

[72] F T Johnsen T H Bloebaum M Avlesen S Spjelkavik and B VikldquoEvaluation of Transport Protocols for Web Servicesrdquo in IEEE MilitaryCommunications and Information Systems Conference (MCC) 2013 pp1ndash6

[73] J L Fernandes I C Lopes J J Rodrigues and S Ullah ldquoPerformanceEvaluation of RESTful Web Services and AMQP Protocolrdquo in FifthIEEE International Conference on Ubiquitous and Future Networks(ICUFN) 2013 pp 810ndash815

[74] J E Luzuriaga M Perez P Boronat J C Cano C Calafate andP Manzoni ldquoA Comparative Evaluation of AMQP and MQTT ProtocolsOver Unstable and Mobile Networksrdquo in 12th Annual IEEE ConsumerCommunications and Networking Conference (CCNC) 2015 pp 931ndash936

[75] A Stanford-Clark and H L Truong ldquoMQTT for Sensor Networks(MQTT-SN) Protocol Specificationrdquo International business machines(IBM) Corporation version vol 1 2013

[76] M H Amaran N A M Noh M S Rohmad and H HashimldquoA Comparison of Lightweight Communication Protocols in RoboticApplicationsrdquo Procedia Computer Science vol 76 pp 400ndash405 2015

[77] J Iyengar and I Swett (2017) Profiling of DTLS for CoAP-Based IoT Applications [Online] Available httpstoolsietforghtmldraft-keoh-dtls-profile-iot-00

[78] S L Keoh S S Kumar and H Tschofenig ldquoSecuring the Internet ofThings A standardization perspectiverdquo IEEE Internet of Things Journalvol 1 no 3 pp 265ndash275 2014

[79] D Thangavel X Ma A Valera H-X Tan and C K-Y Tan ldquoPer-formance Evaluation of MQTT and CoAP via a Common Middlewarerdquoin IEEE Ninth International Conference on Intelligent Sensors SensorNetworks and Information Processing (ISSNIP) 2014 pp 1ndash6

[80] P Qian N Wang and R Tafazolli ldquoAchieving Robust Mobile WebContent Delivery Performance Based on Multiple Coordinated QUICConnectionsrdquo IEEE Access vol 6 pp 11 313ndash11 328 2018

[81] S Cook B Mathieu P Truong and I Hamchaoui ldquoQUIC Better forwhat and for whomrdquo in IEEE International Conference on Communi-cations (ICC) 2017 pp 1ndash6

[82] A Sathiaseelan and J Crowcroft ldquoInternet on the Move Challenges andSolutionsrdquo ACM SIGCOMM Computer Communication Review vol 43no 1 pp 51ndash55 2013

[83] Igvita (2012) QUIC Experiments for Aka-mai [Online] Available httpswwwigvitacom20120719latency-the-new-web-performance-bottleneck

[84] P Megyesi Z Kraumlmer and S Molnaacuter ldquoHow Quick is QUICrdquo in 2016IEEE International Conference on Communications (ICC) 2016 pp1ndash6

[85] M Belshe and R Peon (2012) SPDY Protocoll [Online] Availablehttpstoolsietforghtmldraft-mbelshe-httpbis-spdy-00

[86] H Bakri C Allison A Miller and I Oliver ldquoHTTP2 and QUIC forVirtual Worlds and the 3D Webrdquo Procedia Computer Science (Elsevier)vol 56 pp 242ndash251 2015

[87] I Oliver C Allison and A Miller ldquoTraffic Management for Multi UserVirtual Environmentsrdquo in The 10th Annual PostGraduate Symposium onThe Convergence of Telecommunications Networking and Broadcasting(PGNET 2009) 2009

[88] I A Oliver A H Miller and C Allison ldquoVirtual Worlds Real TrafficInteraction and Adaptationrdquo in Proceedings of the first annual ACMSIGMM conference on Multimedia systems (ACM) 2010 pp 305ndash316

[89] A Langley W Chang N Mavrogiannopoulos J Strombergson andS Josefsson ldquoChacha20-poly1305 cipher suites for transport layersecurity (tls)rdquo Tech Rep 2016

[90] L Rizzo ldquoNetmap A Novel Framework for Fast Packet IOrdquo in 21stUSENIX Security Symposium (USENIX Security 12) 2012 pp 101ndash112

[91] M Paolino N Nikolaev J Fanguede and D Raho ldquoSnabbSwitchUser Space Virtual Switch Benchmark and Performance Optimizationfor NFVrdquo in IEEE Conference on Network Function Virtualization andSoftware Defined Network (NFV-SDN) 2015 pp 86ndash92

[92] Intel (2014) Data Plane Development Kit [Online] Availablehttpswwwdpdkorg

[93] I Swett (2015) QUIC Next generation multiplexed transport over UDP[Online] Available httpswwwnanogorgsitesdefaultfilesmeetingsNANOG64105120150603_Rogan_Quic_Next_Generation_v1pdf

I Introduction

I-A TCP and UDP

I-B TLS and DTLS

I-C Contributions

II QUIC

II-A Connection Establishment

II-B Connection Migration

II-C Security

II-D Multiplexing

II-E Flow and Congestion Control

III Integration and Implementation of MQTT with QUIC

III-A Architecture

III-B Definitions Methods and Assumptions

III-C Server-agent

III-C1 main()

III-C2 process_rx_packets()

III-C3 quic_input_message()

III-D Client-agent

III-D1 main()

III-D2 mqtt_message_initializer()

III-D3 mqtt_client_connect()

III-D4 initialize_quic()

III-D5 start_connect()

III-D6 quic_stream_gen()

III-E Common APIs and functions

III-E1 do_handshake()

III-E2 do_handshake_once()

III-E3 crypt_quic_message()

III-E4 protocol_connect()

III-E5 quic_dispatcher()

III-F Code Reduction

IV Evaluation

IV-A Overhead of Connection Establishment

IV-B Head-of-Line Blocking

IV-C TCP Half-Open Connections

IV-D Connection Migration

V Related Work

V-A IoT Application Layer Protocols

V-A1 MQTT (Message Queuing Telemetry Transport)

V-A2 XMPP (Extensible Messaging and Presence Protocol)

V-A3 REST (Representational State Transfer)

V-A4 AMQP (Advanced Message Queuing Protocol)

V-A5 MQTT-SN (Message Queuing Telemetry Transport - Sensor Networks)

V-A6 CoAP (Constrained Application Protocol)

V-B QUIC

VI Conclusion

References

2

end up out of state This undefined state is referred to asTCP half-open connections [21] A half-open connectionconsumes resources such as memory and processor time Inaddition it can impose serious threats such as SYN flooding[22] [23]

ndash If packets are dropped infrequently during a data flowthe receiver has to wait for dropped packets to be re-transmitted in order to complete the packet re-ordering Thisphenomena which impedes packet delivery performance iscalled the head-of-line blocking [24]ndash[26]Despite the aforementioned shortcomings several IoT ap-

plication layer protocols rely on TCP and some of themoffer mechanisms to remedy these shortcomings For exampleMQTT [27] employs application layer keep-alive messagesto keep the connection alive This mechanism also enablesMQTT to detect connection breakdown and release the re-sources

Another transport layer protocol used in IoT networksis UDP Generally UDP is suitable for applications whereconnection reliability is not essential Although this is notacceptable in many IoT scenarios several IoT application layerprotocols rely on UDP due to its lower overhead compared toTCP These protocols usually include mechanisms to supportreliability and block transmission (eg CoAP [28])

B TLS and DTLS

In addition to reliability it is essential for IoT applicationsto employ cryptographic protocols to secure end-to-end dataexchange over transport layer TLS [29] is the most commonconnection-oriented and stateful client-server cryptographicprotocol Symmetric encryption in TLS enables authenticatedconfidential and integrity-preserved communication betweentwo devices The key for this symmetric encryption is gener-ated during the TLS handshake and is unique per connectionIn order to establish a connection the TLS handshake canrequire up to two round-trips between the server and clientHowever since connections might be dropped due to phe-nomenons such as sleep phases connection migration andpacket loss the overhead of establishing secure connectionsimposes high overhead In order to address this concern alighter version of TLS for datagrams named DTLS (DatagramTransport Layer Security) [30] has been introduced UnlikeTLS DTLS does not require a reliable transport protocol asit can encrypt or decrypt out-of-order packets Therefore itcan be used with UDP Although the security level offeredby DTLS is equivalent to TLS some of the technical dif-ferences include the adoption of stream ciphers is prohib-ited and an explicit sequence number is included in everyDTLS message Compared to TLS DTLS is more suitablefor resource-constrained devices communicating through anunreliable channel However similar to TLS the underlyingmechanism in DTLS has been primarily designed for point-to-point communication This creates a challenge to secure one-to-many connections such as broadcasting and multicasting Inaddition since DTLS identifies connections based on sourceIP and port number it does not support connection migration[31] Furthermore DTLS handshake packets are large and may

fragment each datagram into several DTLS records where eachrecord is fit into an IP datagram This can potentially causerecord overhead [32]

C Contributions

Given the shortcomings of UDP and TCP we argue that theenhancement of transport layer protocols is a necessary step inthe performance improvement of IoT applications In order toaddress this concern this paper presents the implementationand studies the integration of QUIC [33] with application layerto address these concerns QUIC is a user space UDP-basedstream-based and multiplexed transport protocol developedby Google According to [14] around 7 of the world-wideInternet traffic employs QUIC This protocol offers all thefunctionalities required to be considered a connection-orientedtransport protocol In addition QUIC solves the numerousproblems faced by other connection-oriented protocols suchas TCP and SCTP [34] Specifically the addressed problemsare reducing the connection setup overhead supporting mul-tiplexing removing the head-of-line blocking supporting con-nection migration and eliminating TCP half-open connectionsQUIC executes a cryptographic handshake that reduces theoverhead of connection establishment by employing knownserver credentials learned from past connections In additionQUIC reduces transport layer overhead by multiplexing severalconnections into a single connection pipeline Furthermoresince QUIC uses UDP it does not maintain connection statusinformation in the transport layer This protocol also eradicatesthe head-of-line blocking delays by applying a lightweightdata-structure abstraction called streams

At present there is no open source or licensed version ofMQTT using QUIC Current MQTT implementations (suchas Paho [35]) rely on TCPTLS to offer reliable and securedelivery of packets Given the potentials of QUIC and itssuitability in IoT scenarios in this paper we implement andstudy the integration of MQTT with QUIC First since the datastructures and message mechanisms of MQTT are intertwinedwith the built-in TCP and TLS APIs it was necessary toredesign these data structures The second challenge was toestablish IPC (Inter-Process Communication) between QUICand MQTT MQTTwTCP utilizes the available APIs foruser space and kernel communication However there is noavailable API for QUIC and MQTT to communicate as theyare both user space processes To address these challengeswe have developed new APIs which are referred to as agentsSpecifically we implemented two types of agents server-agent and client-agent where the former handles the broker-specific operations and the latter handles the functionalities ofpublisher and subscriber This paper presents all the functionsdeveloped and explains the connection establishment andmessage exchange functionalities by presenting their respec-tive algorithms The third challenge was to strip QUIC ofmechanisms not necessary for IoT scenarios QUIC is a webtraffic protocol composed of a heavy code footprint (15GB[36]) We have significantly reduced the code footprint (toaround 22MB) by eliminating non-IoT related code segmentssuch as loop network and proxy backend support

3

Client ServerServer

Inchoate CHLO

REJ

Complete CHLO

Encrypted request

SHLO

Encrypted response


Encrypted request

Complete CHLO

SHLO

Encrypted response


Client

1-RTT

DataExchange

DataExchange




II QUIC





Flags (8)


QUICHeader














4

C Security


D Multiplexing





QUIC Server

MQTT Broker


Layer

Transport Layer

QUIC Client


Broker

Publisher

Application Layer

Transport Layer

QUIC Client


Subscriber

Application Layer

Transport Layer

UDP

IP

UDP

IP IP

UDP

Network Layer

Network Layer

Network Layer

MQTT Broker

Server-Agent

QUIC Server

MQTT Publisher

Client-Agent

QUIC Client

MQTT Subscriber

Client-Agent

QUIC Client





A Architecture


5


process_rx_packets





start_connect

quic_stream_gen

do_handshake

do_handshake_once


Resume0-RTT

crypt_quic_message



Common APIs































6

















C Server-agent






7





19 return



30 return

D Client-agent












42 return



56 return



8







29 return


42 return










9








34 return





βs ) rarr m


52 return





8 return


c )rarr mα7



s ) rarr mβ8



22 return







10

Clientmc pkj

Serverms







key_settlement



















11















lsquo





T(0) = ε



















12







Clientmc pkj

Serverms



key_settlement





final_data



F Code Reduction



Subscriber

Internet


Broker



Broker

BrokerLocated in

Washington DC

Subscriber


Publisher


California


IV EVALUATION











13

Subscriber Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


Connect ACK (MQTT)

ACK (TCP)


ACK (TCP)

Client Hello (TLS)

Server Hello (TLS)





Data Exchange


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

Publisher Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


ACK (TCP)

Connect ACK (MQTT)


Publish ACK (MQTT)

Client Hello (TLS)

Server Hello (TLS)





1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

ACK (TCP)


Data Exchange

Publisher Broker


REJ (QUIC)






Connect ACK (MQTT)


Publish ACK (MQTT)


Data Exchange

1

2

3

4

5

6

7

8

9

10

Subscriber Broker





Connect ack (MQTT)





Data Exchange

1

2

3

4

5

6

7

8

9

10

11

12






Connect ack (MQTT)




Publish ACK (MQTT)

Data Exchange

Publisher Broker

1

2

3

4

5

6

7

8

9

10






Connect ACK (MQTT)


Publish ACK (MQTT)

1

2

3

4

5

6

7

8

Data Exchange

(a) MQTTwTCP

(b) MQTTwQUIC 1-RTT

(c) MQTTwTCP 0-RTT



0

5

10

15

20

25

30

35

Num

ber o

f Pac

kets









14











1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

200

400

600

800

Mem

ory

Usa

ge (M

B)

MQTTwTCPMQTTwQUIC

TCPQUIC

SubscribingClients

Publishersstarted

receving

Publishersrestarted

ConnectingClients


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

20

40

60

80

100

Proc

esso

r Util

izat

ion

()

MQTTwTCPMQTTwQUIC

TCPQUIC

PublisherRestarted

ConnectingClients

SubscribingClients

Publisherstarted

Receiving



15







0 2 4 6 8 10 12 14 16Minutes

0

5

10

15

20

25

30

Thro

ughp

ut (K

bps)

MQTTwTCPMQTTwQUIC

Changed IP address

Changed IP address

Changed IP address





V RELATED WORK








16








B QUIC





17



radicX

radicX X

radic radic



radic




radic



VI CONCLUSION





REFERENCES















18




















































19





























I Introduction

I-A TCP and UDP

I-B TLS and DTLS

I-C Contributions

II QUIC



II-C Security

II-D Multiplexing



III-A Architecture


III-C Server-agent

III-C1 main()



III-D Client-agent

III-D1 main()













IV Evaluation





V Related Work








V-B QUIC

VI Conclusion

References

3

Client ServerServer

Inchoate CHLO

REJ

Complete CHLO

Encrypted request

SHLO

Encrypted response


Encrypted request

Complete CHLO

SHLO

Encrypted response


Client

1-RTT

DataExchange

DataExchange




II QUIC





Flags (8)


QUICHeader














4

C Security


D Multiplexing





QUIC Server

MQTT Broker


Layer

Transport Layer

QUIC Client


Broker

Publisher

Application Layer

Transport Layer

QUIC Client


Subscriber

Application Layer

Transport Layer

UDP

IP

UDP

IP IP

UDP

Network Layer

Network Layer

Network Layer

MQTT Broker

Server-Agent

QUIC Server

MQTT Publisher

Client-Agent

QUIC Client

MQTT Subscriber

Client-Agent

QUIC Client





A Architecture


5


process_rx_packets





start_connect

quic_stream_gen

do_handshake

do_handshake_once


Resume0-RTT

crypt_quic_message



Common APIs































6

















C Server-agent






7





19 return



30 return

D Client-agent












42 return



56 return



8







29 return


42 return










9








34 return





βs ) rarr m


52 return





8 return


c )rarr mα7



s ) rarr mβ8



22 return







10

Clientmc pkj

Serverms







key_settlement



















11















lsquo





T(0) = ε



















12







Clientmc pkj

Serverms



key_settlement





final_data



F Code Reduction



Subscriber

Internet


Broker



Broker

BrokerLocated in

Washington DC

Subscriber


Publisher


California


IV EVALUATION











13

Subscriber Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


Connect ACK (MQTT)

ACK (TCP)


ACK (TCP)

Client Hello (TLS)

Server Hello (TLS)





Data Exchange


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

Publisher Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


ACK (TCP)

Connect ACK (MQTT)


Publish ACK (MQTT)

Client Hello (TLS)

Server Hello (TLS)





1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

ACK (TCP)


Data Exchange

Publisher Broker


REJ (QUIC)






Connect ACK (MQTT)


Publish ACK (MQTT)


Data Exchange

1

2

3

4

5

6

7

8

9

10

Subscriber Broker





Connect ack (MQTT)





Data Exchange

1

2

3

4

5

6

7

8

9

10

11

12






Connect ack (MQTT)




Publish ACK (MQTT)

Data Exchange

Publisher Broker

1

2

3

4

5

6

7

8

9

10






Connect ACK (MQTT)


Publish ACK (MQTT)

1

2

3

4

5

6

7

8

Data Exchange

(a) MQTTwTCP

(b) MQTTwQUIC 1-RTT

(c) MQTTwTCP 0-RTT



0

5

10

15

20

25

30

35

Num

ber o

f Pac

kets









14











1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

200

400

600

800

Mem

ory

Usa

ge (M

B)

MQTTwTCPMQTTwQUIC

TCPQUIC

SubscribingClients

Publishersstarted

receving

Publishersrestarted

ConnectingClients


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

20

40

60

80

100

Proc

esso

r Util

izat

ion

()

MQTTwTCPMQTTwQUIC

TCPQUIC

PublisherRestarted

ConnectingClients

SubscribingClients

Publisherstarted

Receiving



15







0 2 4 6 8 10 12 14 16Minutes

0

5

10

15

20

25

30

Thro

ughp

ut (K

bps)

MQTTwTCPMQTTwQUIC

Changed IP address

Changed IP address

Changed IP address





V RELATED WORK








16








B QUIC





17



radicX

radicX X

radic radic



radic




radic



VI CONCLUSION





REFERENCES















18




















































19





























I Introduction

I-A TCP and UDP

I-B TLS and DTLS

I-C Contributions

II QUIC



II-C Security

II-D Multiplexing



III-A Architecture


III-C Server-agent

III-C1 main()



III-D Client-agent

III-D1 main()













IV Evaluation





V Related Work








V-B QUIC

VI Conclusion

References

4

C Security


D Multiplexing





QUIC Server

MQTT Broker


Layer

Transport Layer

QUIC Client


Broker

Publisher

Application Layer

Transport Layer

QUIC Client


Subscriber

Application Layer

Transport Layer

UDP

IP

UDP

IP IP

UDP

Network Layer

Network Layer

Network Layer

MQTT Broker

Server-Agent

QUIC Server

MQTT Publisher

Client-Agent

QUIC Client

MQTT Subscriber

Client-Agent

QUIC Client





A Architecture


5


process_rx_packets





start_connect

quic_stream_gen

do_handshake

do_handshake_once


Resume0-RTT

crypt_quic_message



Common APIs































6

















C Server-agent






7





19 return



30 return

D Client-agent












42 return



56 return



8







29 return


42 return










9








34 return





βs ) rarr m


52 return





8 return


c )rarr mα7



s ) rarr mβ8



22 return







10

Clientmc pkj

Serverms







key_settlement



















11















lsquo





T(0) = ε



















12







Clientmc pkj

Serverms



key_settlement





final_data



F Code Reduction



Subscriber

Internet


Broker



Broker

BrokerLocated in

Washington DC

Subscriber


Publisher


California


IV EVALUATION











13

Subscriber Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


Connect ACK (MQTT)

ACK (TCP)


ACK (TCP)

Client Hello (TLS)

Server Hello (TLS)





Data Exchange


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

Publisher Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


ACK (TCP)

Connect ACK (MQTT)


Publish ACK (MQTT)

Client Hello (TLS)

Server Hello (TLS)





1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

ACK (TCP)


Data Exchange

Publisher Broker


REJ (QUIC)






Connect ACK (MQTT)


Publish ACK (MQTT)


Data Exchange

1

2

3

4

5

6

7

8

9

10

Subscriber Broker





Connect ack (MQTT)





Data Exchange

1

2

3

4

5

6

7

8

9

10

11

12






Connect ack (MQTT)




Publish ACK (MQTT)

Data Exchange

Publisher Broker

1

2

3

4

5

6

7

8

9

10






Connect ACK (MQTT)


Publish ACK (MQTT)

1

2

3

4

5

6

7

8

Data Exchange

(a) MQTTwTCP

(b) MQTTwQUIC 1-RTT

(c) MQTTwTCP 0-RTT



0

5

10

15

20

25

30

35

Num

ber o

f Pac

kets









14











1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

200

400

600

800

Mem

ory

Usa

ge (M

B)

MQTTwTCPMQTTwQUIC

TCPQUIC

SubscribingClients

Publishersstarted

receving

Publishersrestarted

ConnectingClients


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

20

40

60

80

100

Proc

esso

r Util

izat

ion

()

MQTTwTCPMQTTwQUIC

TCPQUIC

PublisherRestarted

ConnectingClients

SubscribingClients

Publisherstarted

Receiving



15







0 2 4 6 8 10 12 14 16Minutes

0

5

10

15

20

25

30

Thro

ughp

ut (K

bps)

MQTTwTCPMQTTwQUIC

Changed IP address

Changed IP address

Changed IP address





V RELATED WORK








16








B QUIC





17



radicX

radicX X

radic radic



radic




radic



VI CONCLUSION





REFERENCES















18




















































19





























I Introduction

I-A TCP and UDP

I-B TLS and DTLS

I-C Contributions

II QUIC



II-C Security

II-D Multiplexing



III-A Architecture


III-C Server-agent

III-C1 main()



III-D Client-agent

III-D1 main()













IV Evaluation





V Related Work








V-B QUIC

VI Conclusion

References

5


process_rx_packets





start_connect

quic_stream_gen

do_handshake

do_handshake_once


Resume0-RTT

crypt_quic_message



Common APIs































6

















C Server-agent






7





19 return



30 return

D Client-agent












42 return



56 return



8







29 return


42 return










9








34 return





βs ) rarr m


52 return





8 return


c )rarr mα7



s ) rarr mβ8



22 return







10

Clientmc pkj

Serverms







key_settlement



















11















lsquo





T(0) = ε



















12







Clientmc pkj

Serverms



key_settlement





final_data



F Code Reduction



Subscriber

Internet


Broker



Broker

BrokerLocated in

Washington DC

Subscriber


Publisher


California


IV EVALUATION











13

Subscriber Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


Connect ACK (MQTT)

ACK (TCP)


ACK (TCP)

Client Hello (TLS)

Server Hello (TLS)





Data Exchange


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

Publisher Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


ACK (TCP)

Connect ACK (MQTT)


Publish ACK (MQTT)

Client Hello (TLS)

Server Hello (TLS)





1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

ACK (TCP)


Data Exchange

Publisher Broker


REJ (QUIC)






Connect ACK (MQTT)


Publish ACK (MQTT)


Data Exchange

1

2

3

4

5

6

7

8

9

10

Subscriber Broker





Connect ack (MQTT)





Data Exchange

1

2

3

4

5

6

7

8

9

10

11

12






Connect ack (MQTT)




Publish ACK (MQTT)

Data Exchange

Publisher Broker

1

2

3

4

5

6

7

8

9

10






Connect ACK (MQTT)


Publish ACK (MQTT)

1

2

3

4

5

6

7

8

Data Exchange

(a) MQTTwTCP

(b) MQTTwQUIC 1-RTT

(c) MQTTwTCP 0-RTT



0

5

10

15

20

25

30

35

Num

ber o

f Pac

kets









14











1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

200

400

600

800

Mem

ory

Usa

ge (M

B)

MQTTwTCPMQTTwQUIC

TCPQUIC

SubscribingClients

Publishersstarted

receving

Publishersrestarted

ConnectingClients


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

20

40

60

80

100

Proc

esso

r Util

izat

ion

()

MQTTwTCPMQTTwQUIC

TCPQUIC

PublisherRestarted

ConnectingClients

SubscribingClients

Publisherstarted

Receiving



15







0 2 4 6 8 10 12 14 16Minutes

0

5

10

15

20

25

30

Thro

ughp

ut (K

bps)

MQTTwTCPMQTTwQUIC

Changed IP address

Changed IP address

Changed IP address





V RELATED WORK








16








B QUIC





17



radicX

radicX X

radic radic



radic




radic



VI CONCLUSION





REFERENCES















18




















































19





























I Introduction

I-A TCP and UDP

I-B TLS and DTLS

I-C Contributions

II QUIC



II-C Security

II-D Multiplexing



III-A Architecture


III-C Server-agent

III-C1 main()



III-D Client-agent

III-D1 main()













IV Evaluation





V Related Work








V-B QUIC

VI Conclusion

References

6

















C Server-agent






7





19 return



30 return

D Client-agent












42 return



56 return



8







29 return


42 return










9








34 return





βs ) rarr m


52 return





8 return


c )rarr mα7



s ) rarr mβ8



22 return







10

Clientmc pkj

Serverms







key_settlement



















11















lsquo





T(0) = ε



















12







Clientmc pkj

Serverms



key_settlement





final_data



F Code Reduction



Subscriber

Internet


Broker



Broker

BrokerLocated in

Washington DC

Subscriber


Publisher


California


IV EVALUATION











13

Subscriber Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


Connect ACK (MQTT)

ACK (TCP)


ACK (TCP)

Client Hello (TLS)

Server Hello (TLS)





Data Exchange


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

Publisher Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


ACK (TCP)

Connect ACK (MQTT)


Publish ACK (MQTT)

Client Hello (TLS)

Server Hello (TLS)





1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

ACK (TCP)


Data Exchange

Publisher Broker


REJ (QUIC)






Connect ACK (MQTT)


Publish ACK (MQTT)


Data Exchange

1

2

3

4

5

6

7

8

9

10

Subscriber Broker





Connect ack (MQTT)





Data Exchange

1

2

3

4

5

6

7

8

9

10

11

12






Connect ack (MQTT)




Publish ACK (MQTT)

Data Exchange

Publisher Broker

1

2

3

4

5

6

7

8

9

10






Connect ACK (MQTT)


Publish ACK (MQTT)

1

2

3

4

5

6

7

8

Data Exchange

(a) MQTTwTCP

(b) MQTTwQUIC 1-RTT

(c) MQTTwTCP 0-RTT



0

5

10

15

20

25

30

35

Num

ber o

f Pac

kets









14











1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

200

400

600

800

Mem

ory

Usa

ge (M

B)

MQTTwTCPMQTTwQUIC

TCPQUIC

SubscribingClients

Publishersstarted

receving

Publishersrestarted

ConnectingClients


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

20

40

60

80

100

Proc

esso

r Util

izat

ion

()

MQTTwTCPMQTTwQUIC

TCPQUIC

PublisherRestarted

ConnectingClients

SubscribingClients

Publisherstarted

Receiving



15







0 2 4 6 8 10 12 14 16Minutes

0

5

10

15

20

25

30

Thro

ughp

ut (K

bps)

MQTTwTCPMQTTwQUIC

Changed IP address

Changed IP address

Changed IP address





V RELATED WORK








16








B QUIC





17



radicX

radicX X

radic radic



radic




radic



VI CONCLUSION





REFERENCES















18




















































19





























I Introduction

I-A TCP and UDP

I-B TLS and DTLS

I-C Contributions

II QUIC



II-C Security

II-D Multiplexing



III-A Architecture


III-C Server-agent

III-C1 main()



III-D Client-agent

III-D1 main()













IV Evaluation





V Related Work








V-B QUIC

VI Conclusion

References

7





19 return



30 return

D Client-agent












42 return



56 return



8







29 return


42 return










9








34 return





βs ) rarr m


52 return





8 return


c )rarr mα7



s ) rarr mβ8



22 return







10

Clientmc pkj

Serverms







key_settlement



















11















lsquo





T(0) = ε



















12







Clientmc pkj

Serverms



key_settlement





final_data



F Code Reduction



Subscriber

Internet


Broker



Broker

BrokerLocated in

Washington DC

Subscriber


Publisher


California


IV EVALUATION











13

Subscriber Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


Connect ACK (MQTT)

ACK (TCP)


ACK (TCP)

Client Hello (TLS)

Server Hello (TLS)





Data Exchange


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

Publisher Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


ACK (TCP)

Connect ACK (MQTT)


Publish ACK (MQTT)

Client Hello (TLS)

Server Hello (TLS)





1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

ACK (TCP)


Data Exchange

Publisher Broker


REJ (QUIC)






Connect ACK (MQTT)


Publish ACK (MQTT)


Data Exchange

1

2

3

4

5

6

7

8

9

10

Subscriber Broker





Connect ack (MQTT)





Data Exchange

1

2

3

4

5

6

7

8

9

10

11

12






Connect ack (MQTT)




Publish ACK (MQTT)

Data Exchange

Publisher Broker

1

2

3

4

5

6

7

8

9

10






Connect ACK (MQTT)


Publish ACK (MQTT)

1

2

3

4

5

6

7

8

Data Exchange

(a) MQTTwTCP

(b) MQTTwQUIC 1-RTT

(c) MQTTwTCP 0-RTT



0

5

10

15

20

25

30

35

Num

ber o

f Pac

kets









14











1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

200

400

600

800

Mem

ory

Usa

ge (M

B)

MQTTwTCPMQTTwQUIC

TCPQUIC

SubscribingClients

Publishersstarted

receving

Publishersrestarted

ConnectingClients


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

20

40

60

80

100

Proc

esso

r Util

izat

ion

()

MQTTwTCPMQTTwQUIC

TCPQUIC

PublisherRestarted

ConnectingClients

SubscribingClients

Publisherstarted

Receiving



15







0 2 4 6 8 10 12 14 16Minutes

0

5

10

15

20

25

30

Thro

ughp

ut (K

bps)

MQTTwTCPMQTTwQUIC

Changed IP address

Changed IP address

Changed IP address





V RELATED WORK








16








B QUIC





17



radicX

radicX X

radic radic



radic




radic



VI CONCLUSION





REFERENCES















18




















































19





























I Introduction

I-A TCP and UDP

I-B TLS and DTLS

I-C Contributions

II QUIC



II-C Security

II-D Multiplexing



III-A Architecture


III-C Server-agent

III-C1 main()



III-D Client-agent

III-D1 main()













IV Evaluation





V Related Work








V-B QUIC

VI Conclusion

References

8







29 return


42 return










9








34 return





βs ) rarr m


52 return





8 return


c )rarr mα7



s ) rarr mβ8



22 return







10

Clientmc pkj

Serverms







key_settlement



















11















lsquo





T(0) = ε



















12







Clientmc pkj

Serverms



key_settlement





final_data



F Code Reduction



Subscriber

Internet


Broker



Broker

BrokerLocated in

Washington DC

Subscriber


Publisher


California


IV EVALUATION











13

Subscriber Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


Connect ACK (MQTT)

ACK (TCP)


ACK (TCP)

Client Hello (TLS)

Server Hello (TLS)





Data Exchange


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

Publisher Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


ACK (TCP)

Connect ACK (MQTT)


Publish ACK (MQTT)

Client Hello (TLS)

Server Hello (TLS)





1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

ACK (TCP)


Data Exchange

Publisher Broker


REJ (QUIC)






Connect ACK (MQTT)


Publish ACK (MQTT)


Data Exchange

1

2

3

4

5

6

7

8

9

10

Subscriber Broker





Connect ack (MQTT)





Data Exchange

1

2

3

4

5

6

7

8

9

10

11

12






Connect ack (MQTT)




Publish ACK (MQTT)

Data Exchange

Publisher Broker

1

2

3

4

5

6

7

8

9

10






Connect ACK (MQTT)


Publish ACK (MQTT)

1

2

3

4

5

6

7

8

Data Exchange

(a) MQTTwTCP

(b) MQTTwQUIC 1-RTT

(c) MQTTwTCP 0-RTT



0

5

10

15

20

25

30

35

Num

ber o

f Pac

kets









14











1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

200

400

600

800

Mem

ory

Usa

ge (M

B)

MQTTwTCPMQTTwQUIC

TCPQUIC

SubscribingClients

Publishersstarted

receving

Publishersrestarted

ConnectingClients


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

20

40

60

80

100

Proc

esso

r Util

izat

ion

()

MQTTwTCPMQTTwQUIC

TCPQUIC

PublisherRestarted

ConnectingClients

SubscribingClients

Publisherstarted

Receiving



15







0 2 4 6 8 10 12 14 16Minutes

0

5

10

15

20

25

30

Thro

ughp

ut (K

bps)

MQTTwTCPMQTTwQUIC

Changed IP address

Changed IP address

Changed IP address





V RELATED WORK








16








B QUIC





17



radicX

radicX X

radic radic



radic




radic



VI CONCLUSION





REFERENCES















18




















































19





























I Introduction

I-A TCP and UDP

I-B TLS and DTLS

I-C Contributions

II QUIC



II-C Security

II-D Multiplexing



III-A Architecture


III-C Server-agent

III-C1 main()



III-D Client-agent

III-D1 main()













IV Evaluation





V Related Work








V-B QUIC

VI Conclusion

References

9








34 return





βs ) rarr m


52 return





8 return


c )rarr mα7



s ) rarr mβ8



22 return







10

Clientmc pkj

Serverms







key_settlement



















11















lsquo





T(0) = ε



















12







Clientmc pkj

Serverms



key_settlement





final_data



F Code Reduction



Subscriber

Internet


Broker



Broker

BrokerLocated in

Washington DC

Subscriber


Publisher


California


IV EVALUATION











13

Subscriber Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


Connect ACK (MQTT)

ACK (TCP)


ACK (TCP)

Client Hello (TLS)

Server Hello (TLS)





Data Exchange


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

Publisher Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


ACK (TCP)

Connect ACK (MQTT)


Publish ACK (MQTT)

Client Hello (TLS)

Server Hello (TLS)





1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

ACK (TCP)


Data Exchange

Publisher Broker


REJ (QUIC)






Connect ACK (MQTT)


Publish ACK (MQTT)


Data Exchange

1

2

3

4

5

6

7

8

9

10

Subscriber Broker





Connect ack (MQTT)





Data Exchange

1

2

3

4

5

6

7

8

9

10

11

12






Connect ack (MQTT)




Publish ACK (MQTT)

Data Exchange

Publisher Broker

1

2

3

4

5

6

7

8

9

10






Connect ACK (MQTT)


Publish ACK (MQTT)

1

2

3

4

5

6

7

8

Data Exchange

(a) MQTTwTCP

(b) MQTTwQUIC 1-RTT

(c) MQTTwTCP 0-RTT



0

5

10

15

20

25

30

35

Num

ber o

f Pac

kets









14











1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

200

400

600

800

Mem

ory

Usa

ge (M

B)

MQTTwTCPMQTTwQUIC

TCPQUIC

SubscribingClients

Publishersstarted

receving

Publishersrestarted

ConnectingClients


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

20

40

60

80

100

Proc

esso

r Util

izat

ion

()

MQTTwTCPMQTTwQUIC

TCPQUIC

PublisherRestarted

ConnectingClients

SubscribingClients

Publisherstarted

Receiving



15







0 2 4 6 8 10 12 14 16Minutes

0

5

10

15

20

25

30

Thro

ughp

ut (K

bps)

MQTTwTCPMQTTwQUIC

Changed IP address

Changed IP address

Changed IP address





V RELATED WORK








16








B QUIC





17



radicX

radicX X

radic radic



radic




radic



VI CONCLUSION





REFERENCES















18




















































19





























I Introduction

I-A TCP and UDP

I-B TLS and DTLS

I-C Contributions

II QUIC



II-C Security

II-D Multiplexing



III-A Architecture


III-C Server-agent

III-C1 main()



III-D Client-agent

III-D1 main()













IV Evaluation





V Related Work








V-B QUIC

VI Conclusion

References

10

Clientmc pkj

Serverms







key_settlement



















11















lsquo





T(0) = ε



















12







Clientmc pkj

Serverms



key_settlement





final_data



F Code Reduction



Subscriber

Internet


Broker



Broker

BrokerLocated in

Washington DC

Subscriber


Publisher


California


IV EVALUATION











13

Subscriber Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


Connect ACK (MQTT)

ACK (TCP)


ACK (TCP)

Client Hello (TLS)

Server Hello (TLS)





Data Exchange


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

Publisher Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


ACK (TCP)

Connect ACK (MQTT)


Publish ACK (MQTT)

Client Hello (TLS)

Server Hello (TLS)





1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

ACK (TCP)


Data Exchange

Publisher Broker


REJ (QUIC)






Connect ACK (MQTT)


Publish ACK (MQTT)


Data Exchange

1

2

3

4

5

6

7

8

9

10

Subscriber Broker





Connect ack (MQTT)





Data Exchange

1

2

3

4

5

6

7

8

9

10

11

12






Connect ack (MQTT)




Publish ACK (MQTT)

Data Exchange

Publisher Broker

1

2

3

4

5

6

7

8

9

10






Connect ACK (MQTT)


Publish ACK (MQTT)

1

2

3

4

5

6

7

8

Data Exchange

(a) MQTTwTCP

(b) MQTTwQUIC 1-RTT

(c) MQTTwTCP 0-RTT



0

5

10

15

20

25

30

35

Num

ber o

f Pac

kets









14











1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

200

400

600

800

Mem

ory

Usa

ge (M

B)

MQTTwTCPMQTTwQUIC

TCPQUIC

SubscribingClients

Publishersstarted

receving

Publishersrestarted

ConnectingClients


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

20

40

60

80

100

Proc

esso

r Util

izat

ion

()

MQTTwTCPMQTTwQUIC

TCPQUIC

PublisherRestarted

ConnectingClients

SubscribingClients

Publisherstarted

Receiving



15







0 2 4 6 8 10 12 14 16Minutes

0

5

10

15

20

25

30

Thro

ughp

ut (K

bps)

MQTTwTCPMQTTwQUIC

Changed IP address

Changed IP address

Changed IP address





V RELATED WORK








16








B QUIC





17



radicX

radicX X

radic radic



radic




radic



VI CONCLUSION





REFERENCES















18




















































19





























I Introduction

I-A TCP and UDP

I-B TLS and DTLS

I-C Contributions

II QUIC



II-C Security

II-D Multiplexing



III-A Architecture


III-C Server-agent

III-C1 main()



III-D Client-agent

III-D1 main()













IV Evaluation





V Related Work








V-B QUIC

VI Conclusion

References

11















lsquo





T(0) = ε



















12







Clientmc pkj

Serverms



key_settlement





final_data



F Code Reduction



Subscriber

Internet


Broker



Broker

BrokerLocated in

Washington DC

Subscriber


Publisher


California


IV EVALUATION











13

Subscriber Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


Connect ACK (MQTT)

ACK (TCP)


ACK (TCP)

Client Hello (TLS)

Server Hello (TLS)





Data Exchange


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

Publisher Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


ACK (TCP)

Connect ACK (MQTT)


Publish ACK (MQTT)

Client Hello (TLS)

Server Hello (TLS)





1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

ACK (TCP)


Data Exchange

Publisher Broker


REJ (QUIC)






Connect ACK (MQTT)


Publish ACK (MQTT)


Data Exchange

1

2

3

4

5

6

7

8

9

10

Subscriber Broker





Connect ack (MQTT)





Data Exchange

1

2

3

4

5

6

7

8

9

10

11

12






Connect ack (MQTT)




Publish ACK (MQTT)

Data Exchange

Publisher Broker

1

2

3

4

5

6

7

8

9

10






Connect ACK (MQTT)


Publish ACK (MQTT)

1

2

3

4

5

6

7

8

Data Exchange

(a) MQTTwTCP

(b) MQTTwQUIC 1-RTT

(c) MQTTwTCP 0-RTT



0

5

10

15

20

25

30

35

Num

ber o

f Pac

kets









14











1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

200

400

600

800

Mem

ory

Usa

ge (M

B)

MQTTwTCPMQTTwQUIC

TCPQUIC

SubscribingClients

Publishersstarted

receving

Publishersrestarted

ConnectingClients


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

20

40

60

80

100

Proc

esso

r Util

izat

ion

()

MQTTwTCPMQTTwQUIC

TCPQUIC

PublisherRestarted

ConnectingClients

SubscribingClients

Publisherstarted

Receiving



15







0 2 4 6 8 10 12 14 16Minutes

0

5

10

15

20

25

30

Thro

ughp

ut (K

bps)

MQTTwTCPMQTTwQUIC

Changed IP address

Changed IP address

Changed IP address





V RELATED WORK








16








B QUIC





17



radicX

radicX X

radic radic



radic




radic



VI CONCLUSION





REFERENCES















18




















































19





























I Introduction

I-A TCP and UDP

I-B TLS and DTLS

I-C Contributions

II QUIC



II-C Security

II-D Multiplexing



III-A Architecture


III-C Server-agent

III-C1 main()



III-D Client-agent

III-D1 main()













IV Evaluation





V Related Work








V-B QUIC

VI Conclusion

References

12







Clientmc pkj

Serverms



key_settlement





final_data



F Code Reduction



Subscriber

Internet


Broker



Broker

BrokerLocated in

Washington DC

Subscriber


Publisher


California


IV EVALUATION











13

Subscriber Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


Connect ACK (MQTT)

ACK (TCP)


ACK (TCP)

Client Hello (TLS)

Server Hello (TLS)





Data Exchange


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

Publisher Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


ACK (TCP)

Connect ACK (MQTT)


Publish ACK (MQTT)

Client Hello (TLS)

Server Hello (TLS)





1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

ACK (TCP)


Data Exchange

Publisher Broker


REJ (QUIC)






Connect ACK (MQTT)


Publish ACK (MQTT)


Data Exchange

1

2

3

4

5

6

7

8

9

10

Subscriber Broker





Connect ack (MQTT)





Data Exchange

1

2

3

4

5

6

7

8

9

10

11

12






Connect ack (MQTT)




Publish ACK (MQTT)

Data Exchange

Publisher Broker

1

2

3

4

5

6

7

8

9

10






Connect ACK (MQTT)


Publish ACK (MQTT)

1

2

3

4

5

6

7

8

Data Exchange

(a) MQTTwTCP

(b) MQTTwQUIC 1-RTT

(c) MQTTwTCP 0-RTT



0

5

10

15

20

25

30

35

Num

ber o

f Pac

kets









14











1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

200

400

600

800

Mem

ory

Usa

ge (M

B)

MQTTwTCPMQTTwQUIC

TCPQUIC

SubscribingClients

Publishersstarted

receving

Publishersrestarted

ConnectingClients


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

20

40

60

80

100

Proc

esso

r Util

izat

ion

()

MQTTwTCPMQTTwQUIC

TCPQUIC

PublisherRestarted

ConnectingClients

SubscribingClients

Publisherstarted

Receiving



15







0 2 4 6 8 10 12 14 16Minutes

0

5

10

15

20

25

30

Thro

ughp

ut (K

bps)

MQTTwTCPMQTTwQUIC

Changed IP address

Changed IP address

Changed IP address





V RELATED WORK








16








B QUIC





17



radicX

radicX X

radic radic



radic




radic



VI CONCLUSION





REFERENCES















18




















































19





























I Introduction

I-A TCP and UDP

I-B TLS and DTLS

I-C Contributions

II QUIC



II-C Security

II-D Multiplexing



III-A Architecture


III-C Server-agent

III-C1 main()



III-D Client-agent

III-D1 main()













IV Evaluation





V Related Work








V-B QUIC

VI Conclusion

References

13

Subscriber Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


Connect ACK (MQTT)

ACK (TCP)


ACK (TCP)

Client Hello (TLS)

Server Hello (TLS)





Data Exchange


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

Publisher Broker

SYN (TCP)

SYN + ACK (TCP)

ACK (TCP)


ACK (TCP)

Connect ACK (MQTT)


Publish ACK (MQTT)

Client Hello (TLS)

Server Hello (TLS)





1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

ACK (TCP)


Data Exchange

Publisher Broker


REJ (QUIC)






Connect ACK (MQTT)


Publish ACK (MQTT)


Data Exchange

1

2

3

4

5

6

7

8

9

10

Subscriber Broker





Connect ack (MQTT)





Data Exchange

1

2

3

4

5

6

7

8

9

10

11

12






Connect ack (MQTT)




Publish ACK (MQTT)

Data Exchange

Publisher Broker

1

2

3

4

5

6

7

8

9

10






Connect ACK (MQTT)


Publish ACK (MQTT)

1

2

3

4

5

6

7

8

Data Exchange

(a) MQTTwTCP

(b) MQTTwQUIC 1-RTT

(c) MQTTwTCP 0-RTT



0

5

10

15

20

25

30

35

Num

ber o

f Pac

kets









14











1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

200

400

600

800

Mem

ory

Usa

ge (M

B)

MQTTwTCPMQTTwQUIC

TCPQUIC

SubscribingClients

Publishersstarted

receving

Publishersrestarted

ConnectingClients


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

20

40

60

80

100

Proc

esso

r Util

izat

ion

()

MQTTwTCPMQTTwQUIC

TCPQUIC

PublisherRestarted

ConnectingClients

SubscribingClients

Publisherstarted

Receiving



15







0 2 4 6 8 10 12 14 16Minutes

0

5

10

15

20

25

30

Thro

ughp

ut (K

bps)

MQTTwTCPMQTTwQUIC

Changed IP address

Changed IP address

Changed IP address





V RELATED WORK








16








B QUIC





17



radicX

radicX X

radic radic



radic




radic



VI CONCLUSION





REFERENCES















18




















































19





























I Introduction

I-A TCP and UDP

I-B TLS and DTLS

I-C Contributions

II QUIC



II-C Security

II-D Multiplexing



III-A Architecture


III-C Server-agent

III-C1 main()



III-D Client-agent

III-D1 main()













IV Evaluation





V Related Work








V-B QUIC

VI Conclusion

References

14











1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

200

400

600

800

Mem

ory

Usa

ge (M

B)

MQTTwTCPMQTTwQUIC

TCPQUIC

SubscribingClients

Publishersstarted

receving

Publishersrestarted

ConnectingClients


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Minutes

0

20

40

60

80

100

Proc

esso

r Util

izat

ion

()

MQTTwTCPMQTTwQUIC

TCPQUIC

PublisherRestarted

ConnectingClients

SubscribingClients

Publisherstarted

Receiving



15







0 2 4 6 8 10 12 14 16Minutes

0

5

10

15

20

25

30

Thro

ughp

ut (K

bps)

MQTTwTCPMQTTwQUIC

Changed IP address

Changed IP address

Changed IP address





V RELATED WORK








16








B QUIC





17



radicX

radicX X

radic radic



radic




radic



VI CONCLUSION





REFERENCES















18




















































19





























I Introduction

I-A TCP and UDP

I-B TLS and DTLS

I-C Contributions

II QUIC



II-C Security

II-D Multiplexing



III-A Architecture


III-C Server-agent

III-C1 main()



III-D Client-agent

III-D1 main()













IV Evaluation





V Related Work








V-B QUIC

VI Conclusion

References

15







0 2 4 6 8 10 12 14 16Minutes

0

5

10

15

20

25

30

Thro

ughp

ut (K

bps)

MQTTwTCPMQTTwQUIC

Changed IP address

Changed IP address

Changed IP address





V RELATED WORK








16








B QUIC





17



radicX

radicX X

radic radic



radic




radic



VI CONCLUSION





REFERENCES















18




















































19





























I Introduction

I-A TCP and UDP

I-B TLS and DTLS

I-C Contributions

II QUIC



II-C Security

II-D Multiplexing



III-A Architecture


III-C Server-agent

III-C1 main()



III-D Client-agent

III-D1 main()













IV Evaluation





V Related Work








V-B QUIC

VI Conclusion

References

16








B QUIC





17



radicX

radicX X

radic radic



radic




radic



VI CONCLUSION





REFERENCES















18




















































19





























I Introduction

I-A TCP and UDP

I-B TLS and DTLS

I-C Contributions

II QUIC



II-C Security

II-D Multiplexing



III-A Architecture


III-C Server-agent

III-C1 main()



III-D Client-agent

III-D1 main()













IV Evaluation





V Related Work








V-B QUIC

VI Conclusion

References

17



radicX

radicX X

radic radic



radic




radic



VI CONCLUSION





REFERENCES















18




















































19





























I Introduction

I-A TCP and UDP

I-B TLS and DTLS

I-C Contributions

II QUIC



II-C Security

II-D Multiplexing



III-A Architecture


III-C Server-agent

III-C1 main()



III-D Client-agent

III-D1 main()













IV Evaluation





V Related Work








V-B QUIC

VI Conclusion

References

18




















































19





























I Introduction

I-A TCP and UDP

I-B TLS and DTLS

I-C Contributions

II QUIC



II-C Security

II-D Multiplexing



III-A Architecture


III-C Server-agent

III-C1 main()



III-D Client-agent

III-D1 main()













IV Evaluation





V Related Work








V-B QUIC

VI Conclusion

References

19





























I Introduction

I-A TCP and UDP

I-B TLS and DTLS

I-C Contributions

II QUIC



II-C Security

II-D Multiplexing



III-A Architecture


III-C Server-agent

III-C1 main()



III-D Client-agent

III-D1 main()













IV Evaluation





V Related Work








V-B QUIC

VI Conclusion

References

Implementation and Analysis of QUIC for MQTT · Implementation and Analysis of QUIC for MQTT Puneet...

Documents

Transcript of Implementation and Analysis of QUIC for MQTT · Implementation and Analysis of QUIC for MQTT Puneet...