Sensor Data Market for the Internet of Things
Ricardo Soares Miranda
Thesis to obtain the Master of Science Degree in
Electrical and Computer Engineering
Supervisor(s): Prof. António Manuel Raminhos Cordeiro GriloProf. Miguel Filipe Leitão Pardal
Examination Committee
Chairperson: Prof. Horácio Cláudio de Campos NetoSupervisor: Prof. António Manuel Raminhos Cordeiro Grilo
Member of the Committee: Prof. Alberto Manuel Ramos da Cunha
June 2018
ii
Declaration
I declare that this document is an original work of my own authorship and that it fulfills all the require-
ments of the Code of Conduct and Good Practices of the Universidade de Lisboa.
iii
iv
Acknowledgments
I would like to start by thanking my Supervisors, Professor Antonio Grilo and Professor Miguel Pardal
for their advices, guidance, motivation and patience. Professor Grilo and Professor Pardal were always
open to hear my doubts and to share their thoughts and knowledge. To my parents, to whom I express
my profound gratitude for their unfailing support and encouragement to complete this chapter in my life.
To my girlfriend, who even in the most stressful moments always provided great motivation and joyful
moments. To all my friends for sharing frustration, progress, knowledge and perspectives.
v
vi
Resumo
Existem actualmente milhoes de sensores ligados a Internet. Estes sensores recolhem os mais variados
tipos de dados, desde temperatura, nıvel de humidade ou som e imagem, passando por localizacao
(atraves de GPS - Global Positioning System) ou o numero de batimentos cardıacos por segundo, por
exemplo. Estes dados podem ser obtidos para fins cientıficos, comerciais ou outros. Contudo, nao existe
nenhuma plataforma ou mercado online em que seja possıvel comprar ou vender dados. No presente
trabalho, propoe-se e implementa-se uma solucao atraves da qual e possıvel realizar essa transaccao.
Sensor Data Market for the Internet of Things (IoT) e uma plataforma em que fornecedores, entidades
ou pessoas, partilham os seus dispositivos ou dados e os consumidores podem comprar dados ou
adquirir o controlo de um dispositivo. Mais do que adquirir dados em tempo real, isto e, um sensor
recolhe uma amostra e o consumidor recebe essa amostra com um atraso inferior a uma determinada
meta temporal, a solucao apresentada permite que uma pessoa ou entidade tenha controlo sobre um
dispositivo. Esta funcionalidade e considerada como sendo diferenciadora, uma vez que nao se trata
apenas de uma troca de dados, mas sim de controlo de um dispositivo. Um cliente pode posicionar
uma camara ou movimentar um robot. A plataforma foi testada e avaliada atraves da realizacao de
casos de uso e de recolha e analise de metricas, tais como atraso entre mensagens ou o racio entre
mensagens recebidas e mensagens nao recebidas. A analise destes resultados permite concluir que a
solucao desenvolvida possibilita a partilha de sensores e dos dados que estes recolhem.
Palavras-chave: sensor, dados, mercado, troca, IoT, controlo
vii
viii
Abstract
Currently, millions of sensors are connected to the Internet. These sensors gather various types of data,
from temperature, humidity, sound and image, to location (through GPS - Global Positioning System)
or number of heartbeats per second, for example. This data can be obtained for scientific or business
purposes, amongst others. However, there is no online platform or marketplace where data can be
exchanged. In the present work, we design and implement a solution through which it is possible to
purchase and sell sensor data. Sensor Data Market for the Internet of Things (IoT) is a platform where
suppliers, entities or individuals, share their devices or data and consumers can buy data or acquire
control of a device. More than acquiring real-time data, i.e., the sensor collects a sample and the
consumer receives that sample with a delay of less than a predefined deadline, the presented solution
allows for an entity to have control over a device. This feature is considered to be differentiating, since
it is not just a data exchange, but the ability to control a sensing device. A customer can position a
camera or move a robot. The platform was tested and evaluated through the implementation of use
cases and the analysis of various parameters such as, delay between messages or the ratio between
incoming messages and unreceived messages. Through analysis of these results we conclude that the
implemented solution allows for customers to share sensors and the data that they collect.
Keywords: sensors, data, market, exchange, IoT, control
ix
x
Contents
Declaration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v
Resumo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
Nomenclature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 Background 5
2.1 Sensing Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.1 Simple Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.1.2 Rich Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 Communication Protocols Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2.1 Hypertext Transfer Protocol (HTTP) . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2.2 Constrained Application Protocol (CoAP) . . . . . . . . . . . . . . . . . . . . . . . 7
2.2.3 Message Queue Telemetry Transport (MQTT) . . . . . . . . . . . . . . . . . . . . . 7
2.3 IoT Cloud Platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.4 Amazon Web Services IoT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.5 S2aaS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.6 Marketplace Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3 Developed S2aaS Solution 15
3.1 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.1.1 Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.1.2 Device Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
xi
3.1.3 Broker Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.1.4 Marketplace Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.2 IoT Marketplace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.3 Simple Sensor Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.4 Rich Sensor Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4 Results and Evaluation 49
4.1 Single Data Measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.2 Long Data Measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.3 Video and Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
5 Conclusions 59
5.1 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Bibliography 63
xii
List of Tables
2.1 Comparison between three IoT Platforms. . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.1 Single Data Results: Processing time for each actor and overall time difference (seconds). 52
4.2 Long Data Results: Processing time for each actor and overall time difference (seconds). 54
4.3 Video and Control Results: Expended time in the TCP connection and overall two-way
latency (seconds). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
xiii
xiv
List of Figures
2.1 Architecture of the AWS IoT platform. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2 Conceptual marketplace model. The platform connects buyers and sellers. . . . . . . . . 14
3.1 General architecture of the Sensor Market for the Internet of Things. . . . . . . . . . . . . 16
3.2 Interaction between each participant upon registering an account. . . . . . . . . . . . . . 17
3.3 Interaction between each participant upon registering a device. . . . . . . . . . . . . . . . 18
3.4 Class Account and implemented methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.5 Classes Wallet and Transaction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.6 Abstract class Data and Class Temperature (a subclass of Data). . . . . . . . . . . . . . . 23
3.7 Abstract class Device and the methods every subclass will inherit. . . . . . . . . . . . . . 23
3.8 Class Sensor (a subclass of Device) and the methods implemented. . . . . . . . . . . . . 24
3.9 Abstract class VideoDevice is the super class of all video-related implemented classes. . 24
3.10 Class Camera implemented considering the functionalities of a controllable camera. . . . 25
3.11 Abstract class Catalog and the subclasses DeviceCatalog and DataCatalog. . . . . . . . 26
3.12 Class Product implemented to act as a item in a stock inventory. . . . . . . . . . . . . . . 26
3.13 Class Stock developed to interact with a Product. . . . . . . . . . . . . . . . . . . . . . . . 27
3.14 Abstract class Customer and the subclasses Buyer and Seller. . . . . . . . . . . . . . . . 28
3.15 Class ServerInterface is the intermediary between the Marketplace and the broker module. 28
3.16 Flowchart of a purchase, either for a Simple Sensor or for a Rich Sensor. . . . . . . . . . 30
3.17 Messages exchanged between each participant regarding a data transaction. . . . . . . . 31
3.18 Messages exchanged between each participant regarding a video transaction. . . . . . . 31
3.19 UML diagram of marketplace classes relationships. . . . . . . . . . . . . . . . . . . . . . . 32
3.20 Sequence of interactions between each participant. The AWS IoT platform is present in
steps 5 and 6. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.21 Workflow of a Data Handler for a simple data transaction. . . . . . . . . . . . . . . . . . . 36
3.22 Workflow of a Data Handler for a long data transaction. . . . . . . . . . . . . . . . . . . . 38
3.23 Single costumer interaction with a sensor. . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.24 Multiple costumers consuming the data stream. . . . . . . . . . . . . . . . . . . . . . . . . 39
3.25 Message exchange - Purchase of video & control. . . . . . . . . . . . . . . . . . . . . . . 41
3.26 Sequence of interactions between each participant in a video & control purchase. . . . . . 42
3.27 Workflow of the Video Broker. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
xv
3.28 Workflow of a Data Handler instance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.29 Single costumer utilizing the controllable video camera. . . . . . . . . . . . . . . . . . . . 46
3.30 Multiple costumers consuming the video stream. . . . . . . . . . . . . . . . . . . . . . . . 47
4.1 Number of allocated Handlers and their lifetime. . . . . . . . . . . . . . . . . . . . . . . . . 50
4.2 Temporal discrepancy between reception and schedule times. . . . . . . . . . . . . . . . . 51
4.3 Average time usage for each participant regarding a Data transaction . . . . . . . . . . . 52
4.4 Temporal discrepancy between reception and schedule times. . . . . . . . . . . . . . . . . 54
4.5 Average time usage for each component regarding a Video and Control transaction. . . . 56
xvi
Chapter 1
Introduction
The Internet of Things ”is a world where physical objects are seamlessly integrated into the information
network, and where the physical objects can become active participants in business processes” [1]. In
this everything-is-connected world, even everyday objects can be integrated into the network (e.g. a
temperature sensor, the coffee machine, a car, etc.). The technology or Internet protocol that enables
such connectivity, for instance, Bluetooth, ZigBee, Radio Radio-frequency identification (RFID), may
change but, nevertheless, the concept of interconnection remains.
The infrastructure and the popularity of IoT make new interaction models possible, such as Sensing-
as-a-Service (S2aaS). The S2aaS model depicts a marketplace where interested consumers trade sen-
sor data. Nowadays, in developed countries, the population is concentrated mostly in big cities and
industrial areas. A Smart City is a good example for the growth of IoT. It is possible to measure a variety
of parameters and data in an urban area, from pollution to automotive traffic, luminosity in the streets,
etc.
1.1 Motivation
A Sensor Data Market will be as strong as the real-life scenarios empowered by it. Without any real-
life scenarios, the marketplace may just be an idea, but with no application. The following is a set of
possibilities enabled by IoT devices equipped with sensors. The scenarios could be integrated in a
marketplace.
Atmospheric pollution control and sports tracking
A network of devices capable of measuring humidity, temperature, wind and CO2 levels in the air
could not only collect data for a private entity, but also supply a governmental institution. The private
entity creates a website through which presents this data to athletes or sports enthusiasts showing
where the cleaner areas are.
Noise pollution control
What if it was possible for a person who is buying a house, for instance, to know if a certain area
1
in a neighbourhood has a high level of noise pollution? Every smartphone and laptop has a built-in
microphone, allowing every person to measure the noise in its proximity. Such data could be uploaded
to a platform and then shown to any interested participant.
Radio frequency spectrum monitoring
Dense urban areas have hundreds of devices connected to a cell tower from a service provider. In
the radio domain, the allocated bands for each company are highly regulated. It is in the best interest of
every party to avoid interference between signals originated by competitors or from adjacent cell towers.
Also, the regulatory entity must ensure that each service provider is operating according to its contracts
and occupying just the agreed spectrum.
Automotive traffic management
In modern cities, managing traffic is a complex problem. Knowing the flow of vehicles in real-time,
how and where to divert the traffic could lead to a better traffic management. Infra-red sensors measur-
ing the number of vehicles crossing its beam, or video cameras, in traffic lights, showing live feeds of a
road or intersect could help to solve this situation.
Directed Advertisement
The growth of social media platforms, such as Facebook or Twitter, enables a sort of directed adver-
tisement or tailored ads focused to a very specific demographic. This new mechanism requires a crucial
element: information (e.g, age, sex, current location, etc). Knowledge about where a certain type of
person usually is or passes by will most certainly assist a company to plan the correct site to place an
ad or a nearby restaurant to send an alert to the smartphone of a potential customer.
All these scenarios have a common factor: data consumers need to find providers of sensor data.
Creating a centralized platform or marketplace, allowing for every person and entity to buy or sell data
from a variety of sensors that collect all sorts of information, might not only accomplish, but also simplify
the scenarios presented above. Furthermore, a universal Sensing-as-a-Service platform has yet to be
developed.
1.2 Objectives
The main goal of this project was to design, implement and test a Sensor Data Market for the Internet
of Things. The solution considers the characteristics and limitations of different sensing devices. Addi-
tionally, it facilitates integration with existing IoT platforms and makes use of communication protocols
specifically designed for sensors. Furthermore, one of the key functionalities of this project is to share
sensors and to buy or sell the data they produce.
2
1.3 Thesis Outline
In Chapter 2 of this document, we start by analysing the state-of-the-art. In Chapter 3, we introduce
the proposed solution and define the structure of the platform. Chapter 4 defines how our work will be
tested and evaluated. Finally, in Chapter 5 we present our conclusions and ideas for future work.
3
4
Chapter 2
Background
This chapter presents the related work developed not only in recent literature, but also from an industry
point of view. A brief study of the current sensing devices is presented in Section 2.1, the selection of the
most adequate communication protocol in Section 2.2. In Section 2.3 we present current IoT platforms
and Section 2.4 highlights the most complete one, in our perspective. Section 2.5 presents an overview
of the existing work developed in S2aaS in recent literature. Section 2.6 presents a brief summary on
the electronic marketplace model, as we will adapt this concept to the IoT context.
2.1 Sensing Devices
One of the most common objects in the world is a smartphone. The number of smartphones is expected
to have reached 2.6× 109 in 2017 and pass 5× 109 by 2020. Additionally, according to Perera et. al [2]
there will be 5× 1010 to 1× 1011 devices connected to the Internet by 2020.
Smartphones are equipped with several embedded sensors (GPS, microphone, camera, etc.) mak-
ing it an interesting study case to be used as a sensor device in S2aaS applications. Some recent
studies such as [3] seek to explore the role of smartphones in a S2aaS architecture. The authors aim to
understand how to integrate these devices and what kind of participation they have, i.e. taking measure-
ments or ordering readings, either automatically or with user interaction. They also address the need
for efficient management of physical (sensors)and software (selection of sensors to be used) resources
and also of energy saving(smartphones). Besides the analysis of the device’s role, the article contains
recommendations for the development of an incentive mechanism, which is a very important component
in order to develop a successful S2aaS solution.
Considering the millions of sensors in existence, their specific characteristics and functionalities it
should be important to differentiate each device in order to properly categorize them. This project at-
tempts to differentiate the devices into two categories: Simple devices and Rich devices. However, this
model only applies to devices or networks of devices capable of communicating with the Internet.
5
2.1.1 Simple Devices
From Machine-to-Machine (M2M) sensor systems to WiFi connected LED lamps, there are millions of
devices connected to the Internet. Some devices perform only simple tasks, such as measuring data
or receiving instructions. Those devices are categorized as simple devices. A smartphone can be
seen as a single simple device composed of several sensors. Each sensor can measure data and
the smartphone is responsible to send it. A network of street lamps with luminosity sensors, could, in
addition to their normal functionalities, share the data measured to the public through a marketplace.
Dozens of temperature, humidity and air pollution sensors could be scattered in a city to create a map
of the city’s most polluted areas, not only to scientific community and to the energy producers, but
also to the public. Despite being considered as simple devices, these sensors can be crucial to a future
Marketplace for the Internet of Things. These devices can be less expensive than more complex devices
and easier to deploy.
2.1.2 Rich Devices
Rich devices can be described as devices capable of execution more complex tasks than the simple
sensors. Furthermore, these devices can require more processing power, create sound or video streams
or receive control requests. Controllable cameras, Unmanned Aerial Vehicles (UAVs), or robots can be
an example of rich devices. A controllable telescope could be available to the public, thus allow for a
person to use this device and observe a specific planet or star, for example. Consider a person who
intends to create a video clip or collect pictures of a natural landscape or monument. A small and
controllable UAV, available through the marketplace, might be the solution of the previous scenario.
2.2 Communication Protocols Analysis
The Internet of Things communication protocol stack is important. The vast majority of devices in the IoT
are small sensors, thus having low processing power and being very energy dependent, but also they
are limited devices and cannot have a full TCP/IP stack. Although the security matter is a critical issue,
the discussion presented below is focused on the interoperability and efficiency of the overall system,
therefore directed to the application layer in the protocol stack. As the devices have limited processing
capability, the communication protocol cannot be too demanding, the overhead should be reduced, the
encryption must be, for now, an add-on. Furthermore, considering that the device is probably connected
to a low capacity battery, the communication process should be efficient and fast: measure, compute
the data and send.
2.2.1 Hypertext Transfer Protocol (HTTP)
One of the most widely known and employed protocols is the Hypertext Transfer Protocol. HTTP [4]
is an application-level protocol characterized by a request/response model and the communication is
6
usually established over TCP/IP. TCP connections do not perform well in the IoT world, due to its pri-
mary characteristic, the connection. By establishing a connection, with delivery assurance method and
congestion control system, it creates an environment prone to be time consuming and resource (energy
and processing power) demanding. Furthermore, HTTP headers occupy a large portion of the mes-
sage, resulting in a low payload size that may cause issues when transmitting larger sets of information.
Nevertheless, it is a widely-utilized protocol, thus allowing a higher interoperability between the various
components of the platform.
Amazon IoT and Azure IoT support HTTP protocol via a Representational State Transfer Application
Programming Interface (REST API), allowing the execution of standard operations in a simple manner.
For example, it is possible to publish data invoking a POST request message in Azure IoT.
2.2.2 Constrained Application Protocol (CoAP)
CoAP [5] is a specialized web transfer protocol for use with constrained nodes and constrained network,
such as Wireless Sensor Networks (WSNs), and also for machine-to-machine (M2M) applications. Un-
like HTTP, CoAP communicates over UDP (User Datagram Protocol) a connection-less transport pro-
tocol, thus not having the issues caused by the TCP connections. CoAP provides a request/response
interaction model, in the same sense as HTTP, although it does not aim to only compress it. Rather,
it optimizes a subset of REST applications common with HTTP. Although CoAP is not a highly-utilized
protocol, by implementing a gateway, it can communicate with HTTP agents, thus increasing the inter-
operability of this protocol. Two other key aspects are the low header overhead, allowing more data
payload in the message, and the processing power requirements, which are very small. However if the
message size increases the probability of message loss in UDP becomes higher than in TCP, therefore
CoAP must retransmit the whole message.
2.2.3 Message Queue Telemetry Transport (MQTT)
MQTT [6] is a widely adopted lightweight publish/subscribe messaging protocol. It is useful for use
with low power sensors and machine-to-machine applications. In IoT a sensor represents a publisher,
gathers data and sends it to a broker. An entity that is interested in a certain topic, for example, sensory
information, connects to a broker and subscribes a topic. Afterwards, it will receive the data collected
that belongs to that topic. This architecture has three main components: Publisher, Broker, Subscriber.
In MQTT a message is published to a topic. Clients may subscribe various topics and every client
subscribed to a topic will receive every message sent to that topic. For the broker this is impervious, as
this element does not read the messages, it only classifies the data into topics and sends the messages
to the subscribers. This broker may represent a problem: every publisher must be configured with the
broker’s address and the broker itself can become a single-point of failure. MQTT communicates via
TCP so, as with HTTP, there may be some issues regarding the performance of the overall transmission
when compared to CoAP (UDP).
7
As discussed further in Section 2.4, Amazon IoT message broker is based on MQTT v3.1.1, but with
some particularities. Azure IoT also natively supports the MQTT protocol.
2.3 IoT Cloud Platforms
One of the easiest impacts to acknowledge of a sensing application is its data output. Zaslavsky et al [7],
discuss the future role of sensing in terms of the needs of information storage, as well as the growing
importance of processing all this data created by sensors or other devices. The authors approach
S2aaS, its architecture, but not in high detail. An example of a S2aaS scenario is described, however the
example is equal to one presented in [8]. There is also a reference to the possibility of cloud computing
acting a service to process the data purchase in a S2aaS platform.
Currently, there are several IoT cloud platforms in the market from several world-renowned compa-
nies: IBM Watson IoT [9], Microsoft Azure IoT Hub [10] and Amazon AWS IoT [11]. In this section an
analysis of those IoT platforms is presented. Additionally, below is brief discussion of the security com-
ponent of each platform that was analysed. The security issue is also an important module of every IoT
cloud platform. One of the largest cyber-attacks recently registered [12] was an DDoS attack targeted
at a DNS service provider. The primary weapon of this attack was the Mirai botnet, which is mainly
composed of infected IoT devices.
• AWS IoT: The AWS IoT message broker and Thing Shadows service encrypt all communications
with Transport Layer Security (TLS). TLS is used to ensure the confidentiality of the application
protocols (MQTT, HTTP) supported by AWS IoT. For MQTT, TLS encrypts the connection between
the device and the broker. TLS client authentication is used by AWS IoT to identify devices. For
HTTP, TLS encrypts the connection between the device and the broker.
• Azure IoT Hub: The communication path between devices and Azure IoT Hub, or between gate-
ways and Azure IoT Hub, is secured using industry-standard TLS with Azure IoT Hub authenticated
using X.509 protocol.
• IBM Watson IoT: The Watson IoT Platform supports connectivity over TLS v1.2. Devices con-
nected use a combination of clientId and the authentication token that is generated when adding
the devices to a platform.
Table 2.1 shows that the three platforms are similar, however the implementation of each element is
obviously different [11][10]. The cloud device, which is a digital representation of the physical device,
varies in a significant manner. The Application Programming Interface (API) and the Software Devel-
opment Kit (SDK) can be developed in several programming languages such as, Python, C, Java and
NodeJS.
AWS IoT can generate a certificate automatically when a new device is registered in the platform.
The certificate is used for device authentication. Thus, ensuring a more secure system than other
schemes, such as user name and password. Creation, management and revocation of certificates
8
can be achieved through the AWS SDK or CLI. It is also possible to disable certificate-authentication,
although not recommended. Azure IoT and IBM IoT implement a similar security scheme. IBM IoT
allows for different security policies, characterized by combinations of authentication via certificate, via
token, or via both. All platforms implement TLS connections in client (device) to server communication
though, IBM platform allows the deactivation of this mechanism.
Azure IoT Hub AWS IoT IBM Watson IoT
Protocols MQTT, HTTP and AMQP HTTP and MQTT HTTP and MQTT
Gateway Possible Possible Possible
Interaction with
API and SDK Yes Yes Yes
CLI Azure CLI + Azure PowerShell AWS CLI Watson CLI (Python)
Cloud Device Device Twin Device Shadow Simple Device
Security X.509 certificates + TLS X.509 certificates + TLS Certificate/Token + TLS
Table 2.1: Comparison between three IoT Platforms.
Azure IoT Hub
The Azure IoT Hub provides a collection of services for connecting and extracting data from sensing
devices. In similarity with existing IoT platforms, Azure IoT Hub enables bidirectional communication
between the devices and the cloud application. One of the main characteristics of this platform is the
support of a variety of communication protocols, directly or through a gateway. Furthermore, Azure IoT
Hub exposes an API capable of sending messages to the devices, if the devices support this mechanism.
Following are presented the key elements in Azure IoT Hub Architecture.
Device twins
Device twins are the cloud representation of the real devices. Each device connected with the IoT
Hub has an individual Device twin. The device’s configuration and metadata are stored in the Device
twin – JSON document.
Gateways
The gateway is deployed to perform a translation between two communication protocols. Azure IoT
Hub defines two types of gateways: protocol gateway and field gateway. The first is only capable of
protocol translation, the latter also provides device management services, for example. A gateway is
required when a sensing device cannot communicate with the IoT Hub in a supported protocol.
Identity registry
This component stores information from devices that are authorized to connect to the IoT Hub. Every
device permitted has an entry in the Identity registry. Additionally, the IoT Hub Identity registry provides
a set of operations to manage the devices connected to the IoT Hub.
9
IBM Watson IoT
The IBM Watson IoT Platform provides a versatile IoT toolkit to connect, manage and extract data from
a wide variety of sensing devices. Watson IoT Platform can communicate or interact with external
applications, thus having the ability to export sensor data to Databases or Real-Time Management
Dashboards. Below are stated the key components of the Watson IoT architecture.
Organizations
Every entity, company or person, has an individual account which is identified by the organization
ID. This mechanism ensures that only authenticated devices or applications can access to the data
associated to your ID.
Gateways
This module acts as a bridge between devices that cannot connect directly to the Watson Platform.
It can communicate over several protocols with the device and over MQTT with the IoT platform;
Devices
The Watson IoT divides devices in two types: managed devices and unmanaged devices. Manage
devices have a management agent and can perform a set operations, such as, update, reboot, shut
down, etc. These are the intelligent devices. The unmanaged devices can only send and receive
messages or commands. Unlike the managed devices, this class doesn’t have a management agent
neither can receive management instructions – dummy devices;
Events
This component is somewhat comparable to the topic in AWS IoT or in MQTT protocol. Devices
publish data using the event mechanism. Devices send data linked to an event and an application or
service can subscribe this event and receive the information.
OpenStack
OpenStack [13] is a cloud operating system that controls large sets of devices or resources throughout
a datacenter, all managed through a dashboard. The OpenStack’s architecture is composed by several
independent modules. These modules provide APIs for the administrator and the users to manage the
resources, send messages between devices and communicate within the OpenStack ecosystem itself.
OpenIoT
The OpenIoT [14] middleware infrastructure will support flexible configuration and deployment of algo-
rithms for collection, and filtering information streams stemming from the internet-connected objects,
while at the same time generating and processing important business/applications events.
10
Discussion
Innovation plays a crucial role in this field. Every company desires to be ahead of the competition and
this goal can be achieved by simplifying the interaction between the user and the platform or by creating
new services or applications. Considering that these three platforms are relatively new, it is expected to
see new functionalities or even changes in current platforms.
After analysing all platforms, the Amazon Web Services IoT platform was identified as being the most
interesting. A more detailed description is presented next.
2.4 Amazon Web Services IoT
The communication architecture of the AWS IoT platform revolves around the concept of topic and state.
The state can be a value, a key-word or a symbol. The topic acts like a channel, a device publishes
a message to one topic and clients can subscribe to these topics to receive new messages sent by
devices. This is how the publish and subscribe model works in AWS IoT. When a device (sensor) sends
data, it updates the current state of a property. This information is stored in the Thing Shadow. The
Thing Shadow can have multiple pairs of property-state that are encoded in JSON format. Also, a client
with an app or using the AWS IoT CLI can modify the state of a property. When one device is registered
in the AWS IoT a copy of it is created, a Thing Shadow. As said above, the properties and attributes
given and the existence of a Registry allow a client to manage its devices. It is possible to update a set
of devices altogether without manually send the data, delete, update (state and attributes) and remove
devices. Using the REST API one can query and list the devices, besides accessing the other available
operations. A device connects and sends/receives data to the AWS over MQTT, HTTP or WebSockets.
The Gateway and the Message Broker allow secure and low-overhead communication between the
devices, the cloud and other applications. By deploying a Rule, via the Rule Engine, it is possible to
store or process data sent by sensors, or even filter it. The device is programmed considering the
AWS IoT architecture, the communication protocols and the framework. The configuration of the device
is accomplished via the AWS IoT SDKs. However, the REST commands simplify the communication
and configuration. Amazon developed a set of SDKs from which a group was designed to be used on
micro-controllers/resource-constrained devices.
The AWS IoT platform is composed of several elements, which perform various functions. These ele-
ments can be observed at Figure 2.1. When analysing this platform some components are immediately
seen as fundamental. Below are stated the key components.
Device Gateway
The Device Gateway mediates the communication between connected devices and the cloud ca-
pabilities. The gateway operates using the publish/subscribe configuration, the MQTT protocol. It also
supports publishing using HTTPS protocol.
Message Broker
11
The Message Broker is a publish/subscribe broker service that enables sending and receiving of
messages to and from AWS IoT. It maintains a list of all client sessions and the subscriptions for each
session and uses topics to route messages from the publisher to the subscriber. In a way, the message
broker contains the Device Gateway. The message broker implementation is based on MQTT protocol,
version 3.1.1, with some particularities. There are some restrains regarding the Quality of Service (QoS),
for example AWS IoT does not support the QoS level 2. There is no guarantee that messages will be
delivered in the correct order. Also the AWS IoT broker does not support persistent sessions and cannot
store any messages, thus by requesting such functionalities the client will be disconnected.
Figure 2.1: Architecture of the AWS IoT platform.
Thing Registry
This component performs the role of a database or a list to store the Things. A Thing can be identified
by its name or attributes, allowing the user to manage, search and update it for example.
Thing Shadow (Device Shadow)
A Thing is a representation of a real device or sensor. A Thing Shadow is a JSON document that is
used to store and retrieve current state information. When a device communicates with the AWS IoT it
sends a new state for its properties. For example, it is possible for a user with an app to turn on a light
bulb. This action is accomplished by updating the state of the propriety (in the Thing Shadow) connected
to that functionality. The AWS IoT will then send a message to the device telling it to turn on.
Rules Engine
A set of instructions that allow the interaction of the devices with other AWS Applications. It is
12
possible to redirect the message to a queue, feed it to a Machine Learning Service, store it in a DB, etc.
2.5 S2aaS
S2aaS model or cloud platform has so far not been the target of many publications.
Perera et. al [8] discuss a detailed architecture of a Sensing System as a Service, and more specifi-
cally a possible structure for an implementation in a Smart City. The authors describe the various layers
of the model and categorize various classes of information, sensors/devices and their owners. It gives
examples of some application opportunities within a Smart City and also describes three imaginary op-
eration scenarios. In the end the authors present several advantages and benefits of S2aaS model.
Another good study on S2aaS was developed by Chang et. al on [15]. Although this article focuses
in the framework of the Mobile S2aaS, the authors discuss and classify the types of sensing services,
sensing activities and devices. The authors design an architecture considering the providers, the clients
and the cloud platform service. The latter is developed to connect all the other elements. The framework
features the role of the Utility Cloud as a proxy to deliver the data from the supplier to the clients. To im-
plement the proposed solution the authors developed a workflow and a prototype of which they present
some results.
The solutions presented in [8] and [15] provide a foundation for future works. We will extend the work
from the solutions discussed above. The Utility Cloud in [15] is a crucial element in the architecture of a
marketplace platform. The analysis [8] of sensors and information is determinant to properly develop a
storage system.
The S2aaS subject is somewhat recent, thus every approach is laying new ground and is opening
new topics to future work or analysis.
Eduard Montserrat, from Universitat Politecnica de Catalunya developed a software allowing a client
to share his sensors with another user of the platform. This project [16] discusses the business model
and implements a web application to manage the sensor network. The authors refer that the network is
implemented through the cloud and is capable performing several actions. The platform allows its users
to add sensors, choose, search, view the available sensors on a map and buy or sell sensor data. The
approach of the business model is very limited, it does not explain how to perform transactions and does
not create the win-win situation, although this particular expression is mentioned several times. The
state-of-the-art discussion of sensor clusters and their classification is useful. The author developed a
back-end structure, created management routines for the sensors and the User Interface (website/web
app) of the platform through which a user is able for example to, create an account, find a specific sensor
or set-up a notification for a predetermined event. A description of the physical architecture, is presented
with little detail. The real sensors are replaced by a mini PC (Raspberry Pi), thus the system was not
evaluated in an actual deployment. Besides, there is no analysis of how the sensors are connected
to the platform, besides the fact that the Raspberry Pi would be running OpenStack OS to interact
with the platform. One critical aspect of a future service such as the one studied in the project is the
communication protocol between the sensors, the platform and inside the platform itself. However, the
13
author does not present an analysis of the Internet protocols suitable for this environment.
Our solution is similar to the work [16], but presents a more detailed analysis of the communication
protocols. The architecture is different and has more functionalities than the solution above. Further-
more, we conduct thorough testing of every module and operation.
2.6 Marketplace Models
Generically the marketplace will allow for consumers and suppliers to meet and to exchange data for
currency. An entity can play both roles: consumer and supplier. One company can buy data from other
entity and at the same time sell information measured by its sensors. The on-line market development by
Amazon has a method to evaluate the sellers, by reviewing the products sold or the arrival time delay, for
example. This feature transfers the certification or grading of the entities away from the platform manager
(or developer) to its users allowing it to self-manage. Figure 2.2 illustrates a simple marketplace model.
Buyers access the marketplace to acquire sensor data provided by the sellers. The marketplace acts an
intermediary between the two sides of the transaction.
Buyer 1
Buyer 2
Buyer 3
Buyer 1
Buyer 2
Buyer 3
Buyer 1
Buyer 2
Buyer 3
Buyer 1
Buyer 2
Buyer 3
Platform 1
B2B Marketplace
Platform 1
B2B Marketplace
Seller 1
Seller 2
Seller 3
Seller 1
Seller 2
Seller 3
Figure 2.2: Conceptual marketplace model. The platform connects buyers and sellers.
2.7 Summary
Considering the project at Section 2.5 and the research that has been so far carried out by the scientific
community, one can say that Sensor-as-a-Service is a viable field to study and to analyse. Besides, there
is still a wide range of subjects in open discussion, for instance, there is no standard communication
protocol for this type of application or even a fully embraced Internet protocol in the IoT world. From the
IoT platforms analysed we concluded that the AWS IoT was the most advantageous platform to integrate
in the Sensor Data Marketplace. The architecture of the AWS IoT platform, similar to the architecture
of the MQTT protocol, and the SDK provided by Amazon, used to communicate with the platform, were
the key aspects of the decision. Furthermore, the protocols HTTP and MQTT were considered to be the
most adequate to implement.
14
Chapter 3
Developed S2aaS Solution
This chapter presents the proposed solution and an analysis of the respective architecture and its com-
ponents. After analysing recent literature and investigating IoT platforms already available it was con-
cluded that, it is still not possible to rent or share a temperature sensor or sell anonymously a set of GPS
position records. Current IoT platforms allow for companies to control and analyse data from sensors
in real time, nevertheless a Sensor Market for the Internet of Things has not yet been developed. The
proposed solution is a market platform where providers have the possibility to sell sensor data or to sell
control of sensors to sensor data buyers. The proposed platform is intended to be: reliable, thus capable
of handling multiple real-time operations; secure, blocking unauthorized access to both data and devices
and user-friendly, allowing inexperienced individuals to use the marketplace. The most recent related
work on S2aaS, states there exists a potential interest for a platform that provides users with the ability
to buy and sell data collected by sensors. Furthermore, the results can constitute the basis for a future
commercial service.
3.1 Architecture
The architecture of the proposed solution is presented in Figure 3.1. The platform is comprises three
modules: device, broker and marketplace.
Every module exports a set of functions and rules, which constitutes an API. This API allows different
modules to communicate with each other. The device module is the foundation of the architecture. This
module plays a crucial role by connecting the broker module to the sensing devices. The broker module
is able to communicate with the device module via the device API. The broker modules exposes an
API to the device and marketplace modules. For example, a client will receive data from a sensing
device via the broker’s API. The marketplace module is responsible for every operation related with
the sensor market. This component handles operations, such as, consumers searching for devices or
buying sensor data/control. The marketplace will not communicate with the device module. When a
buyer purchases data, the marketplace sends this information to the broker module, which will facilitate
a communication path between the sensing device and the buyer. The marketplace module stores and
15
manages user’s account information, e.g. email, password and accountID, as well as, broker-related
records and sensor’s information and specification.
BrokerBroker
Broker API
DeviceDevice
MarketPlaceMarketPlace
Market API
Broker API Device API
IoT Marketplace
IoT Marketplace
VideoVideo
DataData
Real DevicesReal Devices AWS IoTAWS IoT
Figure 3.1: General architecture of the Sensor Market for the Internet of Things.
3.1.1 Use Cases
There will be a variety of actions that a user can perform. Some of these actions are somewhat similar
to what a person is able to find in online markets, for instance Amazon or eBay. One key aspect to notice
is that there is no platform as the one being developed in this project.
Below, a sequence of operations to be implemented in the platform is described. Besides the oper-
ations themselves, a new user must firstly sign in or register a new account. At this point, the user can
choose to either register and associate a device to his/her account or create a request for a specific set
of data. Some operations are considered critical or fundamental, for instance to register an account, to
add/register a device, to manage the devices or to perform a data transaction.
Register an Account
Every client that either publishes its devices or buys sensor data, must have an account. The account
groups the person’s or the company’s identity, online wallet and set of devices, as well as other infor-
mation. Every user has a username, email address and associated password. These are the three
basic parameters required for the system to recognize a client. In order to register itself, the customer
must submit a username, an email and a password. Furthermore, the client must choose its profile:
buyer and/or seller. Buyer if he/she intends to purchase sensor data or seller if he/she intends to share
a sensor through the marketplace. The marketplace validates the information and attempts to create
16
an account. If successfully, it will create a new account and add it to the respective pool. Figure 3.2
presents a set of methods and messages exchanged to accomplish this operation.
ViewView
1: Register an account
:Actor:Actor :Marketplace:Marketplace :Control:Control
4: Registration result
3: Create a new Buyer/Seller
3.1: Verify the information
received
1.1: Submit the username
1.3: Submit the password1.2: Submit the e-mail1.1: Submit the username
5: Present the result
Figure 3.2: Interaction between each participant upon registering an account.
Register a Device
In order for a client to publish or allow for other clients to use his/her device, the latter must be registered
and associated to an account. As there cannot be two devices with the same name in the same account,
the user must enter a new device name.
There are millions of sensors all over the world, and thousands of them can measure the temperature
or the air pressure, however there are a few parameters that differ from device to device. The sensor’s
universally unique identifier (UUID), name, model or type and serial number are usually such parame-
ters, although the first three were considered to be enough to characterize a device. The user submits
these three parameters together with a set of characteristics of the device. For a controllable camera,
the customer must submit the IP address and the image resolution. The marketplace processes the
request and validate the information. Afterwards, the new Device is added associated to its owner via
the list of devices in his/her account. Finally, a Product is created and added to the marketplace’s Stock.
Figure 3.3 illustrates the messages exchanged and the operations performed in this scenario.
Data Transaction
Transactions are the core operations of the marketplace. When it is not possible to perform a data
transaction, then a customer cannot purchase sensor data from an available sensor or request the
17
ViewView
1: Register a device
:Actor:Actor :Marketplace:Marketplace :Control:Control
4: Registration result
3: Create a new Device
3.1: Attemp to perform
the operation
1.3: Submit the UUID of the device1.2: Submit the type of device1.1: Submit the name
5: Present the result
1.4: Submit the device’s characteristics (video resolution, IP address, etc.)
4: Create a new Product
4.1: Attemp to perform
the operation
5: Add the Product to
the Stock5.1: Attemp to perform
the operation
Figure 3.3: Interaction between each participant upon registering a device.
control of a robot or camera. For instance, when an owner removes his/her devices from sale, a customer
must not be able to purchase the sensing devices that were removed. Furthermore, if a customer intends
to purchase a time-slot of a controllable camera and that slot was already purchased the marketplace
must cancel the transaction, thus ensuring no overlaps or double purchases. For each data transaction
the marketplace must validate if the customer exists, if there is sufficient balance to purchase the desired
item, amongst other verifications. Also, the device has to exist and be available for purchase.
Next, an analysis of each module is presented.
3.1.2 Device Module
When designing and developing an architecture for a given application, the variety of devices may
present an impediment or be a difficult problem to solve due to the integration of different hardware
and/or communication protocols, for example. Additionally, the message payload format may be dif-
ferent, from device to device. Sometimes bitwise operations are required to translate the payload into
intelligible information. Not all devices can communicate in the same manner, since they may be de-
signed to use different radio frequencies and communication protocols. Gateways may be needed to
hide the heterogeneity of devices from the information systems that use them. Generically the gate-
way can be any sort of software or hardware implementation that enables the communication between
two sides. This translation agent helps to expand the range of devices that can be connected to the
platform. Nevertheless, some actions may not be supported by a sensor, i.e. schedule a sensing task.
18
Considering that the device’s owner has full power over the device, one should be able to shut down the
device, reboot it or stop any further sensing activity. The device module abstracts all these heterogeneity
aspects, allowing other components of the system to access the sensor nodes in the same way.
As discussed in Section 2.4, the AWS IoT’s Shadow Thing is a virtual representation of a real sensing
device. The integration of the AWS IoT into the device module provides a simpler and wider communi-
cation interface for the device module to export. A sensing device publishes its data to its topic via the
MQTT protocol, for example. The broker module is able to interact with a topic by means of AWS IoT’s
interface. Also, the sensor’s owner can send control messages to the sensor through the same method,
i.e., the owner can send a message to reboot the sensor, via the Shadow Thing’s topic. Therefore, the
device module is composed of real devices, gateways and the AWS IoT platform.
To implement a standard language of communication, it is vital to develop an API to achieve interop-
erability between the broker and the sensing devices or the gateway. The device API allows a variety of
actions or commands, such as:
• To establish a connection between sensing device and broker module.
• To translate the payload into usable data;
• To schedule a reading task.
3.1.3 Broker Module
The broker module is the link between the marketplace and every sensor. This module is able to com-
municate with both the device module and with the marketplace module. In greater detail, the broker
module is implemented through a Data/Video Broker and Data/Video Handler. The first is responsible
for receiving and managing new requests originating from the marketplace, while the latter, establishes
and controls a communication path between sensing device and buyer/seller. Both components have
different implementations for each type of sensor. Nevertheless, the broker always communicates with
the marketplace module via a RESTful HTTP API. The broker defines a set of URLs allowing for the
marketplace or the device modules to send requests, i.e., purchase orders, data or control commands.
The broker module API implements actions such as:
• To process the purchase of a video and/or camera control;
• To process the purchase of a single data measurement;
• To process the purchase a long data measurement;
• To create a Handler for each type of purchase;
• To send control messages to a sensing device;
• To establish a communication path;
• Other actions.
19
3.1.4 Marketplace Module
As previously discussed, every person has a device, smartphone or tablet, with a set of sensors built-in.
A marketplace is the intermediary between the buyer, who wants to gather data from sensors, and the
seller, who has sensor data to sell. This module aims to create such environment where, for example, a
person can place a temperature sensor on the street where he/she lives and sell the measured data to
the country’s meteorological service.
Any entity will be able to search through the list of available devices, select one or more and send a
request. This request can be to acquire data, either already collected or a future reading, or to control a
sensor. An owner of a sensor can designate a rate or price for a single data measurement, a collection of
measurements. This entity, as mentioned before, is a representation of a person or company. An entity
can own a sensor or device and buy or sell data. For example, subject A buys data from one sensor
that belongs to subject B for a cost of X. The platform will transfer the currency to A and will send an
instruction to the broker module to transfer the purchased data to subject B. Each device is associated
to one entity. Nevertheless, the latter can have multiple devices or none. To implement this relation, a
virtual replica of a device is created. The virtual device identifies a real sensor with a variety of fields or
characteristics, i.e. device’s name, serial number, manufacturer, location, etc. Although a virtual device
is utilised to represent a real sensing device, it is also of the utmost importance when not identifying
a real device. When a client purchases real-time data from a sensing device, the data is published to
the sensor’s topic. In order to receive the data, the client listens to the sensor’s topic, via his/her virtual
device. In this case, the virtual device does not represent a real sensing device. The whole architecture
is developed around the role of the virtual device. This entity can identify a real sensor, but it is also
possible to use it just as a virtual device. This issue is addressed in Section 3.3 in greater detail.
A very common feature in on-line markets is a rating system, where users can rate and review other
users, allowing the market to auto-regulate itself. Thus, an entity selling fake data can be excluded or
down-rated by other users, thus preventing the fraudulent user to continue its actions.
Once more, the existence of a standard language of communication and set of instructions is crucial
to create a well-defined channel of communication between modules. The marketplace API implements
a variety of actions, some of which visible to the other modules, such as:
• To create or eliminate an entity;
• To add or remove (create or eliminate) a device, a virtual device;
• To exchange data between entities;
• To export some device’s fields;
• To export data belonging to an account;
• Other management actions.
• To add currency to an account;
20
• To transfer currency;
• To send actions to a sensor, i.e. schedule a reading;
• To search and list devices;
• To purchase data collected from the sensors.
3.2 IoT Marketplace
In this section, it is presented, in detail, the implementation of the marketplace module. The marketplace
is the core element for the implementation and development of a future marketplace for IoT. Through
this platform, consumers and suppliers will be able to exchange data and control of sensing devices
for currency. As discussed in Section 3.1.4, the marketplace stores and manages all the information
necessary to its operation.
Specification
One of the most utilized programming language for the development of modern applications is Java
[17]. This prototype for a marketplace was developed in Java, thus ensuring a greater compatibility with
several Internet applications, such as, AWS IoT, AWS EC2 and Google Database. Furthermore, there
is a wide availability of Java Integrated Development Environments (IDEs) and libraries. Moreover, it is
not necessary to rewrite the entire platform if one intends to update or modify a specific module or class.
When communicating with the broker it is necessary to exchange information. The interaction between
each actor occurs according to a client-server architecture model and the server does not store any sort
of state information amid request. The information has to be organized and should not be sent as plain
text of the HTTP Request. JSON [18] is a much-utilised data format in current online communication.
The information exchanged between the marketplace and the broker is in JSON format. The rich and
simple sensor broker’s implementation will be discussed in Sections 3.4 and 3.3, respectively. The
marketplace’s structure and implementation is presented below.
Account
In every online platform or service that requires registration, a user has an account. Considering the
service/platform, this account may hold more or less information. The Account class implemented in the
marketplace is a very simple object, containing the email, password and state. Besides the constructor
(create a new Account) one can perform tasks such as: log in, log out and change a password. Figure
3.4 presents the Account class and implemented methods. An Account is integrated in a Customer
class, since by itself it cannot perform any operation as regards the marketplace. The Customer class
will be explained in detail, below.
Wallet
21
Figure 3.4: Class Account and implemented methods.
In order to purchase or sell a customer must have a virtual wallet. The Wallet implemented in the
marketplace contains solely the customer’s current balance. As with the Account class, the Wallet is
also integrated in the Customer class. This class allows a customer to check his current balance, to
add funds to his wallet. The Marketplace can change the balance when a data or control transaction is
performed.
Transaction
When two customers exchange data or control of a sensing device for currency, a record of that
exchange must be kept and associated with both consumer and supplier. The Transaction class imple-
mented in the marketplace is the record of an exchange. It stores two unique IDs, one for the consumer
and one for the supplier, the value (in terms of currency) of the operation and a timestamp for when the
transaction occurs. This class is also integrated in the Customer class. However, to create a log, the
Customer class contains a list of Transaction objects. To create a registry using a list of instances from
the class Transaction, may not be the most efficient manner, nevertheless is a simple and somewhat
reliable way to achieve it. Figure 3.5 illustrates classes Wallet and Transaction. Wallets implements
methods to manage the balance and Transaction implements a constructor only.
Figure 3.5: Classes Wallet and Transaction.
Data
Data is an abstract class that represents the data measured by a sensor. This abstract class stores
the Universal Unique Identifier (UUID) of the sensor to which it belongs. Moreover, it should be imple-
mented (subclassed) for each type of data measured by the sensors in the marketplace. If the market-
place integrates a sensor that measures temperature, humidity, air pressure, light intensity or localization
there should be an implementation of each data type.
Temperature
As discussed above, Temperature is a type of data measured by a sensor. To enable trade of this
data it is required for the marketplace to have a class Temperature that implements the abstract class,
Data. Temperature stores the sensor’s precision, the minimum and maximum temperatures it is capable
of measuring and the unit of measurement. Other types of data should mirror this implementation.
22
Figure 3.6 presents abstract class Data and Temperature and inheritance relationship.
Figure 3.6: Abstract class Data and Class Temperature (a subclass of Data).
Device
The marketplace will have different types of sensors, from temperature sensors to web cameras or
remote-controlled UAVs. A web camera will not have the same characteristics as a temperature sensor.
Device is an abstract class that contains the common features of every device. It stores the UUID of the
sensor/device and the ID of its owner (customer ID). Figure 3.7 illustrates abstract class Device. The
methods implemented in Device are available to its subclasses.
Figure 3.7: Abstract class Device and the methods every subclass will inherit.
Sensor
Sensor is a subclass of Device and represents every sensor that participates in the marketplace and
that is connected to the AWS IoT. This class stores an instance of class Data, associating the sensor
to the type of data it measures, and two components from the AWS IoT: the sensor’s name and topic.
If the name or the topic are changed in the AWS IoT, they must also be changed in the marketplace,
otherwise it will not be possible for the marketplace to communicate with the sensor or for a buyer to
receive the data he/she purchased. This class implements its normal constructors, getters and setters,
plus a constructor to initiate the topic and the super class fields, only. This constructor is invoked when a
customer intends to buy real-time data (listen to a sensor’s topic for a time interval) via AWS IoT. There
is no real device involved from the buyer’s side; this customer, only has a topic associated with a virtual
device in the AWS IoT. Figure 3.8 illustrates the implementation of Sensor.
VideoDevice
VideoDevice, a subclass of Device, is also an abstract class. Since a video device may or may
not be controllable, or even be a mobile video device, VideoDevice is a more comprehensive class,
23
Figure 3.8: Class Sensor (a subclass of Device) and the methods implemented.
that should be subclassed for more specific devices. This class contains the IP and port of the video
stream, the IP and port of the control module and the width and height of the video resolution. If the
device has no control module, the control’s IP and port should not be set. For a stationary camera,
the implemented subclass could store its GPS location and the direction where is the camera pointing.
Figure 3.9 illustrates abstract class VideoDevice.
Figure 3.9: Abstract class VideoDevice is the super class of all video-related implemented classes.
Camera
As the designation suggests, this class represents the characteristics of a real web camera. This
class should be applied to every web camera that offers the consumer the ability to purchase its video
stream and to control the camera. This class should store additional information. For example, if a
camera has specific characteristics, such as, pan and tilt movement, zoom, and IR sensor or night
vision capabilities. Figure 3.10 presents class Camera.
Catalog
For a customer to find a specific temperature sensor or a controllable web camera, every item avail-
able in the marketplace must be stored in a list to allow an item to be removed, searched and ordered.
Catalog is an abstract class that contains a list of objects and a set of abstract methods. Hence, rep-
24
Figure 3.10: Class Camera implemented considering the functionalities of a controllable camera.
resents a catalog that a customer can use to find a sensor. Considering that there will be two types
of items in the marketplace, data and devices, each type should have its own catalog, i.e., a specific
implementation of Catalog. These subclasses are the final catalog where a customer will search for a
specific sensor or where the marketplace will store a new device submitted by a supplier. Figure 3.11
illustrates abstract class Catalog and its relationship to DeviceCatalog and DataCatalog classes.
DeviceCatalog
DeviceCatalog is a subclass of Catalog. It stores a list of Devices thus, is a catalog of Devices.
In addition, this subclass overrides some methods of Catalog, implementing the algorithms to find a
specific Device and retrieve it or add a new one to the catalog. As previously discussed, Sensor (a
temperature sensor) and Camera (a web camera) are subclasses of Device, thus the list may have
objects of different classes, each representing a type of sensing device. In present-day marketplaces
and stores, a customer searches for an item based on its characteristics. The current implementation of
the DeviceCatalog does not allow a customer to search for a web camera considering its characteristics,
if it has a specific video resolution, IR filter or is capable of zooming. Instead, the customer can only
search for a device via its UUID. The same issue occurs regarding the other sensing devices: a customer
cannot search for a temperature sensor considering its characteristics, if it measures in Celsius or Kelvin,
with a precision of a tenth of unit, for example.
DataCatalog
DataCatalog is very similar to the DeviceCatalog class. It is a sub class of Catalog and stores a list
of Sensors. However, this catalog is intended to store information of the buyer’s virtual device from AWS
IoT. This catalog does not store purchasable devices or data. On the contrary, when data is bought,
the marketplace will search through this catalog to find the buyer’s virtual device, so that the buyer can
receive the data. In the DataCatalog each Sensor is searched via its owner ID.
Product
The catalog lists all the devices in the marketplace, independently of the type of sensor, the cost or
if it is available for purchase. Nevertheless, a customer may temporarily choose not to sell its sensor,
making it unavailable to other customers. In addition, when one customer purchases the control of a web
camera during a time interval. It is important to guarantee that there is no collision with other customer’s
period. Product is a class that addresses these issues. Every item in the DeviceCatalog will have a
Product associated to it. Product stores the item’s UUID, price, availability and type, for example. To
25
Figure 3.11: Abstract class Catalog and the subclasses DeviceCatalog and DataCatalog.
verify if there will be any collision when purchasing the control of a web camera, or any other type of
controllable device, Product keep a list of timestamps, representing the start and the end of each time
interval. To confirm that there is no collision, a method to compare these time intervals is implemented.
As discussed before, the UUID allows for unique identification of each Product. Figure 3.12 presents
class Product.
Figure 3.12: Class Product implemented to act as a item in a stock inventory.
Stock
As the designation implies, this class represents the stock of products presented in the marketplace’s
catalog. Stock stores a list of Products and implements several methods to manage it. These methods
allow to add, retrieve or cancel (make a Product unavailable) a Product. One other method, checks if
a Product is in the list and invokes the time validation method if necessary. Figure 3.13 presents class
Stock.
In present-day marketplaces and stores, a catalog presents to the customer all (or a set of) the items
26
on sale. The stock is the inventory or record of the items in that catalog. One customer selects an
item in the catalog. The marketplace checks if there are items available to sell in the inventory. If there
are, the order is successful. If not, the customer cannot purchase the selected item. The Catalog and
Stock implemented in this marketplace were based on the catalog and stock described above. When a
customer attempts to purchase an item from the DeviceCatalog, the Marketplace will verify if the item
exists in the Stock and if it is available for purchase. If it is available, the order proceeds to the next step.
Figure 3.13: Class Stock developed to interact with a Product.
Customer
As already discussed, customer is a person or entity. A customer can be a supplier, a consumer or
both. The same customer can purchase data from a temperature sensor and sell to another customer the
ability to control his camera. Customer is an abstract class containing all the information of a customer.
This class stores a Wallet, an Account, a Transaction list and a Product list. In addition, it stores the
customer’s first and last name, and a unique ID. As presented above, Wallet represents the virtual
wallet of a customer, and, for every successful exchange the Marketplace will interact with this object to
perform the respective operation. This interaction is possible through the methods made available by the
Customer class. Customer performs a crucial role in a future marketplace. If by any means this class or
a similar is not correctly designed or implemented, the marketplace will most certainly fail or have serious
issues. Those issues could be security issues or simple performance faults. One customer could not
purchase data, because the system did not store all the necessary information, for example. Figure 3.14
illustrates the relationship between Customer and its subclasses. Buyer and Seller classes inherit the
implemented methods in Customer, thus being able to interact with Account, Product, Transaction and
Wallet classes.
Buyer
Buyer class is one of the two classes that implement Customer. This class, as the designation
suggests, is a class for all the buyers in marketplace. All the specific methods for buyers should be
implemented in this class.
Seller
Seller is the second implementation of Customer. This class represents each seller in marketplace.
All the specific methods for sellers should be implemented in this class.
27
Figure 3.14: Abstract class Customer and the subclasses Buyer and Seller.
ServerInterface
When a customer intends to purchase data or control of a sensing device, the Marketplace will
perform a set of standard verifications to the order itself, to the customer and to the sensing device.
After this, it attempts to communicate with the server or broker of the corresponding device. The class
ServerInterface acts as a representative of the Marketplace. This class implements a set of methods that
communicate with the outside via HTTP Requests. A URL path and a JSON object characterize each
request. The URL path is the URL of the server or database and the JSON object is the information to be
sent. The implemented methods can return one of the following: a JSON object, if the customer intends
to purchase the control of a sensing device or data stored in a database; a string, if the customer intends
to purchase data from a sensor, over a period of time. Figure 3.15 presents class ServerInterface and
the methods implemented to interact with the broker module.
Figure 3.15: Class ServerInterface is the intermediary between the Marketplace and the broker module.
Marketplace
As described before, a marketplace is an environment where one customer is able to exchange a
set of data or the control of a sensing device for currency. This characterization is valid considering
the scope of this project, whereupon the concept of Sensor Data Market for the Internet of Things is
analysed.
The classes, interfaces, variables and lists presented before are, essentially, the pieces of the mar-
ketplace. How one piece communicates and interacts with another is implemented in the marketplace
itself. All the pieces are put together in the class Marketplace. This class contains every variable and
implements every method required to run the marketplace. In greater detail, the concept of marketplace
presented here requires five elements:
• Customers, persons or entities that purchase and sell data;
28
• Sensing devices, to collect data or to be controlled;
• Catalogs, to store and present sensing devices to the customers;
• Stock, the inventory of each sensing device;
• Broker, the bridge between the sensing device and the customer/marketplace.
If one of these elements is not present, the marketplace will not succeed. Without sensing devices,
there is nothing to offer to the customers, no data to be collected and no device to be controlled. If it is
not possible to organize and store devices in a presentable and searchable way, the customers will never
find the device or data they wish. If a stock/inventory is not implemented, although it is possible to find
sensing devices (through the catalog), when a customer attempts to purchase data, the device might
be offline or not available at that moment. Thus, the client would pay for something that would never
receive. Even worse, the device, a controllable device, could already be in use by another customer,
resulting in two customers trying to control the same device at the same time. Without a communication
path between the marketplace and the sensing devices, the customer will never receive data collected
by a sensor or will never be able to control a device. Lastly, if there are no customers, the marketplace
will fail, for obvious reasons.
Analysing with greater detail the methods developed in the Marketplace class it is possible to ob-
serve a similar structure to the one previously presented. Firstly, a set of procedures to create a new
customer, Buyer or Seller, a new Device and a new Product. A Device will belong to a Customer and
a Product will be associated to a Device. Next, the methods to interact with the list of Customers and
with the catalogs, DataCatalog and DeviceCatalog are implemented. It is possible to add, retrieve or
delete elements, find a specific element or to present the catalogs. Considering the Stock’s structure
and purpose, Marketplace also implements a group of procedures to interact with the Stock. It is pos-
sible to add and retrieve a Product, verify if it is available to purchase or mark it as unavailable. With
the methods and variables already presented it is possible to create a marketplace. However, two cus-
tomers cannot exchange data for currency, yet. In order to exchange data or control of a sensing device
between customers, the Marketplace class implements specific methods and protocols via the ServerIn-
terface class. Figure 3.16 presents a purchase’s flowchart. The realized purchase methods require the
following: a UUID, the device’s unique identifier; two Customers, the Buyer and the Seller; an instant of
time (timestamp) or a period of time. As Figure 3.16 indicates, the Marketplace starts by verifying if the
UUID matches with an existing UUID in the DeviceCatalog and if the Product is available (in the Stock).
After these verifications, the Marketplace will attempt to retrieve the Seller, owner of the device that the
Buyer wishes, to proceed with the purchase order. If the Buyer has enough funds to fulfil the purchase
order, the procedure will create a sale specification, or sale spec, and send it to the broker. Depending
on the broker’s reply, the Marketplace will add a Transaction to both parties, Buyer and Seller, modify
their balance accordingly, and update the Product’s characteristics. Lastly, the Marketplace will present
the outcome of the purchase operation: a URL or the purchased data if successful, an error message
otherwise. Sale specification or sale spec, is the designation of a set of data crucial to perform any ex-
change in the marketplace. This information is in JSON format and is sent with the HTTP request to the
29
broker or to the database. Considering the item being purchased, the sale spec may contain the seller’s
name and topic, the buyer’s topic (associated with the AWS IoT) and the time interval. For another type
of item, it may contain the IP address of the video stream and of the control module, amongst other
information. The sale spec is critical. Without the information it contains, the broker will not be able to
connect buyers to sensors or send data to the buyer. A set of methods is implemented to create the
sale spec, send it to the broker via a ServerInterface, and handle the broker’s response. As previously
discussed, this information is sent in the HTTP request to the broker. An instance of the ServerInterface
is responsible for creating the request, sending it and receiving the reply.
Start
UUID,
Customer,
Timestamps
hasDeviceCatalog
Success
True
Fail
hasProductStock
True
True
purchase
False
UUID,
Buyer,
Seller,
Timestamps
Figure 3.16: Flowchart of a purchase, either for a Simple Sensor or for a Rich Sensor.
30
Figure 3.17 and Figure 3.18 present the exchange of messages, in a timeline model, between each
module. The marketplace must deliver the sale spec of the purchase to the broker. Without this step, the
broker cannot setup or establish a connection between the sensing device and the buyer. Figure 3.16
illustrates that when the marketplace finishes processing a purchase order (with success) it will send an
HTTP request to the broker. The broker handles the request and sends a reply. Then, the marketplace
will show the customer the response: a URL, for a video transaction and a setup reply message for a
data transaction.
:Actor:Actor :MarketPlace:MarketPlace :Broker:Broker
1: Purchase
2: Show Response
1.1: HTTP POST Request
1.2: HTTP Response
Figure 3.17: Messages exchanged between each participant regarding a data transaction.
:Actor:Actor :MarketPlace:MarketPlace :Broker:Broker
1: Purchase
2: Show URL
1.1: HTTP POST Request
1.2: HTTP Response
Figure 3.18: Messages exchanged between each participant regarding a video transaction.
UML Specification
Figure 3.19 presents the complete UML of the marketplace implementation. It is possible to observe
the inheritance mechanism, from the abstract class Device to the sub class Camera, for instance. The
creation of abstract classes and the usage of inheritance ensures the possibility to integrate new types
of sensing devices and data in future works. If a sensor measures the humidity, the air pressure, or even
its geo-localization, Data can be sub-classed/extended to address the new data type. The same can
occur with the Device abstract class. If one wishes to add a new device, such as, a robot with a video
module and a control module, one can simply create a new class that extends Device or VideoDevice.
Figure 3.19 also illustrates how classes interact with each other, highlighting how the class Marketplace
interacts with the other classes. Marketplace acts as the hub, thus, it creates, stores and manages the
other components.
31
Figure 3.19: UML diagram of marketplace classes relationships.
3.3 Simple Sensor Products
The smartphone is a great example of a device containing multiple sensors, such as, temperature sen-
sor, GPS, accelerometer, microphone. Furthermore, a smartphone can be looked at as a device with
limited power resources and processing capability. It is possible to compare this equipment with an ag-
gregate of simple sensors. Considering a network of temperature sensors with a GSM/WiFi module and
a battery deployed throughout a city, one could create a temperature map, for instance. Every smart-
phone connected to the marketplace is also able to contribute, by taking measurements and sending
them.
Specification
Considering the two types of sensing devices defined in this project and their specific characteristics,
as discussed in Section 2.1.1, the development and implementation of the broker module was also
divided. This section presents the architecture and implementation for the simple sensors. In Figure 3.1
it is possible to observe that there is no direct communication path between the marketplace module
and the device module. The broker acts as the intermediary, thus ensuring a clear separation from
customers/Marketplace and the sensing devices. Moreover, the broker module is solely responsible
for creating and managing the communication link between the other modules. The broker’s module
implementation for sensing devices is composed of:
• AWS IoT platform, the foundation of the communication infrastructure;
• Data Broker, the entry point of communication;
• Data Handler (Simple mode), responsible for managing the customer-device connection in simple
data transactions;
• Data Handler (Long mode), responsible for managing the customer-device connection in long data
transactions.
32
Furthermore, the marketplace module accesses the purchased data via the Buyer Client, a AWS
IoT client, and the device module implements a gateway to establish a communication link between the
sensor and the AWS IoT platform.
As discussed in Section 2, the AWS IoT platform was selected, to be the infrastructure behind
the implementation for the simple sensors. The platform was designed around the MQTT protocol,
a lightweight publish and subscribe messaging protocol. Through AWS IoT platform each owner of a
simple sensor has a Thing, a virtual representation of the real sensor. However, a Thing can also be a
logical entity, i.e., it may not represent a real device. This characteristic is particularly useful to represent
a buyer, as it will be explained below. Every simple sensor is connected to the AWS IoT platform via
its Thing and publishes data to a topic, designated as topic S. Additionally, it is possible to associate
a Shadow to a Thing. The Shadow can be used to send control messages to the sensor, also via the
AWS IoT. Consider the following: a customer/buyer purchases all data measurements from a sensor for
a period of time; All data messages from the sensor are published to topic S and, since, the customer
cannot communicate directly with the sensor, the customer does not have access to topic S. As pre-
sented above, a Thing can also be a logical entity. By associating a Thing to the customer and creating
a topic, topic B, it is now possible for the buyer to receive all the data measurements published in topic S
without requiring access to that topic. Sensing devices publish data to their topic (topic S). Every buyer
has a virtual device and a topic (topic B) associated to that virtual device. The broker, namely the Data
Handler, will subscribe to topic S and copy the messages to topic B. This mechanism will be discussed
in greater detail below.
AWS IoT developed and published a SDK (Software Development Kit) meant to assist in building
applications that use the AWS IoT platform. The SDK [19] allowed to integrate the platform directly
in the broker module, namely in the Data Handlers and the Buyer Client. These components were
developed in Python [20]. Furthermore, the Data Broker is an HTTP server also developed in Python.
It assigns to each valid data request or control order a Data Handler. The Data Handlers are able to
publish messages, subscribe to topics and use a device’s Shadow, through the SDK.
Figure 3.20 exemplifies the sequence of interactions between each participant in a data purchase.
The customer initiates the whole process by making a purchase in the Marketplace. The Marketplace
communicates with the Data Broker, which in turn creates an instance of a Data Handler and replies back
to the Marketplace. At this point, the Data Handler subscribes to topic S and publishes any received
message to topic B. The Buyer Client allows the customer to receive every data measurement made by
the sensor. It should be highlighted the role of AWS IoT platform in steps 5 and 6. The platform provides
its communication infrastructure allowing the Data Handler to interact with the device and the client.
Device
As discussed above, the simple sensor implemented is a temperature sensor. This sensor is in fact,
the CPU temperature sensor of a Raspberry Pi. The Raspberry Pi is connected to the Internet in order
to be able to send the data it measures and to receive instructions. The described setup aims to simulate
a sensing device composed of a temperature sensor and a GSM/WiFi module. As pointed above, the
33
MarketPlace
2
4
Data Handler
3 5
1
Handler
1
2
3
Data Broker
6
Figure 3.20: Sequence of interactions between each participant. The AWS IoT platform is present insteps 5 and 6.
AWS’s library implemented communicates via the MQTT protocol. The device’s communication module
or gateway was also implemented by means of the same AWS library, thus ensuring total compatibility.
The MQTT was selected due to its characteristics regarding the energy and processing constraints of a
remote sensing device or a device not connected to the grid. The software implemented in the device
executes, recurrently, two tasks: temperature measurement and data (message) publication. Also, right
before publishing a new message, the device verifies if there is a new message for itself, such as a
restart command. If there is, it processes that message.
The device could not communicate directly with the AWS IoT platform using the MQTT protocol, as it
was not designed to perform that task. Thus, it was necessary to implement a script to allow the device
to interact with the AWS IoT platform. The software was developed in Python and acts as a gateway.
The device was configured to measure the CPU’s temperature periodically and is also capable of making
a measurement upon request, for instance, when the Data Handler sends a data request to the device
via the device’s Shadow.
Data Broker
The broker module is the intermediary between customer and device. The Data Broker is an HTTP
server designed to process every data transaction sent from the marketplace module. Two modes of
data transaction were defined, namely simple and long. A customer/owner wishes to send a command
or control message to a sensor, for instance a customer requests a device to switch on/off a LED or
the owner intends to remotely reboot his sensor. These situations are considered to be simple data
transactions. A customer requests a data measurement at a specific time-instant. For instance, a client
wants to collect the measured temperature in a particular location at 18h00. Again, this situation is
considered to be a simple data transaction, namely a simple data purchase. The presented scenarios
are characterized as being an isolated data request or order. By opposition, a long data transaction can
be described as a request that occurs for a time period. A customer wishes to collect all measurements
from a sensor starting at 13h00 and ending at 16h00, for instance. Hence, is it important to process
each transaction type separately. Thus, the HTTP server maps two URIs. As discussed above, the
34
salespec is adapted to each type of data transaction: for a simple transaction it contains the seller’s
device name, the seller’s device topic, the buyer’s topic and the timestamp for when the order/data
must be processed/measured; for a long transaction it is composed of the same first three data fields
of the simple transaction, plus the start time and the end time of the time-window associated to the
data purchase. An HTTP request containing the salespec is sent to the data broker only when the
Marketplace validates and concludes a data transaction. If the request is destined to one of the mapped
URIs, the broker attempts to process the salespec. The specification of a transaction is sent in the HTTP
request as a JSON object. Upon success the broker will create a sub-process of a Handler. Handler is
a Python script responsible for managing the actual transaction of measured data or control messages.
Finally, the Data Broker will resume and wait for a new HTTP request.
Data Handler (Simple mode)
As previously stated, AWS IoT created a SDK in Python enabling a direct communication with the
platform’s broker whilst using the MQTT protocol. The Data Handler for simple transactions was de-
signed to send a control message to a device or to request a data measurement and deliver the data
to the customer. According to the MQTT protocol, the sensor publishes data or messages in a topic,
designated as topic S. If someone subscribes to topic S, he/she will receive every message published by
the sensor. Consider the following scenario: a customer requests a data measurement at 16h00. The
marketplace could simply send to the customer the sensor’s topic, topic S. The client has now full access
to topic S. To ensure that the customer only receives the purchased data, AWS IoT would have to man-
age the subscriptions to topic S and cancel any subscription as soon as a client receives the purchased
data. This solution was not implemented, since it is not framed in the scope of this project. One possible
solution would be to delegate the responsibility to the sensing device. The device would be responsible
for creating a new topic for each customer and destroy it after completing the request. This scenario is
highly discouraged. The sensing device should never be responsible for managing data transactions.
Hence, the broker module should handle every data transaction, ensuring that the costumer can never
interact directly with a device, the sale specification of each purchase/request is enforced, and the de-
vice is only required to publish data measurements. The Data Handler is the intermediary between the
sensor and the customer. Bearing in mind the scenario described above, in which a customer requests
a data measurement: The Data Handler receives the sale specification from the Data Broker. After-
wards, it subscribes to topic S and sends the data measurement request or the control message to the
sensor via the sensor’s shadow topic, as previously discussed. Upon receiving a message from topic
S, the Data Handler verifies if the time of reception of the message satisfies a predetermined deviation
margin. If successful, the message is published to the buyer’s topic, topic B. Otherwise, when the instant
of reception surpasses the upper limit, an error message is published to topic B. However, if it does not
reach the lower limit, the Data Handler remains listening to topic S. Finally, after publishing to topic B the
Handler terminates itself.
Figure 3.21 illustrates the workflow for a Data Handler’s instance. It should be highlighted that the
Data Handler “sleeps” until the current time reaches the timestamp of the sale specification minus a
35
setup time of four seconds (this time interval was chosen based on visual inspection of the time required
to establish all connections). This difference is essential to create the necessary connections to AWS
IoT and ensuring the Data Handler is ready to send the data request or control message on time.
Start
Read Arguments
Sleep until current
time is the
scheduled time
Create a thread
Connect to AWS IoT
(Seller) & subscribe
topic S
Create a threadConnect to AWS IoT
(Device Shadow)
Update the Shadow
New message?
No
End
Yes
Connect to AWS IoT
(Buyer)
Send message to
topic B & signal to
exit
End
Signal?
0
1
End
Figure 3.21: Workflow of a Data Handler for a simple data transaction.
Data Handler (Long mode)
Similarly, to the Data Handler for simple transactions, the Data Handler for long transactions was
developed in Python and implements the AWS IoT SDK. This Data Handler was designed to subscribe
the seller’s sensor topic and publish any received message to the buyer’s topic. As discussed above,
to request AWS IoT to implement this very specific solution would be outside this project’s scope. As-
signing to the sensing device the responsibility to manage each publish and subscribe connection is
also highly discouraged. From a hardware perspective it would be a very demanding task. Moreover,
36
bearing in mind the architecture of the marketplace this solution would be inadequate. Consider the
following scenario: a customer wishes to receive every data measurement between 16h00 and 18h00.
The marketplace could simply send to the customer the sensor’s topic, topic S. However, as already
discussed there should be clear separation between customer and sensor. The Data Handler acts as
the intermediary in the above scenario. Thus, ensuring the customer does not have full access to topic S
nor to the sensor. Furthermore, the Data Handler is responsible to send every message published by the
sensor during the time-window sent by the Marketplace. According with the presented scenario, when
some customer purchases data measurements during a period of time, the Data Handler subscribes to
topic S. Upon receiving a new message, the Data Handler will publish that message to the buyer’s topic,
topic B. This interaction terminates when the current time surpasses the end time of the purchase. At
this moment, the Data Handler shuts down and the buyer will no longer receive messages, in topic B,
from sensor.
Figure 3.22 illustrates the workflow for a Data Handler’s instance. The Data Broker starts a Data
Handler and sends all the required information. The Data Handler initiates, processes the information
and sleeps until the schedule time. Upon resuming, it creates two AWS IoT clients, one in order to
subscribe to topic S and the second to publish to topic B. This Data Handler is alive for the duration of
the time-window purchased by the customer. When the end of the time-window is reached, the Handler
terminates itself. It should be highlighted that the Data Handler “sleeps” until the current time reaches
the start time of the sale specification minus a setup time, ten seconds. This difference is essential
to create the necessary connections to AWS IoT and ensuring the Data Handler is ready to publish
new messages to topic B. The setup time is longer, as this Data Handler creates to AWS IoT clients
consecutively, whereas the Data Handler, for simple transactions, used a thread to create one of the
clients. Thus, reducing the setup time.
Buyer Client
As previously stated, the Data Handler will publish messages from topic S to topic B. Nevertheless,
the customer is not able to directly read the messages in topic B, since, he/she does not have access to
the AWS IoT platform. One could argue that a valid solution is to give access to the AWS IoT platform.
Although a possible suggestion, this project addresses this situation in a different manner. Once again,
a direct-access policy is avoided by creating a broker, in this case between topic B and the customer.
The intermediary subscribes to topic B and stores every received message in a text file. This file could
be read by the Marketplace and shown to the customer. A different approach would be to store each
message in a database accessible to the Marketplace. Thus, providing close to real-time access to the
measurements collected by the sensor.
Data Stream
Figure 3.23 presents the simplest scenario of a data purchase. Topic S, the sensor’s topic, is sub-
scribed by a single Data Handler. The Data Handler belongs to the customer that purchased the data
measurements and will publish every message from topic S to topic B, the buyer’s topic. As previously
discussed, the customer is never directly connected to the device. The broker, specifically, the Handler
37
creates the link between sensor and customer by subscribing topic S and publishing in topic B. Figure
3.24 illustrates a scenario where the sensor’s measurements are consumed by multiples customers.
Each customer has a dedicated Data Handler, represented by a single Data Handler, subscribing the
same topic S. Nevertheless, each Handler publishes messages to its buyer’s topic. The sensing de-
vice will publish to only one topic, topic S, independently of the number of customers that purchased
access to the data measurements. Each Customer-Handler and Handler-Sensor connection is estab-
lished through the AWS IoT. The platform performs a crucial role in the scenarios illustrated by Figure
3.23 and Figure 3.24.
Start
Read Arguments
Sleep until current
time is the
scheduled time
Create a thread
Connect to AWS IoT
(Seller) & subscribe
topic S
New message?
No
Yes
Publish message to
topic B
Connect to AWS IoT
(Buyer)
EndTime>CurrentTime
No
End
Yes
Figure 3.22: Workflow of a Data Handler for a long data transaction.
38
Data Handler
Gateway
MQTT MQTT
Data Handler
Gateway
MQTT
MQTT
MQTT
MQTT
MQTT
MQTT
Figure 3.23: Single costumer interaction with a sensor.
Data Handler
Gateway
MQTT MQTT
Data Handler
Gateway
MQTT
MQTT
MQTT
MQTT
MQTT
MQTT
Figure 3.24: Multiple costumers consuming the data stream.
39
3.4 Rich Sensor Products
Throughout time, technological advances have enabled the construction and development of more so-
phisticated devices. Starting from simple radio frequency-controlled robots, such as radio-controlled
cars, to fully operational UAV (unmanned aerial vehicle) controlled via a satellite-link. As discussed in
Section 2.1.2, a rich sensor is a sensing device that, amongst other characteristics, can be controlled
via the Internet. This type of devices can create a live video stream, move themselves, etc. The rich
sensor implemented in this work is an IP controllable web camera. This device is both controllable and
able to create a video stream. Not integrating this type of sensing devices in a marketplace may have a
serious impact in its success, due to the ever-growing number of these rich sensors in the world.
Specification
As shown in Section 3.1 the marketplace module does not communicate directly with the device module.
The agent responsible to connect those modules is the broker. Moreover, after a successful purchase
of a time-slot on a rich sensor, the customer will communicate with the sensor via the broker module.
With this module, the buyer can control the device or receive any type of data stream, may it be video or
sound. The broker module is composed of:
• Video Broker, the entry point of communication;
• Handler, responsible for managing the customer-device connection;
• IP/TCP Tunnel, the line of communication;
• Web page, responsible for presenting the video stream and control mechanism.
Video Broker is the element, in the broker’s module, visible to the marketplace module. It is respon-
sible for receiving every request, related to a rich sensor, created by the marketplace. In greater detail,
the Video Broker is an HTTP server developed in Java. As discussed in Section 3.1, the communication
between each module should be implemented following a RESTful pattern. Thus, the Video Broker was
implemented according with this pattern. It exposes a URL, used by marketplace to send an HTTP
POST request to the broker module; its operation does not vary with previous requests.
The customer (through a web page) cannot communicate directly with the sensing device. Firstly,
since it would be necessary to install a client-side software to validate/authenticate the customer, to
setup the connection with the sensing device and to translate the requests made to the sensing device.
Secondly, the management of the connection would be client-side, which is not recommended. There-
fore, it is perfectly reasonable to implement a server-side management agent to all the connections
between customer and device. This agent is the Handler. In detail, the Handler must create, maintain
and destroy an IP/TCP tunnel, for the video stream, and interpret and send (to the device) any control
request issued by the customer.
Figure 3.25 presents a brief diagram for a new request. This diagram illustrates how each participant
acts and with whom it communicates, assuming each request is valid. When a customer purchases
40
a slot of time, the Marketplace will send an HTTP request to the broker, specifically the Video Broker.
The Video Broker will create an instance of Handler and send a reply so that the Marketplace is able
to create the URL. Once the customer opens the URL, the Handler created before will setup the video
IP/TCP tunnel. At this point the customer is able to watch the video stream. To control the web camera
the customer should press the control buttons. Each control button press will send an HTTP request to
the Handler, that is processed and relayed to the camera.
:Actor:Actor :MarketPlace:MarketPlace :Master:Master
1: Purchase
2: Show URL
:Handler:Handler
1.1: HTTP POST Request
1.1.1: Create the Handler
1.2: Sleep1.3: Start server
1.3: HTTP Response
ViewView
3: Open URL
3.1: HTTP GET Request
3.1.1: Create Video tunnel
3.2: Video Stream
4: Control Button
4.1: HTTP GET Request
4.1.1: Process the request
4.2: HTTP Response
Figure 3.25: Message exchange - Purchase of video & control.
Figure 3.26 exemplifies the sequence of interactions between each participant in a video and control
purchase. The customer initiates the whole process by making a purchase in the Marketplace. The
Marketplace will communicate with the Video Broker, which in turn creates an instance of the class
Video Handler and replies back to the Marketplace. At this point, the customer will receive the URL on
which he/she will be able to watch the video stream and control the device. When the customer opens
the URL (inside the purchased time window), the Handler acknowledges that connection and interacts
with the video device. Step 6 and step 7 represent the continuous interaction between customer and
video device.
41
Handler
Video
CAM CONTROL
Video
HTTP
Handler
Video
CAM CONTROL
Video
HTTP
Video
Video
MarketPlace
2
4
Handler
3 7
1
Master
5
6
Figure 3.26: Sequence of interactions between each participant in a video & control purchase.
Device
As presented above, the rich sensor implemented is a IP controllable web camera. This device can
create a video stream and associate it to an IP and a port by creating a server. As it is controllable, the
control module has an IP and port. The separation between video and control is essential. Ensuring
a division between the two prevents a customer, which only purchased the visualisation and not the
control, to use the video IP/TCP connection to attempt to control the camera. The device’s control
module is a program developed in C. This program manages a TCP socket to receive control messages
from the Handler’s server, translate them and move the camera.
Video Broker
The broker module is the connecting link between customer and device. The Video Broker, is a class
that creates the HTTP server exposed to the Marketplace. When invoked, it associates the HTTP server
to an inner class designated Register. This class handles each HTTP request received by the server.
After determining if the request itself is valid, it will process the salespec sent by the marketplace. The
salespec is a JSON object with a predefined structure and content. If the JSON data inside the HTTP
request received by the server does not comply with the established structure or is missing any field,
the operation will stop and fail. The specification contains the following fields: the device’s UUID; the
video stream IP; the video stream port; the device’s control module IP; the device’s control module port;
the start time; the end time; an authentication string or hash. The salespec’s structure was designed
considering the device implemented. Different devices with different characteristics could require a new
salespec structure. Once processing is finished, the broker will create a file to store all the information
from the specification, create an instance of Handler and send a reply to the marketplace. The reply
contains the port associated to the HTTP server of the Handler previously created.
Figure 3.27 illustrates the workflow for an instance of Video Broker. Firstly, maps a URI path in the
HTTP server and binds it to a new object of Register. After that, it starts the HTTP server. Each valid
HTTP received request is processed. If successful, an instance of Handler will be created, otherwise
only a reply will be sent to the origin of the request. The HTTP server of the Video Broker will remain
alive until it is manually terminated.
42
Start
Map a URI path
and set an
HTTPHandler
New HTTP
Request?
Yes
Is the URI path
mapped?
Yes
Process the
Request
Create a
Handler’s
instance
Fail
Reply
End request
processing
No
No
Success
Start HTTP server
Figure 3.27: Workflow of the Video Broker.
Video Handler
Handler is a class that implements all the methods and a set of inner classes required to interact with
the customer and the device. When invoked the Handler will read the stored file created by the Video
Broker. After obtaining the necessary information, it will sleep until the beginning of the time window
bought by the customer. Once resumed, the program creates an HTTP server and waits for a request
from the customer. This server will not only process each control request sent by the customer, but
also manage the IP/TCP connections for the video stream and to control the device. The customer, by
opening the web page, activates a mechanism that sends an HTTP request to the Handler’s server. This
request triggers the creation of a connection (IP/TCP tunnel) between the customer’s address and the
video stream address. This connection acts as a link. From the customer’s perspective the address of
the video stream is the Handler’s server address, thus ensuring that the video stream address is never
43
shown to the customers. Once a customer sends a control request, via one of the buttons from the web
page, the server will authenticate the origin of the request. Upon a positive authentication, the request
is parsed and sent to the device’s control module. When the current time reaches the end of the time
window, each Handler’s instance will terminate the video IP/TCP tunnel, close the connection to the
control module, stop the HTTP server and terminate itself. Hence, each customer will no longer be able
to view the video stream nor control the device.
Figure 3.28 illustrates the workflow for an instance of a Data Handler.
Start
New HTTP
Request?
Yes
Is the URI path
mapped?
Yes
Process the
Request
If valid, create
video
connection
Reply
End request
processing
Map a set of URI
path to their
HTTPHandlers
Initialize the
Handler
Retrieve data
from stored
file
Sleep until current
time is start time
Start the HTTP
server
Set a timer for
termination
Curren t time
equal to end
time?
No
Yes
Close the video
connection
Close the control
connetion
End
No
If valid, send a
control
message
Video Request
Control Request
Figure 3.28: Workflow of a Data Handler instance.
44
Video Stream
As discussed in Section 3.1.4, for a customer to add a web camera or other video device with live
video stream capability, he/she must submit the IP and port of the video stream. Consider the following
scenario: The system is developed in such a way that every customer connects directly to the source
of the video stream, the web camera. The camera could have a WiFi or a cellular Internet connection.
Either way, if several customers purchase overlapping time-slots for this device, they would be viewing
the video stream simultaneously. This would cause a massive consumption of processing resources
and bandwidth in the device. Now, considering a different scenario: The system has an intermediary
between the customer and the video device. The customer will never be directly connected to the other
end of the stream and will never know the IP and port of the video device, assuming it is the same of the
video stream. Additionally, the number of customers, consuming the video stream, does not influence
the device’s resources. Considering the advantages and disadvantages of both scenarios, this project
implements the latter. The Handler acts as the intermediary, in the second scenario. Thus, employing a
separation of concerns, the customer simply opens a web page and sends HTTP requests; the Handler
manages both connections, supplies the processing resources and bandwidth. The IP/TCP tunnel is
once again implemented to avoid a direct connection between customer and device.
Figure 3.29 presents the simplest scenario of a video and control purchase. The camera’s video
stream is viewed by a single customer. In this situation, only one customer has interaction with the video
device. It is clearly visible the existence of two separate data streams: the video stream originated by
the camera and the camera control stream. As previously discussed, the customer is never directly
connected to the device. The broker, specifically the Handler, creates a IP/TCP tunnel to patch the
video stream to the customer. Also, it receives any HTTP (control) request and relays such request
to the device. Figure 3.30 illustrates a scenario where the camera’s video stream is consumed by two
or more customers. Nevertheless, only one customer is able to control the device. To the right of the
Handler only two data streams exist. On the other hand, to the left of the Handler it is possible to
observe four data streams. Of those, one is dedicated to control requests. The remaining are video
streams. The video device will create only one video stream, independently of the number of customers
that purchased that video stream. The Handler will, for each video customer (a customer that purchased
only the video stream), create a IP/TCP tunnel. Thus, the bandwidth required to transmit each stream
is allocated in the Handler’s network and not in the device’s network. Moreover, the device does not
need to create and process a video stream for each customer, therefore, not increasing the processing
requirements of the camera.
3.5 Summary
In this chapter we presented the architecture and each component of our solution. The Marketplace
is where costumers are able to share their devices and purchase sensor data or control of a device.
The Marketplace can be considered as the core of the solution. Rich and Simple Devices have specific
hardware and functional characteristics, as such, it was necessary to implement different approaches in
45
Handler
Video
CAM CONTROL
Video
HTTP
Handler
Video
CAM CONTROL
Video
HTTP
Video
Video
MarketPlace
2
4
Handler
3 6
1
Video Broker
5
Figure 3.29: Single costumer utilizing the controllable video camera.
order to create a structured communication channel between customer and device. Furthermore, within
the Simple Device development, simple and long types of data transactions required a dedicated design
and implementation. The AWS IoT platform was integrated into the implementation for Simple Devices
and acts as the foundation, enabling the communication between customers and devices. Rich Devices
can create a video stream and have a control mechanism, for example. These devices are considered
to be the more complex and demanding sensors and the implementation took into consideration the
fact that it is possible to have multiple customers consume the video stream, but only one being able to
control the sensor at any time.
46
Handler
Video
CAM CONTROL
Video
HTTP
Handler
Video
CAM CONTROL
Video
HTTP
Video
Video
MarketPlace
2
4
Handler
3 6
1
Video Broker
5
Figure 3.30: Multiple costumers consuming the video stream.
47
48
Chapter 4
Results and Evaluation
The platform was evaluated through quantitative and qualitative methods. The implementation of use
cases and the deployment of several sensors constitute the qualitative method. The platform must
be able to receive data from different types of sensors, ranging from simpler sensing devices (e.g.,
temperature sensors) to more complex sensors (e.g., video camera). The quantitative evaluation was
performed through various parameters, including the time delay between the instant from which the
sensor data is sent, to the access time of the consumer. Additionally, an evaluation was performed
regarding the overhead of finding the sensors and retrieving the sensor data from the marketplace
versus a direct interaction.
The hardware used in the following experiments is:
• Raspberry Pi 3 model B with Raspbian OS, a Quad Core 1.2GHz Broadcom BCM2837 CPU, 1GB
RAM and Internet access via WiFi;
• The temperature sensor (simple sensor) was the CPU temperature sensor of the Raspberry Pi 3
model B;
• The IP controllable Web Camera (rich sensor) was a Logitech Quickcam Orbit AF connected to
the Raspberry Pi 3 model B via USB;
• The IoT Marketplace was hosted on a Windows OS machine with a Quad Core Intel Core i5-
6300HQ CPU, 8GB RAM and Internet access via WiFi;
• The broker module (Video/Data broker and handler) was hosted on a regional (UK based) AWS
Elastic Compute Cloud virtual machine with Ubuntu OS, 1 core Intel Xeon virtual-CPU, 1GiB RAM
and Low-to-Moderate Network Performance.
4.1 Single Data Measurements
This experiment consists of scheduling three hundred measurements to the same sensor and receiving
the respective data. In greater detail, the assessment is divided in three series of one hundred mea-
surements, respectively. For each series, the Marketplace is set up with a pool of sensors, buyers and
49
sellers. One buyer and one sensor are selected and each data request is originated from the same buyer
to the same sensor. In the Marketplace, each request is generated with a one-second interval. However,
from the sensor’s perspective, two consecutive data measurements are separated by a sixteen second
interval, i.e., the sensor measures the temperature every sixteen seconds.
This test can be broken down into five stages. In the first stage, the Marketplace processes the
purchase request, checks if the sensor exists, if the scheduling is possible, and communicates the pur-
chase to the Broker (Data Broker). In the second stage, the Broker receives and stores the information
sent from the Marketplace and creates one Handler for this request. In the third stage, the Handler
resumes, sends a state update to the sensor via the Shadow Thing (AWS IoT). While sending the state
update, the Handler subscribes to the sensor’s topic. Fourth stage, the Handler detects a message (in
the subscribed topic) and replicates this message to the buyer’s topic. The fifth stage is the reception of
the sensor data. The buyer is already subscribed to its own topic in order to receive the sensor data.
To better evaluate the response time and performance (amongst other parameters) of the Broker and
the Handlers, the requests are created with one second interval from each other. Since every measure-
ment is set to occur in the future, the Handler will be inactive until such time is reached. Consider a set
of one hundred requests: a data measurement is taken every sixteen seconds; thus, the first measure-
ment is scheduled to happen sixteen seconds in the future and the second is scheduled to occur sixteen
seconds afterwards. After ninety-nine seconds all Handlers would have been created. A Handler is de-
stroyed shortly after taking a measurement. Therefore, in ninety-nine seconds, six Handlers have been
destroyed. So, at one point, there will be ninety-four instances with allocated memory simultaneously.
This setting approximates a possible real scenario in which a sensor is highly requested.
Figure 4.1 illustrates the number of allocated Handlers simultaneously, for the first three measure-
ments, and for how long. Since, the Handler is destroyed after publishing the message to the buyer’s
topic, it may stay alive for a few seconds more, regarding the scheduled instant.
0
|
#1
#2
#3
48
| 32
|
16
| asdasdat [seconds]
Figure 4.1: Number of allocated Handlers and their lifetime.
Figure 4.2 presents the time difference between the reception time and the scheduled time of each
measurement. These values have a high accuracy, since the Marketplace (which originates the schedul-
ing timestamp) and the client (which originates the reception timestamp) share the same clock: both
programs were running on the same machine. The values shown were collected throughout the third
series of one hundred measurements. In this series, the mean value for the time difference is 1.23 sec-
onds. While analysing the graph, it is possible to observe that a considerable number of samples are in
the interval from 1 to 1.5 seconds. Moreover, only six samples are above the 2 second mark, being that
the maximum in the series is 3.44 seconds.
50
0 20 40 60 80 100
0.5
1.0
1.5
2.0
2.5
3.0
3.5
Single Data Measurement
Index
Tim
e D
iffer
ence
(se
cond
s)
Figure 4.2: Temporal discrepancy between reception and schedule times.
To evaluate the performance of the proposed solution, it is crucial to assess the time required to
process each purchase order or each message replication. Figure 4.3 presents the proportion of pro-
cessing time to the total time, identifying its components using different colors. The Marketplace takes
more than fifty percent, the Handler more than forty percent and the Broker consumes no more than one
percent. These values are in accordance with what was stated in Section 3.3. The Marketplace must
validate all the information in the purchase order and communicate with the Broker. The Broker, upon
receiving a request from the Marketplace, stores the information and creates the Handler. The Handler
must communicate with the AWS IoT to activate the sensor, receive the measurement and then send a
copy to the client. Therefore, it is expected the Marketplace and Handler require more processing time
than the Broker.
Table 4.1 presents the overall statistical results calculated from the data collected throughout this
experiment.
Considering that the Marketplace only finishes processing when it receives a reply from the Broker
and the Client only receives a message after it is published by the Handler, one would expect that a
high processing time in the Broker or in the Handler could interfere with the processing time of the
Marketplace or delay a message to the Client. Nevertheless, this hypothesis is not verified. Analysing
the collected data, a higher time difference is not due to a higher processing time in the Handler. The
variation in the time difference, for instance, may be caused by the processing inside the AWS IoT
platform itself.
In Table 4.1, the mean time difference is 1.35 seconds. Picturing a real scenario where a client
51
0%
25%
50%
75%
100%
Average
Per
cent
age
Broker
Handler
MarketPlace
Time Usage
Figure 4.3: Average time usage for each participant regarding a Data transaction
schedules a measurement from a sensor, this value does not represent an issue to the feasibility of
this service. From a set of three hundred scheduled measurements, only one measurement was not
received by the client, which demonstrates a high success rate.
4.2 Long Data Measurements
This experiment is designed to analyse the performance of the implemented solution when it executes
one of the main operations. The procedure consists in purchasing real-time sensor data during a time
interval. In greater detail, the assessment is divided in three series of forty-five minutes, respectively.
For each series, the Marketplace is set up with a pool of sensors, buyers and sellers. Five buyers
Market Broker Handler Time Difference
Mean 1.05E-01 1.66E-03 8.68E-02 01.35
Standard Deviation 2.09E-02 5.20E-04 4.72E-03 00.63
95% Confidence Interval 2.37E-03 5.88E-05 5.34E-04 00.07
Maximum 3.38E-01 9.00E-03 1.00E-01 03.60
Minimum 9.87E-02 1.37E-03 8.00E-02 00.32
Table 4.1: Single Data Results: Processing time for each actor and overall time difference (seconds).
52
and one sensor are selected and each buyer generates a long data measurement request to the same
sensor. Those requests are created with a one-second interval. The sensor was configured to measure
and publish data to its topic every thirty seconds, even if there is no one listening. Considering that
the sensor does not have the ability to know if there are any subscribers to its topic, it will publish data
continuously. Each buyer purchases forty-five minutes of subscription to the sensor’s topic. Bearing in
mind the thirty-second halt at the sensor, it is expected that each buyer obtains ninety measurements.
To approximate this experiment to a potential real scenario, instead of having only one buyer listening to
the sensor’s topic, all five buyers are listening to the topic simultaneously.
This scenario can be broken down in four stages. In the first stage, the Marketplace processes the
purchase request by validating all the information from the buyer, the seller, the sensor and the request
itself. This stage culminates by sending the request to the Broker. In the second stage, the Broker
receives and stores the information, to subsequently create one Handler. In the third stage, the Handler
resumes its activity, then subscribes to the sensor’s topic and replicates any message to the buyer’s
topic. The last stage is dedicated to the reception of the sensor data through the subscription of the
buyer’s topic.
Throughout the experiment, each client, stores all received messages and generates a timestamp
for the instant of reception of those messages. As stated on Section 3.3, one message contains the
measured temperature and the timestamp of the measurement. The sensor is a Raspberry Pi, wherein
the time and date are configured with the default specifications. All five Clients were running on the
same machine, thus having the exact same clock. Bearing in mind that clients and sensor were on
different machines, therefore not being completely synchronized with each other, it is considered a
1 second maximum desynchronization. This 1 second is the maximum desynchronization between
the client’s clock and the sensor’s clock. Through visual inspection and some experimentation, the
desynchronization was always less than 1 second.
Even though it is not possible to guarantee total synchronization, Figure 4.4 presents the time differ-
ence between instant of reception (mean) and the instant of measurement.
These samples were collected during one of the three series from this experiment. It is possible to
observe that, on only two occasions, the delay between the measurement and reception is greater than
0.6 seconds, wherein the maximum time difference is 0.93 seconds. Also, more than 60 percent of the
samples are below the mean value. Even considering a desynchronization of 1 second (the sensor’s
clock is 1 second ahead of the actual time), the mean time difference, in that situation, is 1.47 seconds.
In a real scenario, a person intends to receive every measurement taken by the sensor in real-time, i.e.,
with a delay small enough to satisfy most applications.
Table 4.2 presents the overall statistical results calculated from the data collected throughout this
experiment.
As discussed in Section 4.2, the results do not support a causality relationship between a higher
processing time and subsequent delays when receiving a message. In this experiment, the mean pro-
cessing time of the Marketplace and the Broker are similar to the values from Table 4.1. However, the
Handler has a smaller processing time when compared to the previous experiment. In this experiment,
53
0 20 40 60 80
0.5
0.6
0.7
0.8
0.9
Long Data Measurement
Index
Tim
e D
iffer
ence
(se
cond
s)
Figure 4.4: Temporal discrepancy between reception and schedule times.
the Handler subscribes the sensor’s topic and publishes to the buyer’s topic. Furthermore, the Handler
must also alert the sensor. This additional task explains the extra processing time.
4.3 Video and Control
This experiment is designed to analyse the performance of the implemented solution when it executes
one of the key operations. The scenario consists in watching a video stream from a video camera whilst
having the ability to control the camera in real-time. Contrary to previous experiments, this assessment
is aimed at more complex sensing devices, e.g., devices capable of creating a video feed and accepting
control requests. This assessment is divided in three series of one minute, respectively. In each series,
Market Broker Handler Time Difference
Mean 1.06E-01 1.44E-03 8.67E-05 00.51
Standard Deviation 7.36E-03 7.10E-05 1.01E-05 00.10
95% Confidence Interval 3.73E-03 3.59E-05 5.41E-07 00.01
Maximum 1.30E-01 1.63E-03 1.68E-04 01.49
Minimum 9.98E-02 1.37E-03 7.20E-05 00.42
Table 4.2: Long Data Results: Processing time for each actor and overall time difference (seconds).
54
the Marketplace is set up with a pool of sensors, buyers and sellers. One buyer and one video camera
are selected to perform this test. The buyer generates one video and control request directed to the
selected camera. This request contains the start time and end time, among other specifications. When
the time limit is reached, the video stream ends and the buttons no longer control the video camera.
This test can be broken down in five stages. In the first stage, the Marketplace processes the
purchase order, through the validation of all the information contained in the order. This stage culminates
by sending the specifications to the Broker (Video Broker). In the second stage, the Broker receives and
stores this information, then, creates one Handler to manage the request. Subsequently, the Broker
communicates with the Marketplace to send crucial information. In the third stage, the Handler resumes
its activity and waits for instructions. In the fourth stage, the client opens the URL generated by the
Marketplace. This action triggers a response in the Handler, which creates the video tunnel and opens a
line of communication with the camera’s control module. Finally, in the fifth stage, the client presses the
control buttons on the webpage, the web browser communicates with the Handler, and the latter sends
a movement order to the camera’s control module.
To evaluate the performance several timestamps, processing times and messages were collected
throughout this experiment. In order to increase the accuracy of this experiment, the agent that gen-
erates the message (to control the camera) and sends it to the Handler is replaced by an application
designated Postman. This application issues the control order, an HTTP request, to the Handler. Post-
man runs a set of four HTTP requests seventeen times. Two consecutive requests are separated by
five-hundred milliseconds. This configuration results in a total of sixty-eight instructions during the one-
minute test period.
For every control order issued, the Postman determined the response time, i.e. the time required to
receive the HTTP reply from the Handler, considered to be the total time. The Handler calculated the time
necessary to create and communicate the control message to the control module of the video camera,
designated TCP, and estimated the total time required to process the request, designated Processing.
This time includes a simple authentication verification, the TCP time and the HTTP reply to the client
(Postman). Figure 4.5 presents the total time and the fractions due to each component. Approximately
sixty percent of the total time is due to the Communication component. This element is considered to
be the time spent outside the Handler and the Postman. Analysing the Handler, the TCP component is
responsible for nearly ninety-eight percent.
Table 4.3 presents the overall statistical results calculated from the data collected throughout this
experiment. The column Aggregate represents the total time registered in the Handler, i.e., the time
required by the TCP communication plus the processing time. The maximum values registered for
both the Postman and the Handler (Aggregate) occur in the first control order. According to [21] the
one-way latency for interactive video applications should be no more than 150 milliseconds. Hence,
two-way latency should be no more than 300 milliseconds. The latency measured corresponds to the
processing and reply of the control request, thus a two-way latency with a mean value of approximately
200 milliseconds. This result is acceptable, considering that the two-way latency should be less than
300 milliseconds. The mean value required for the Handler to fully process a request in approximately
55
0%
25%
50%
75%
100%
Total Time Handler Time
Average
Per
cent
age
Communication
Handler
Processing
TCP
Time Usage
Figure 4.5: Average time usage for each component regarding a Video and Control transaction.
Aggregate TCP Postman
Mean 5.73E-02 5.65E-02 2.01E-01
Standard Deviation 3.44E-03 1.61E-03 1.28E-02
95% Confidence Interval 4.72E-04 2.21E-04 1.76E-03
Maximum 8.23E-02 6.48E-02 2.84E-01
Minimum 5.39E-02 5.275E-02 1.93E-01
Table 4.3: Video and Control Results: Expended time in the TCP connection and overall two-way latency(seconds).
80 milliseconds, thus having some impact in the delay between instructing the camera to move and
observing that movement in the video stream.
4.4 Discussion
While developing and executing the two experiments for the Simple Device, the sensor sent more than
one thousand messages through the AWS IoT. Only five messages were not received by the client, a
0.005 fail ratio. Overall, the implemented solution proved to be: robust, when the customer attempted
to open the video URL in another computer or browser, the Video Handler detected this abnormal
behaviour and cancelled the request; effective, considering a 0.005 fail ratio in data transaction tests;
fast, regarding the low-delay visually observed during the video and control experiments and the delay
of the data transactions. The results are promising, the mean delay between measuring and receiving
is 1.54 seconds for the Single Measurements and 0.51 seconds (or 1.51 seconds, considering a 1
second desynchronization) for the Long Data Measurements. In the final experiment (Rich Device),
both the video stream and the control module proved to be responsive and accurate. The results were
56
obtained using a personal computer to run the Marketplace, a regional (UK based) virtual machine from
Amazon Web Services to host the broker module and Amazon Web Services IoT platform to support
the Device module. Configuring dedicated machines to host the marketplace module and the broker
module could improve the results. Optimizing AWS IoT platform to the specific requirements of the
Sensor Data Market may decrease the delay and the setup time necessary to establish the connections
to the platform. Nevertheless, the delay between receiving a data measurement and the instant of the
measurement itself may not be an issue for some applications or scenarios. For instance, a network of
constrained sensing devices (sensors with low processing power and limited connectivity) collects data
every two hours. The data is stored locally in the devices and uploaded once a day to a gateway. After
receiving the data measurements of all sensors, the gateway publishes it to the AWS IoT platform. In
this scenario the delay should not be an evaluation parameter.
57
58
Chapter 5
Conclusions
Through the research performed in Chapter 2 it was found there is no platform or service capable of
allowing two participants to exchange sensor data. We propose a Sensor Data Market for the Internet
of Things. This solution aims to be the intermediary between data suppliers and data consumers. Thus,
implementing a S2aaS model, where a sensor is available to any customer, in a service-like structure.
The Amazon Web Services Internet of Things platform was integrated in the proposed to solution,
due to its architecture and implemented protocols, such as MQTT and HTTP. The AWS IoT infrastructure
regarding the MQTT protocol is the foundation of the Simple Device implementation.
As discussed in Section 2.1, the devices were divided into two categories: Simple Device and Rich
Device. Two sets of tests were designed and conducted, one set for the Simple Device and the other for
the Rich Device. Nevertheless, both sets aimed to collect information in order to evaluate the proposed
solution.
Regarding the Simple Devices, the tests can be divided in two categories: Single Data tests, in which
a customer purchases an individual data measurement a at specific time instant; and Long Data tests, in
which a customer purchases all scheduled measurements during a window of time. Both tests scenarios
aim to replicate real circumstances. A Raspberry Pi 3 Model B was configured to collect the temperature
of its CPU, thus simulating an actual temperature sensor. Considering the Single Data tests, the results
show a mean time difference of 1.35 seconds between the schedule time and the instant of reception
and a maximum delay of 3.60 seconds. Thus, all received messages (ninety-nine out of one hundred)
were obtained in less than five seconds. The Long Data test results show a mean time difference of
0.51 seconds and a maximum of 1.49 seconds. Again, all received data were obtained in less than five
seconds.
Regarding the Rich Devices, the tests consisted in purchasing control of a sensing device during a
time interval. This test aims to replicate a real scenario in which a customer purchases both, the control
and video stream of the camera for period of time. A controllable camera was connected to a Raspberry
Pi 3 Model B, thus simulating an actual IP controllable camera. The results show mean two-way latency
of approximately two hundred milliseconds. Throughout this experiment there was no critical failures,
i.e., there was no interruption in the video stream nor regarding the control of the camera.
59
Considering the results presented above as well as the use cases described in Section 3.1, we
conclude that the platform designed and implemented in this work is a valid prototype/proof-of-concept
of a Sensor Data Market for the Internet of Things.
5.1 Future Work
Some points can be highlighted to improve the current solution:
• Study current business models and analyse possible trends to implement a successful business
model;
• The implemented solution cannot create a virtual representation of a real sensor in the AWS IoT
platform, namely a Thing. Upon the registration of a new device the platform must associate a
Thing to that device;
• As presented in Section 2, the security was considered to be an add-on in the scope of this solution.
Nevertheless, this issue is very important and should be addressed;
• Add privacy-preserving features, both to protect privacy of buyers and sellers, but also the privacy
of people potentially surveilled by such a system;
• Provide the customer with information regarding the performance of a device. Gather and provide
information, such as, the measurement’s accuracy or the time difference between the instant of
measurement and instant of reception of sensing data;
• Implementing more types of devices will widen the range of sensors that can be integrated in the
marketplace, thus strengthening a future Sensor Data Market;
• Design and implement a web interface to interact with the platform will enrich the current solution;
• Implement a database to store information of customers, accounts, transactions, devices. It could
also be used to store data collected from sensing devices;
• Creating and presenting new scenarios, such as the ones discussed in Section 1, may demon-
strate, even more, the importance and necessity of a Sensor Data Market. Furthermore, it may
attract clients and companies to join the platform;
• Consider the integration of a resell business model. For instance, a customer (customer X) pur-
chases all video images of a camera or all data measurements of a sensor in a time interval.
Afterwards, another customer (customer Y) intends to acquire the data sold to customer X. Thus,
the marketplace should verify if customer X wants to sell his data and perform a new data trans-
action. Furthermore, customer X purchases the right to all data measurements from a sensor in
a time window, i.e., no other customer is able to purchase data from that sensor during that pe-
riod directly from the owner of the sensor. Therefore, the marketplace should verify if customer X
60
wants to sell his data. If so, the data flow should be optimized to deliver data to every customer in
real-time;
• Integration with payment systems, both traditional currency, and crypto currencies.
61
62
Bibliography
[1] S. Haller, S. Karnouskos, and C. Schroth, The Internet of Things in an Enterprise Context.
Berlin, Heidelberg: Springer Berlin Heidelberg, 2009, pp. 14–28, Accessed: 3 July 2018. [Online].
Available: http://dx.doi.org/10.1007/978-3-642-00985-3 2
[2] C. Perera, A. Zaslavsky, C. H. Liu, M. Compton, P. Christen, and D. Georgakopoulos, “Sensor
search techniques for sensing as a service architecture for the internet of things,” IEEE Sensors
Journal, vol. 14, no. 2, pp. 406–420, Feb 2014.
[3] X. Sheng, J. Tang, X. Xiao, and G. Xue, “Sensing as a service: Challenges, solutions and future
directions,” IEEE Sensors Journal, vol. 13, no. 10, pp. 3733–3741, Oct 2013.
[4] P. J. Leach, T. Berners-Lee, J. C. Mogul, L. Masinter, R. T. Fielding, and J. Gettys,
“Hypertext transfer protocol – HTTP/1.1,” Accessed: 3 July 2018. [Online]. Available:
https://tools.ietf.org/html/rfc2616
[5] Z. Shelby, K. Hartke, and C. Bormann, “The Constrained Application Protocol (CoAP),” Accessed:
3 July 2018. [Online]. Available: https://tools.ietf.org/html/rfc7252
[6] T. Jaffey, “MQTT and CoAP, IoT Protocols,” Accessed: 3 July 2018. [Online]. Available:
https://eclipse.org/community/eclipse newsletter/2014/february/article2.php
[7] A. B. Zaslavsky, C. Perera, and D. Georgakopoulos, “Sensing as a service and big data,” CoRR, vol.
abs/1301.0159, 2013, Accessed: 3 July 2018. [Online]. Available: http://arxiv.org/abs/1301.0159
[8] C. Perera, A. Zaslavsky, P. Christen, and D. Georgakopoulos, “Sensing as a service model for smart
cities supported by internet of things,” Trans. Emerg. Telecommun. Technol., vol. 25, no. 1, pp.
81–93, Jan. 2014, Accessed: 3 July 2018. [Online]. Available: http://dx.doi.org/10.1002/ett.2704
[9] I. Corp., “Getting started with Watson IoT Platform,” Accessed: 3 July 2018. [Online]. Available:
https://console.ng.bluemix.net/docs/services/IoT/index.html
[10] Microsoft, “Azure solutions for Internet of Things,” Accessed: 3 July 2018. [Online]. Available:
https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-what-is-azure-iot
[11] A. W. Services, “What Is AWS IoT? - AWS IoT,” Accessed: 3 July 2018. [Online]. Available:
http://docs.aws.amazon.com/iot/latest/developerguide/what-is-aws-iot.html
63
[12] N. Woolf. (2016) DDoS attack that disrupted internet was largest of its kind in history, experts say.
Accessed: 3 July 2018. [Online]. Available: https://www.theguardian.com/technology/2016/oct/26/
ddos-attack-dyn-mirai-botnet
[13] OpenStack. (2016) Open Source Cloud Computing Software. Accessed: 3 July 2018. [Online].
Available: https://www.openstack.org/
[14] OpenIoT. (2016) The Open Source Internet of Things Project. Accessed: 3 July 2018. [Online].
Available: https://github.com/OpenIotOrg/openiot
[15] C. Chang, S. N. Srirama, and M. Liyanage, “A service-oriented mobile cloud middleware frame-
work for provisioning mobile sensing as a service,” in 2015 IEEE 21st International Conference on
Parallel and Distributed Systems (ICPADS), Dec 2015, pp. 124–131.
[16] E. Montserrat and F. Xhafa, “Sensors as a service in the cloud,” Master’s thesis, Universitat
Politecnica de Catalunya, Catalunya, Spain, April 2015.
[17] Oracle, “Java programming language,” Accessed: 3 July 2018. [Online]. Available: https:
//docs.oracle.com/javase/8/docs/technotes/guides/language/index.html
[18] T. Bray, “The JavaScript Object Notation (JSON) Data Interchange Format,” Accessed: 3 July
2018. [Online]. Available: https://tools.ietf.org/html/rfc8259
[19] A. W. Services, “AWS IoT Device SDK - Amazon Web Services,” Accessed: 3 July 2018. [Online].
Available: https://aws.amazon.com/iot/sdk/
[20] P. S. Foundation, “Welcome to python.org,” Accessed: 3 July 2018. [Online]. Available:
https://www.python.org/
[21] T. Szigeti and C. Hattingh, End-to-End QoS Network Design: Quality of Service in LANs, WANs,
and VPNs. Cisco Press, 2004.
64
Top Related