Sensor Data Market for the Internet of Things · Sensor Data Market for the Internet of Things...

Sensor Data Market for the Internet of Things

Ricardo Soares Miranda

Thesis to obtain the Master of Science Degree in

Electrical and Computer Engineering

Supervisor(s): Prof. António Manuel Raminhos Cordeiro GriloProf. Miguel Filipe Leitão Pardal

Examination Committee

Chairperson: Prof. Horácio Cláudio de Campos NetoSupervisor: Prof. António Manuel Raminhos Cordeiro Grilo

Member of the Committee: Prof. Alberto Manuel Ramos da Cunha

June 2018

Declaration

I declare that this document is an original work of my own authorship and that it fulfills all the require-

ments of the Code of Conduct and Good Practices of the Universidade de Lisboa.

iii

Acknowledgments

I would like to start by thanking my Supervisors, Professor Antonio Grilo and Professor Miguel Pardal

for their advices, guidance, motivation and patience. Professor Grilo and Professor Pardal were always

open to hear my doubts and to share their thoughts and knowledge. To my parents, to whom I express

my profound gratitude for their unfailing support and encouragement to complete this chapter in my life.

To my girlfriend, who even in the most stressful moments always provided great motivation and joyful

moments. To all my friends for sharing frustration, progress, knowledge and perspectives.

v

Resumo

Existem actualmente milhoes de sensores ligados a Internet. Estes sensores recolhem os mais variados

tipos de dados, desde temperatura, nıvel de humidade ou som e imagem, passando por localizacao

(atraves de GPS - Global Positioning System) ou o numero de batimentos cardıacos por segundo, por

exemplo. Estes dados podem ser obtidos para fins cientıficos, comerciais ou outros. Contudo, nao existe

nenhuma plataforma ou mercado online em que seja possıvel comprar ou vender dados. No presente

trabalho, propoe-se e implementa-se uma solucao atraves da qual e possıvel realizar essa transaccao.

Sensor Data Market for the Internet of Things (IoT) e uma plataforma em que fornecedores, entidades

ou pessoas, partilham os seus dispositivos ou dados e os consumidores podem comprar dados ou

adquirir o controlo de um dispositivo. Mais do que adquirir dados em tempo real, isto e, um sensor

recolhe uma amostra e o consumidor recebe essa amostra com um atraso inferior a uma determinada

meta temporal, a solucao apresentada permite que uma pessoa ou entidade tenha controlo sobre um

dispositivo. Esta funcionalidade e considerada como sendo diferenciadora, uma vez que nao se trata

apenas de uma troca de dados, mas sim de controlo de um dispositivo. Um cliente pode posicionar

uma camara ou movimentar um robot. A plataforma foi testada e avaliada atraves da realizacao de

casos de uso e de recolha e analise de metricas, tais como atraso entre mensagens ou o racio entre

mensagens recebidas e mensagens nao recebidas. A analise destes resultados permite concluir que a

solucao desenvolvida possibilita a partilha de sensores e dos dados que estes recolhem.

Palavras-chave: sensor, dados, mercado, troca, IoT, controlo

vii

Abstract

Currently, millions of sensors are connected to the Internet. These sensors gather various types of data,

from temperature, humidity, sound and image, to location (through GPS - Global Positioning System)

or number of heartbeats per second, for example. This data can be obtained for scientific or business

purposes, amongst others. However, there is no online platform or marketplace where data can be

exchanged. In the present work, we design and implement a solution through which it is possible to

purchase and sell sensor data. Sensor Data Market for the Internet of Things (IoT) is a platform where

suppliers, entities or individuals, share their devices or data and consumers can buy data or acquire

control of a device. More than acquiring real-time data, i.e., the sensor collects a sample and the

consumer receives that sample with a delay of less than a predefined deadline, the presented solution

allows for an entity to have control over a device. This feature is considered to be differentiating, since

it is not just a data exchange, but the ability to control a sensing device. A customer can position a

camera or move a robot. The platform was tested and evaluated through the implementation of use

cases and the analysis of various parameters such as, delay between messages or the ratio between

incoming messages and unreceived messages. Through analysis of these results we conclude that the

implemented solution allows for customers to share sensors and the data that they collect.

Keywords: sensors, data, market, exchange, IoT, control

ix

Contents

Declaration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii

Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

Resumo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

Nomenclature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1 Introduction 1

1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Background 5

2.1 Sensing Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.1.1 Simple Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.1.2 Rich Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.2 Communication Protocols Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.2.1 Hypertext Transfer Protocol (HTTP) . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.2.2 Constrained Application Protocol (CoAP) . . . . . . . . . . . . . . . . . . . . . . . 7

2.2.3 Message Queue Telemetry Transport (MQTT) . . . . . . . . . . . . . . . . . . . . . 7

2.3 IoT Cloud Platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.4 Amazon Web Services IoT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.5 S2aaS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.6 Marketplace Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3 Developed S2aaS Solution 15

3.1 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.1.1 Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3.1.2 Device Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

xi

3.1.3 Broker Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.1.4 Marketplace Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.2 IoT Marketplace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.3 Simple Sensor Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.4 Rich Sensor Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

4 Results and Evaluation 49

4.1 Single Data Measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

4.2 Long Data Measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

4.3 Video and Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

4.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

5 Conclusions 59

5.1 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

Bibliography 63

xii

List of Tables

2.1 Comparison between three IoT Platforms. . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

4.1 Single Data Results: Processing time for each actor and overall time difference (seconds). 52

4.2 Long Data Results: Processing time for each actor and overall time difference (seconds). 54

4.3 Video and Control Results: Expended time in the TCP connection and overall two-way

latency (seconds). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

xiii

List of Figures

2.1 Architecture of the AWS IoT platform. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.2 Conceptual marketplace model. The platform connects buyers and sellers. . . . . . . . . 14

3.1 General architecture of the Sensor Market for the Internet of Things. . . . . . . . . . . . . 16

3.2 Interaction between each participant upon registering an account. . . . . . . . . . . . . . 17

3.3 Interaction between each participant upon registering a device. . . . . . . . . . . . . . . . 18

3.4 Class Account and implemented methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3.5 Classes Wallet and Transaction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3.6 Abstract class Data and Class Temperature (a subclass of Data). . . . . . . . . . . . . . . 23

3.7 Abstract class Device and the methods every subclass will inherit. . . . . . . . . . . . . . 23

3.8 Class Sensor (a subclass of Device) and the methods implemented. . . . . . . . . . . . . 24

3.9 Abstract class VideoDevice is the super class of all video-related implemented classes. . 24

3.10 Class Camera implemented considering the functionalities of a controllable camera. . . . 25

3.11 Abstract class Catalog and the subclasses DeviceCatalog and DataCatalog. . . . . . . . 26

3.12 Class Product implemented to act as a item in a stock inventory. . . . . . . . . . . . . . . 26

3.13 Class Stock developed to interact with a Product. . . . . . . . . . . . . . . . . . . . . . . . 27

3.14 Abstract class Customer and the subclasses Buyer and Seller. . . . . . . . . . . . . . . . 28

3.15 Class ServerInterface is the intermediary between the Marketplace and the broker module. 28

3.16 Flowchart of a purchase, either for a Simple Sensor or for a Rich Sensor. . . . . . . . . . 30

3.17 Messages exchanged between each participant regarding a data transaction. . . . . . . . 31

3.18 Messages exchanged between each participant regarding a video transaction. . . . . . . 31

3.19 UML diagram of marketplace classes relationships. . . . . . . . . . . . . . . . . . . . . . . 32

3.20 Sequence of interactions between each participant. The AWS IoT platform is present in

steps 5 and 6. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.21 Workflow of a Data Handler for a simple data transaction. . . . . . . . . . . . . . . . . . . 36

3.22 Workflow of a Data Handler for a long data transaction. . . . . . . . . . . . . . . . . . . . 38

3.23 Single costumer interaction with a sensor. . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.24 Multiple costumers consuming the data stream. . . . . . . . . . . . . . . . . . . . . . . . . 39

3.25 Message exchange - Purchase of video & control. . . . . . . . . . . . . . . . . . . . . . . 41

3.26 Sequence of interactions between each participant in a video & control purchase. . . . . . 42

3.27 Workflow of the Video Broker. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

xv

3.28 Workflow of a Data Handler instance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

3.29 Single costumer utilizing the controllable video camera. . . . . . . . . . . . . . . . . . . . 46

3.30 Multiple costumers consuming the video stream. . . . . . . . . . . . . . . . . . . . . . . . 47

4.1 Number of allocated Handlers and their lifetime. . . . . . . . . . . . . . . . . . . . . . . . . 50

4.2 Temporal discrepancy between reception and schedule times. . . . . . . . . . . . . . . . . 51

4.3 Average time usage for each participant regarding a Data transaction . . . . . . . . . . . 52

4.4 Temporal discrepancy between reception and schedule times. . . . . . . . . . . . . . . . . 54

4.5 Average time usage for each component regarding a Video and Control transaction. . . . 56

xvi

Chapter 1

Introduction

The Internet of Things ”is a world where physical objects are seamlessly integrated into the information

network, and where the physical objects can become active participants in business processes” [1]. In

this everything-is-connected world, even everyday objects can be integrated into the network (e.g. a

temperature sensor, the coffee machine, a car, etc.). The technology or Internet protocol that enables

such connectivity, for instance, Bluetooth, ZigBee, Radio Radio-frequency identification (RFID), may

change but, nevertheless, the concept of interconnection remains.

The infrastructure and the popularity of IoT make new interaction models possible, such as Sensing-

as-a-Service (S2aaS). The S2aaS model depicts a marketplace where interested consumers trade sen-

sor data. Nowadays, in developed countries, the population is concentrated mostly in big cities and

industrial areas. A Smart City is a good example for the growth of IoT. It is possible to measure a variety

of parameters and data in an urban area, from pollution to automotive traffic, luminosity in the streets,

etc.

1.1 Motivation

A Sensor Data Market will be as strong as the real-life scenarios empowered by it. Without any real-

life scenarios, the marketplace may just be an idea, but with no application. The following is a set of

possibilities enabled by IoT devices equipped with sensors. The scenarios could be integrated in a

marketplace.

Atmospheric pollution control and sports tracking

A network of devices capable of measuring humidity, temperature, wind and CO2 levels in the air

could not only collect data for a private entity, but also supply a governmental institution. The private

entity creates a website through which presents this data to athletes or sports enthusiasts showing

where the cleaner areas are.

Noise pollution control

What if it was possible for a person who is buying a house, for instance, to know if a certain area

1

in a neighbourhood has a high level of noise pollution? Every smartphone and laptop has a built-in

microphone, allowing every person to measure the noise in its proximity. Such data could be uploaded

to a platform and then shown to any interested participant.

Radio frequency spectrum monitoring

Dense urban areas have hundreds of devices connected to a cell tower from a service provider. In

the radio domain, the allocated bands for each company are highly regulated. It is in the best interest of

every party to avoid interference between signals originated by competitors or from adjacent cell towers.

Also, the regulatory entity must ensure that each service provider is operating according to its contracts

and occupying just the agreed spectrum.

Automotive traffic management

In modern cities, managing traffic is a complex problem. Knowing the flow of vehicles in real-time,

how and where to divert the traffic could lead to a better traffic management. Infra-red sensors measur-

ing the number of vehicles crossing its beam, or video cameras, in traffic lights, showing live feeds of a

road or intersect could help to solve this situation.

Directed Advertisement

The growth of social media platforms, such as Facebook or Twitter, enables a sort of directed adver-

tisement or tailored ads focused to a very specific demographic. This new mechanism requires a crucial

element: information (e.g, age, sex, current location, etc). Knowledge about where a certain type of

person usually is or passes by will most certainly assist a company to plan the correct site to place an

ad or a nearby restaurant to send an alert to the smartphone of a potential customer.

All these scenarios have a common factor: data consumers need to find providers of sensor data.

Creating a centralized platform or marketplace, allowing for every person and entity to buy or sell data

from a variety of sensors that collect all sorts of information, might not only accomplish, but also simplify

the scenarios presented above. Furthermore, a universal Sensing-as-a-Service platform has yet to be

developed.

1.2 Objectives

The main goal of this project was to design, implement and test a Sensor Data Market for the Internet

of Things. The solution considers the characteristics and limitations of different sensing devices. Addi-

tionally, it facilitates integration with existing IoT platforms and makes use of communication protocols

specifically designed for sensors. Furthermore, one of the key functionalities of this project is to share

sensors and to buy or sell the data they produce.

2

1.3 Thesis Outline

In Chapter 2 of this document, we start by analysing the state-of-the-art. In Chapter 3, we introduce

the proposed solution and define the structure of the platform. Chapter 4 defines how our work will be

tested and evaluated. Finally, in Chapter 5 we present our conclusions and ideas for future work.

3

Chapter 2

Background

This chapter presents the related work developed not only in recent literature, but also from an industry

point of view. A brief study of the current sensing devices is presented in Section 2.1, the selection of the

most adequate communication protocol in Section 2.2. In Section 2.3 we present current IoT platforms

and Section 2.4 highlights the most complete one, in our perspective. Section 2.5 presents an overview

of the existing work developed in S2aaS in recent literature. Section 2.6 presents a brief summary on

the electronic marketplace model, as we will adapt this concept to the IoT context.

2.1 Sensing Devices

One of the most common objects in the world is a smartphone. The number of smartphones is expected

to have reached 2.6× 109 in 2017 and pass 5× 109 by 2020. Additionally, according to Perera et. al [2]

there will be 5× 1010 to 1× 1011 devices connected to the Internet by 2020.

Smartphones are equipped with several embedded sensors (GPS, microphone, camera, etc.) mak-

ing it an interesting study case to be used as a sensor device in S2aaS applications. Some recent

studies such as [3] seek to explore the role of smartphones in a S2aaS architecture. The authors aim to

understand how to integrate these devices and what kind of participation they have, i.e. taking measure-

ments or ordering readings, either automatically or with user interaction. They also address the need

for efficient management of physical (sensors)and software (selection of sensors to be used) resources

and also of energy saving(smartphones). Besides the analysis of the device’s role, the article contains

recommendations for the development of an incentive mechanism, which is a very important component

in order to develop a successful S2aaS solution.

Considering the millions of sensors in existence, their specific characteristics and functionalities it

should be important to differentiate each device in order to properly categorize them. This project at-

tempts to differentiate the devices into two categories: Simple devices and Rich devices. However, this

model only applies to devices or networks of devices capable of communicating with the Internet.

5

2.1.1 Simple Devices

From Machine-to-Machine (M2M) sensor systems to WiFi connected LED lamps, there are millions of

devices connected to the Internet. Some devices perform only simple tasks, such as measuring data

or receiving instructions. Those devices are categorized as simple devices. A smartphone can be

seen as a single simple device composed of several sensors. Each sensor can measure data and

the smartphone is responsible to send it. A network of street lamps with luminosity sensors, could, in

addition to their normal functionalities, share the data measured to the public through a marketplace.

Dozens of temperature, humidity and air pollution sensors could be scattered in a city to create a map

of the city’s most polluted areas, not only to scientific community and to the energy producers, but

also to the public. Despite being considered as simple devices, these sensors can be crucial to a future

Marketplace for the Internet of Things. These devices can be less expensive than more complex devices

and easier to deploy.

2.1.2 Rich Devices

Rich devices can be described as devices capable of execution more complex tasks than the simple

sensors. Furthermore, these devices can require more processing power, create sound or video streams

or receive control requests. Controllable cameras, Unmanned Aerial Vehicles (UAVs), or robots can be

an example of rich devices. A controllable telescope could be available to the public, thus allow for a

person to use this device and observe a specific planet or star, for example. Consider a person who

intends to create a video clip or collect pictures of a natural landscape or monument. A small and

controllable UAV, available through the marketplace, might be the solution of the previous scenario.

2.2 Communication Protocols Analysis

The Internet of Things communication protocol stack is important. The vast majority of devices in the IoT

are small sensors, thus having low processing power and being very energy dependent, but also they

are limited devices and cannot have a full TCP/IP stack. Although the security matter is a critical issue,

the discussion presented below is focused on the interoperability and efficiency of the overall system,

therefore directed to the application layer in the protocol stack. As the devices have limited processing

capability, the communication protocol cannot be too demanding, the overhead should be reduced, the

encryption must be, for now, an add-on. Furthermore, considering that the device is probably connected

to a low capacity battery, the communication process should be efficient and fast: measure, compute

the data and send.

2.2.1 Hypertext Transfer Protocol (HTTP)

One of the most widely known and employed protocols is the Hypertext Transfer Protocol. HTTP [4]

is an application-level protocol characterized by a request/response model and the communication is

6

usually established over TCP/IP. TCP connections do not perform well in the IoT world, due to its pri-

mary characteristic, the connection. By establishing a connection, with delivery assurance method and

congestion control system, it creates an environment prone to be time consuming and resource (energy

and processing power) demanding. Furthermore, HTTP headers occupy a large portion of the mes-

sage, resulting in a low payload size that may cause issues when transmitting larger sets of information.

Nevertheless, it is a widely-utilized protocol, thus allowing a higher interoperability between the various

components of the platform.

Amazon IoT and Azure IoT support HTTP protocol via a Representational State Transfer Application

Programming Interface (REST API), allowing the execution of standard operations in a simple manner.

For example, it is possible to publish data invoking a POST request message in Azure IoT.

2.2.2 Constrained Application Protocol (CoAP)

CoAP [5] is a specialized web transfer protocol for use with constrained nodes and constrained network,

such as Wireless Sensor Networks (WSNs), and also for machine-to-machine (M2M) applications. Un-

like HTTP, CoAP communicates over UDP (User Datagram Protocol) a connection-less transport pro-

tocol, thus not having the issues caused by the TCP connections. CoAP provides a request/response

interaction model, in the same sense as HTTP, although it does not aim to only compress it. Rather,

it optimizes a subset of REST applications common with HTTP. Although CoAP is not a highly-utilized

protocol, by implementing a gateway, it can communicate with HTTP agents, thus increasing the inter-

operability of this protocol. Two other key aspects are the low header overhead, allowing more data

payload in the message, and the processing power requirements, which are very small. However if the

message size increases the probability of message loss in UDP becomes higher than in TCP, therefore

CoAP must retransmit the whole message.

2.2.3 Message Queue Telemetry Transport (MQTT)

MQTT [6] is a widely adopted lightweight publish/subscribe messaging protocol. It is useful for use

with low power sensors and machine-to-machine applications. In IoT a sensor represents a publisher,

gathers data and sends it to a broker. An entity that is interested in a certain topic, for example, sensory

information, connects to a broker and subscribes a topic. Afterwards, it will receive the data collected

that belongs to that topic. This architecture has three main components: Publisher, Broker, Subscriber.

In MQTT a message is published to a topic. Clients may subscribe various topics and every client

subscribed to a topic will receive every message sent to that topic. For the broker this is impervious, as

this element does not read the messages, it only classifies the data into topics and sends the messages

to the subscribers. This broker may represent a problem: every publisher must be configured with the

broker’s address and the broker itself can become a single-point of failure. MQTT communicates via

TCP so, as with HTTP, there may be some issues regarding the performance of the overall transmission

when compared to CoAP (UDP).

7

As discussed further in Section 2.4, Amazon IoT message broker is based on MQTT v3.1.1, but with

some particularities. Azure IoT also natively supports the MQTT protocol.

2.3 IoT Cloud Platforms

One of the easiest impacts to acknowledge of a sensing application is its data output. Zaslavsky et al [7],

discuss the future role of sensing in terms of the needs of information storage, as well as the growing

importance of processing all this data created by sensors or other devices. The authors approach

S2aaS, its architecture, but not in high detail. An example of a S2aaS scenario is described, however the

example is equal to one presented in [8]. There is also a reference to the possibility of cloud computing

acting a service to process the data purchase in a S2aaS platform.

Currently, there are several IoT cloud platforms in the market from several world-renowned compa-

nies: IBM Watson IoT [9], Microsoft Azure IoT Hub [10] and Amazon AWS IoT [11]. In this section an

analysis of those IoT platforms is presented. Additionally, below is brief discussion of the security com-

ponent of each platform that was analysed. The security issue is also an important module of every IoT

cloud platform. One of the largest cyber-attacks recently registered [12] was an DDoS attack targeted

at a DNS service provider. The primary weapon of this attack was the Mirai botnet, which is mainly

composed of infected IoT devices.

• AWS IoT: The AWS IoT message broker and Thing Shadows service encrypt all communications

with Transport Layer Security (TLS). TLS is used to ensure the confidentiality of the application

protocols (MQTT, HTTP) supported by AWS IoT. For MQTT, TLS encrypts the connection between

the device and the broker. TLS client authentication is used by AWS IoT to identify devices. For

HTTP, TLS encrypts the connection between the device and the broker.

• Azure IoT Hub: The communication path between devices and Azure IoT Hub, or between gate-

ways and Azure IoT Hub, is secured using industry-standard TLS with Azure IoT Hub authenticated

using X.509 protocol.

• IBM Watson IoT: The Watson IoT Platform supports connectivity over TLS v1.2. Devices con-

nected use a combination of clientId and the authentication token that is generated when adding

the devices to a platform.

Table 2.1 shows that the three platforms are similar, however the implementation of each element is

obviously different [11][10]. The cloud device, which is a digital representation of the physical device,

varies in a significant manner. The Application Programming Interface (API) and the Software Devel-

opment Kit (SDK) can be developed in several programming languages such as, Python, C, Java and

NodeJS.

AWS IoT can generate a certificate automatically when a new device is registered in the platform.

The certificate is used for device authentication. Thus, ensuring a more secure system than other

schemes, such as user name and password. Creation, management and revocation of certificates

8

can be achieved through the AWS SDK or CLI. It is also possible to disable certificate-authentication,

although not recommended. Azure IoT and IBM IoT implement a similar security scheme. IBM IoT

allows for different security policies, characterized by combinations of authentication via certificate, via

token, or via both. All platforms implement TLS connections in client (device) to server communication

though, IBM platform allows the deactivation of this mechanism.

Azure IoT Hub AWS IoT IBM Watson IoT

Protocols MQTT, HTTP and AMQP HTTP and MQTT HTTP and MQTT

Gateway Possible Possible Possible

Interaction with

API and SDK Yes Yes Yes

CLI Azure CLI + Azure PowerShell AWS CLI Watson CLI (Python)

Cloud Device Device Twin Device Shadow Simple Device

Security X.509 certificates + TLS X.509 certificates + TLS Certificate/Token + TLS

Table 2.1: Comparison between three IoT Platforms.

Azure IoT Hub

The Azure IoT Hub provides a collection of services for connecting and extracting data from sensing

devices. In similarity with existing IoT platforms, Azure IoT Hub enables bidirectional communication

between the devices and the cloud application. One of the main characteristics of this platform is the

support of a variety of communication protocols, directly or through a gateway. Furthermore, Azure IoT

Hub exposes an API capable of sending messages to the devices, if the devices support this mechanism.

Following are presented the key elements in Azure IoT Hub Architecture.

Device twins

Device twins are the cloud representation of the real devices. Each device connected with the IoT

Hub has an individual Device twin. The device’s configuration and metadata are stored in the Device

twin – JSON document.

Gateways

The gateway is deployed to perform a translation between two communication protocols. Azure IoT

Hub defines two types of gateways: protocol gateway and field gateway. The first is only capable of

protocol translation, the latter also provides device management services, for example. A gateway is

required when a sensing device cannot communicate with the IoT Hub in a supported protocol.

Identity registry

This component stores information from devices that are authorized to connect to the IoT Hub. Every

device permitted has an entry in the Identity registry. Additionally, the IoT Hub Identity registry provides

a set of operations to manage the devices connected to the IoT Hub.

9

IBM Watson IoT

The IBM Watson IoT Platform provides a versatile IoT toolkit to connect, manage and extract data from

a wide variety of sensing devices. Watson IoT Platform can communicate or interact with external

applications, thus having the ability to export sensor data to Databases or Real-Time Management

Dashboards. Below are stated the key components of the Watson IoT architecture.

Organizations

Every entity, company or person, has an individual account which is identified by the organization

ID. This mechanism ensures that only authenticated devices or applications can access to the data

associated to your ID.

Gateways

This module acts as a bridge between devices that cannot connect directly to the Watson Platform.

It can communicate over several protocols with the device and over MQTT with the IoT platform;

Devices

The Watson IoT divides devices in two types: managed devices and unmanaged devices. Manage

devices have a management agent and can perform a set operations, such as, update, reboot, shut

down, etc. These are the intelligent devices. The unmanaged devices can only send and receive

messages or commands. Unlike the managed devices, this class doesn’t have a management agent

neither can receive management instructions – dummy devices;

Events

This component is somewhat comparable to the topic in AWS IoT or in MQTT protocol. Devices

publish data using the event mechanism. Devices send data linked to an event and an application or

service can subscribe this event and receive the information.

OpenStack

OpenStack [13] is a cloud operating system that controls large sets of devices or resources throughout

a datacenter, all managed through a dashboard. The OpenStack’s architecture is composed by several

independent modules. These modules provide APIs for the administrator and the users to manage the

resources, send messages between devices and communicate within the OpenStack ecosystem itself.

OpenIoT

The OpenIoT [14] middleware infrastructure will support flexible configuration and deployment of algo-

rithms for collection, and filtering information streams stemming from the internet-connected objects,

while at the same time generating and processing important business/applications events.

10

Discussion

Innovation plays a crucial role in this field. Every company desires to be ahead of the competition and

this goal can be achieved by simplifying the interaction between the user and the platform or by creating

new services or applications. Considering that these three platforms are relatively new, it is expected to

see new functionalities or even changes in current platforms.

After analysing all platforms, the Amazon Web Services IoT platform was identified as being the most

interesting. A more detailed description is presented next.

2.4 Amazon Web Services IoT

The communication architecture of the AWS IoT platform revolves around the concept of topic and state.

The state can be a value, a key-word or a symbol. The topic acts like a channel, a device publishes

a message to one topic and clients can subscribe to these topics to receive new messages sent by

devices. This is how the publish and subscribe model works in AWS IoT. When a device (sensor) sends

data, it updates the current state of a property. This information is stored in the Thing Shadow. The

Thing Shadow can have multiple pairs of property-state that are encoded in JSON format. Also, a client

with an app or using the AWS IoT CLI can modify the state of a property. When one device is registered

in the AWS IoT a copy of it is created, a Thing Shadow. As said above, the properties and attributes

given and the existence of a Registry allow a client to manage its devices. It is possible to update a set

of devices altogether without manually send the data, delete, update (state and attributes) and remove

devices. Using the REST API one can query and list the devices, besides accessing the other available

operations. A device connects and sends/receives data to the AWS over MQTT, HTTP or WebSockets.

The Gateway and the Message Broker allow secure and low-overhead communication between the

devices, the cloud and other applications. By deploying a Rule, via the Rule Engine, it is possible to

store or process data sent by sensors, or even filter it. The device is programmed considering the

AWS IoT architecture, the communication protocols and the framework. The configuration of the device

is accomplished via the AWS IoT SDKs. However, the REST commands simplify the communication

and configuration. Amazon developed a set of SDKs from which a group was designed to be used on

micro-controllers/resource-constrained devices.

The AWS IoT platform is composed of several elements, which perform various functions. These ele-

ments can be observed at Figure 2.1. When analysing this platform some components are immediately

seen as fundamental. Below are stated the key components.

Device Gateway

The Device Gateway mediates the communication between connected devices and the cloud ca-

pabilities. The gateway operates using the publish/subscribe configuration, the MQTT protocol. It also

supports publishing using HTTPS protocol.

Message Broker

11

The Message Broker is a publish/subscribe broker service that enables sending and receiving of

messages to and from AWS IoT. It maintains a list of all client sessions and the subscriptions for each

session and uses topics to route messages from the publisher to the subscriber. In a way, the message

broker contains the Device Gateway. The message broker implementation is based on MQTT protocol,

version 3.1.1, with some particularities. There are some restrains regarding the Quality of Service (QoS),

for example AWS IoT does not support the QoS level 2. There is no guarantee that messages will be

delivered in the correct order. Also the AWS IoT broker does not support persistent sessions and cannot

store any messages, thus by requesting such functionalities the client will be disconnected.

Figure 2.1: Architecture of the AWS IoT platform.

Thing Registry

This component performs the role of a database or a list to store the Things. A Thing can be identified

by its name or attributes, allowing the user to manage, search and update it for example.

Thing Shadow (Device Shadow)

A Thing is a representation of a real device or sensor. A Thing Shadow is a JSON document that is

used to store and retrieve current state information. When a device communicates with the AWS IoT it

sends a new state for its properties. For example, it is possible for a user with an app to turn on a light

bulb. This action is accomplished by updating the state of the propriety (in the Thing Shadow) connected

to that functionality. The AWS IoT will then send a message to the device telling it to turn on.

Rules Engine

A set of instructions that allow the interaction of the devices with other AWS Applications. It is

12

possible to redirect the message to a queue, feed it to a Machine Learning Service, store it in a DB, etc.

2.5 S2aaS

S2aaS model or cloud platform has so far not been the target of many publications.

Perera et. al [8] discuss a detailed architecture of a Sensing System as a Service, and more specifi-

cally a possible structure for an implementation in a Smart City. The authors describe the various layers

of the model and categorize various classes of information, sensors/devices and their owners. It gives

examples of some application opportunities within a Smart City and also describes three imaginary op-

eration scenarios. In the end the authors present several advantages and benefits of S2aaS model.

Another good study on S2aaS was developed by Chang et. al on [15]. Although this article focuses

in the framework of the Mobile S2aaS, the authors discuss and classify the types of sensing services,

sensing activities and devices. The authors design an architecture considering the providers, the clients

and the cloud platform service. The latter is developed to connect all the other elements. The framework

features the role of the Utility Cloud as a proxy to deliver the data from the supplier to the clients. To im-

plement the proposed solution the authors developed a workflow and a prototype of which they present

some results.

The solutions presented in [8] and [15] provide a foundation for future works. We will extend the work

from the solutions discussed above. The Utility Cloud in [15] is a crucial element in the architecture of a

marketplace platform. The analysis [8] of sensors and information is determinant to properly develop a

storage system.

The S2aaS subject is somewhat recent, thus every approach is laying new ground and is opening

new topics to future work or analysis.

Eduard Montserrat, from Universitat Politecnica de Catalunya developed a software allowing a client

to share his sensors with another user of the platform. This project [16] discusses the business model

and implements a web application to manage the sensor network. The authors refer that the network is

implemented through the cloud and is capable performing several actions. The platform allows its users

to add sensors, choose, search, view the available sensors on a map and buy or sell sensor data. The

approach of the business model is very limited, it does not explain how to perform transactions and does

not create the win-win situation, although this particular expression is mentioned several times. The

state-of-the-art discussion of sensor clusters and their classification is useful. The author developed a

back-end structure, created management routines for the sensors and the User Interface (website/web

app) of the platform through which a user is able for example to, create an account, find a specific sensor

or set-up a notification for a predetermined event. A description of the physical architecture, is presented

with little detail. The real sensors are replaced by a mini PC (Raspberry Pi), thus the system was not

evaluated in an actual deployment. Besides, there is no analysis of how the sensors are connected

to the platform, besides the fact that the Raspberry Pi would be running OpenStack OS to interact

with the platform. One critical aspect of a future service such as the one studied in the project is the

communication protocol between the sensors, the platform and inside the platform itself. However, the

13

author does not present an analysis of the Internet protocols suitable for this environment.

Our solution is similar to the work [16], but presents a more detailed analysis of the communication

protocols. The architecture is different and has more functionalities than the solution above. Further-

more, we conduct thorough testing of every module and operation.

2.6 Marketplace Models

Generically the marketplace will allow for consumers and suppliers to meet and to exchange data for

currency. An entity can play both roles: consumer and supplier. One company can buy data from other

entity and at the same time sell information measured by its sensors. The on-line market development by

Amazon has a method to evaluate the sellers, by reviewing the products sold or the arrival time delay, for

example. This feature transfers the certification or grading of the entities away from the platform manager

(or developer) to its users allowing it to self-manage. Figure 2.2 illustrates a simple marketplace model.

Buyers access the marketplace to acquire sensor data provided by the sellers. The marketplace acts an

intermediary between the two sides of the transaction.

Buyer 1

Buyer 2

Buyer 3

Buyer 1

Buyer 2

Buyer 3

Buyer 1

Buyer 2

Buyer 3

Buyer 1

Buyer 2

Buyer 3

Platform 1

B2B Marketplace

Platform 1

B2B Marketplace

Seller 1

Seller 2

Seller 3

Seller 1

Seller 2

Seller 3

Figure 2.2: Conceptual marketplace model. The platform connects buyers and sellers.

2.7 Summary

Considering the project at Section 2.5 and the research that has been so far carried out by the scientific

community, one can say that Sensor-as-a-Service is a viable field to study and to analyse. Besides, there

is still a wide range of subjects in open discussion, for instance, there is no standard communication

protocol for this type of application or even a fully embraced Internet protocol in the IoT world. From the

IoT platforms analysed we concluded that the AWS IoT was the most advantageous platform to integrate

in the Sensor Data Marketplace. The architecture of the AWS IoT platform, similar to the architecture

of the MQTT protocol, and the SDK provided by Amazon, used to communicate with the platform, were

the key aspects of the decision. Furthermore, the protocols HTTP and MQTT were considered to be the

most adequate to implement.

14

Chapter 3

Developed S2aaS Solution

This chapter presents the proposed solution and an analysis of the respective architecture and its com-

ponents. After analysing recent literature and investigating IoT platforms already available it was con-

cluded that, it is still not possible to rent or share a temperature sensor or sell anonymously a set of GPS

position records. Current IoT platforms allow for companies to control and analyse data from sensors

in real time, nevertheless a Sensor Market for the Internet of Things has not yet been developed. The

proposed solution is a market platform where providers have the possibility to sell sensor data or to sell

control of sensors to sensor data buyers. The proposed platform is intended to be: reliable, thus capable

of handling multiple real-time operations; secure, blocking unauthorized access to both data and devices

and user-friendly, allowing inexperienced individuals to use the marketplace. The most recent related

work on S2aaS, states there exists a potential interest for a platform that provides users with the ability

to buy and sell data collected by sensors. Furthermore, the results can constitute the basis for a future

commercial service.

3.1 Architecture

The architecture of the proposed solution is presented in Figure 3.1. The platform is comprises three

modules: device, broker and marketplace.

Every module exports a set of functions and rules, which constitutes an API. This API allows different

modules to communicate with each other. The device module is the foundation of the architecture. This

module plays a crucial role by connecting the broker module to the sensing devices. The broker module

is able to communicate with the device module via the device API. The broker modules exposes an

API to the device and marketplace modules. For example, a client will receive data from a sensing

device via the broker’s API. The marketplace module is responsible for every operation related with

the sensor market. This component handles operations, such as, consumers searching for devices or

buying sensor data/control. The marketplace will not communicate with the device module. When a

buyer purchases data, the marketplace sends this information to the broker module, which will facilitate

a communication path between the sensing device and the buyer. The marketplace module stores and

15

manages user’s account information, e.g. email, password and accountID, as well as, broker-related

records and sensor’s information and specification.

BrokerBroker

Broker API

DeviceDevice

MarketPlaceMarketPlace

Market API

Broker API Device API

IoT Marketplace

IoT Marketplace

VideoVideo

DataData

Real DevicesReal Devices AWS IoTAWS IoT

Figure 3.1: General architecture of the Sensor Market for the Internet of Things.

3.1.1 Use Cases

There will be a variety of actions that a user can perform. Some of these actions are somewhat similar

to what a person is able to find in online markets, for instance Amazon or eBay. One key aspect to notice

is that there is no platform as the one being developed in this project.

Below, a sequence of operations to be implemented in the platform is described. Besides the oper-

ations themselves, a new user must firstly sign in or register a new account. At this point, the user can

choose to either register and associate a device to his/her account or create a request for a specific set

of data. Some operations are considered critical or fundamental, for instance to register an account, to

add/register a device, to manage the devices or to perform a data transaction.

Register an Account

Every client that either publishes its devices or buys sensor data, must have an account. The account

groups the person’s or the company’s identity, online wallet and set of devices, as well as other infor-

mation. Every user has a username, email address and associated password. These are the three

basic parameters required for the system to recognize a client. In order to register itself, the customer

must submit a username, an email and a password. Furthermore, the client must choose its profile:

buyer and/or seller. Buyer if he/she intends to purchase sensor data or seller if he/she intends to share

a sensor through the marketplace. The marketplace validates the information and attempts to create

16

an account. If successfully, it will create a new account and add it to the respective pool. Figure 3.2

presents a set of methods and messages exchanged to accomplish this operation.

ViewView

1: Register an account

:Actor:Actor :Marketplace:Marketplace :Control:Control

4: Registration result

3: Create a new Buyer/Seller

3.1: Verify the information

received

1.1: Submit the username

1.3: Submit the password1.2: Submit the e-mail1.1: Submit the username

5: Present the result

Figure 3.2: Interaction between each participant upon registering an account.

Register a Device

In order for a client to publish or allow for other clients to use his/her device, the latter must be registered

and associated to an account. As there cannot be two devices with the same name in the same account,

the user must enter a new device name.

There are millions of sensors all over the world, and thousands of them can measure the temperature

or the air pressure, however there are a few parameters that differ from device to device. The sensor’s

universally unique identifier (UUID), name, model or type and serial number are usually such parame-

ters, although the first three were considered to be enough to characterize a device. The user submits

these three parameters together with a set of characteristics of the device. For a controllable camera,

the customer must submit the IP address and the image resolution. The marketplace processes the

request and validate the information. Afterwards, the new Device is added associated to its owner via

the list of devices in his/her account. Finally, a Product is created and added to the marketplace’s Stock.

Figure 3.3 illustrates the messages exchanged and the operations performed in this scenario.

Data Transaction

Transactions are the core operations of the marketplace. When it is not possible to perform a data

transaction, then a customer cannot purchase sensor data from an available sensor or request the

17

ViewView

1: Register a device

:Actor:Actor :Marketplace:Marketplace :Control:Control

4: Registration result

3: Create a new Device

3.1: Attemp to perform

the operation

1.3: Submit the UUID of the device1.2: Submit the type of device1.1: Submit the name

5: Present the result

1.4: Submit the device’s characteristics (video resolution, IP address, etc.)

4: Create a new Product

4.1: Attemp to perform

the operation

5: Add the Product to

the Stock5.1: Attemp to perform

the operation

Figure 3.3: Interaction between each participant upon registering a device.

control of a robot or camera. For instance, when an owner removes his/her devices from sale, a customer

must not be able to purchase the sensing devices that were removed. Furthermore, if a customer intends

to purchase a time-slot of a controllable camera and that slot was already purchased the marketplace

must cancel the transaction, thus ensuring no overlaps or double purchases. For each data transaction

the marketplace must validate if the customer exists, if there is sufficient balance to purchase the desired

item, amongst other verifications. Also, the device has to exist and be available for purchase.

Next, an analysis of each module is presented.

3.1.2 Device Module

When designing and developing an architecture for a given application, the variety of devices may

present an impediment or be a difficult problem to solve due to the integration of different hardware

and/or communication protocols, for example. Additionally, the message payload format may be dif-

ferent, from device to device. Sometimes bitwise operations are required to translate the payload into

intelligible information. Not all devices can communicate in the same manner, since they may be de-

signed to use different radio frequencies and communication protocols. Gateways may be needed to

hide the heterogeneity of devices from the information systems that use them. Generically the gate-

way can be any sort of software or hardware implementation that enables the communication between

two sides. This translation agent helps to expand the range of devices that can be connected to the

platform. Nevertheless, some actions may not be supported by a sensor, i.e. schedule a sensing task.

18

Considering that the device’s owner has full power over the device, one should be able to shut down the

device, reboot it or stop any further sensing activity. The device module abstracts all these heterogeneity

aspects, allowing other components of the system to access the sensor nodes in the same way.

As discussed in Section 2.4, the AWS IoT’s Shadow Thing is a virtual representation of a real sensing

device. The integration of the AWS IoT into the device module provides a simpler and wider communi-

cation interface for the device module to export. A sensing device publishes its data to its topic via the

MQTT protocol, for example. The broker module is able to interact with a topic by means of AWS IoT’s

interface. Also, the sensor’s owner can send control messages to the sensor through the same method,

i.e., the owner can send a message to reboot the sensor, via the Shadow Thing’s topic. Therefore, the

device module is composed of real devices, gateways and the AWS IoT platform.

To implement a standard language of communication, it is vital to develop an API to achieve interop-

erability between the broker and the sensing devices or the gateway. The device API allows a variety of

actions or commands, such as:

• To establish a connection between sensing device and broker module.

• To translate the payload into usable data;

• To schedule a reading task.

3.1.3 Broker Module

The broker module is the link between the marketplace and every sensor. This module is able to com-

municate with both the device module and with the marketplace module. In greater detail, the broker

module is implemented through a Data/Video Broker and Data/Video Handler. The first is responsible

for receiving and managing new requests originating from the marketplace, while the latter, establishes

and controls a communication path between sensing device and buyer/seller. Both components have

different implementations for each type of sensor. Nevertheless, the broker always communicates with

the marketplace module via a RESTful HTTP API. The broker defines a set of URLs allowing for the

marketplace or the device modules to send requests, i.e., purchase orders, data or control commands.

The broker module API implements actions such as:

• To process the purchase of a video and/or camera control;

• To process the purchase of a single data measurement;

• To process the purchase a long data measurement;

• To create a Handler for each type of purchase;

• To send control messages to a sensing device;

• To establish a communication path;

• Other actions.

19

3.1.4 Marketplace Module

As previously discussed, every person has a device, smartphone or tablet, with a set of sensors built-in.

A marketplace is the intermediary between the buyer, who wants to gather data from sensors, and the

seller, who has sensor data to sell. This module aims to create such environment where, for example, a

person can place a temperature sensor on the street where he/she lives and sell the measured data to

the country’s meteorological service.

Any entity will be able to search through the list of available devices, select one or more and send a

request. This request can be to acquire data, either already collected or a future reading, or to control a

sensor. An owner of a sensor can designate a rate or price for a single data measurement, a collection of

measurements. This entity, as mentioned before, is a representation of a person or company. An entity

can own a sensor or device and buy or sell data. For example, subject A buys data from one sensor

that belongs to subject B for a cost of X. The platform will transfer the currency to A and will send an

instruction to the broker module to transfer the purchased data to subject B. Each device is associated

to one entity. Nevertheless, the latter can have multiple devices or none. To implement this relation, a

virtual replica of a device is created. The virtual device identifies a real sensor with a variety of fields or

characteristics, i.e. device’s name, serial number, manufacturer, location, etc. Although a virtual device

is utilised to represent a real sensing device, it is also of the utmost importance when not identifying

a real device. When a client purchases real-time data from a sensing device, the data is published to

the sensor’s topic. In order to receive the data, the client listens to the sensor’s topic, via his/her virtual

device. In this case, the virtual device does not represent a real sensing device. The whole architecture

is developed around the role of the virtual device. This entity can identify a real sensor, but it is also

possible to use it just as a virtual device. This issue is addressed in Section 3.3 in greater detail.

A very common feature in on-line markets is a rating system, where users can rate and review other

users, allowing the market to auto-regulate itself. Thus, an entity selling fake data can be excluded or

down-rated by other users, thus preventing the fraudulent user to continue its actions.

Once more, the existence of a standard language of communication and set of instructions is crucial

to create a well-defined channel of communication between modules. The marketplace API implements

a variety of actions, some of which visible to the other modules, such as:

• To create or eliminate an entity;

• To add or remove (create or eliminate) a device, a virtual device;

• To exchange data between entities;

• To export some device’s fields;

• To export data belonging to an account;

• Other management actions.

• To add currency to an account;

20

• To transfer currency;

• To send actions to a sensor, i.e. schedule a reading;

• To search and list devices;

• To purchase data collected from the sensors.

3.2 IoT Marketplace

In this section, it is presented, in detail, the implementation of the marketplace module. The marketplace

is the core element for the implementation and development of a future marketplace for IoT. Through

this platform, consumers and suppliers will be able to exchange data and control of sensing devices

for currency. As discussed in Section 3.1.4, the marketplace stores and manages all the information

necessary to its operation.

Specification

One of the most utilized programming language for the development of modern applications is Java

[17]. This prototype for a marketplace was developed in Java, thus ensuring a greater compatibility with

several Internet applications, such as, AWS IoT, AWS EC2 and Google Database. Furthermore, there

is a wide availability of Java Integrated Development Environments (IDEs) and libraries. Moreover, it is

not necessary to rewrite the entire platform if one intends to update or modify a specific module or class.

When communicating with the broker it is necessary to exchange information. The interaction between

each actor occurs according to a client-server architecture model and the server does not store any sort

of state information amid request. The information has to be organized and should not be sent as plain

text of the HTTP Request. JSON [18] is a much-utilised data format in current online communication.

The information exchanged between the marketplace and the broker is in JSON format. The rich and

simple sensor broker’s implementation will be discussed in Sections 3.4 and 3.3, respectively. The

marketplace’s structure and implementation is presented below.

Account

In every online platform or service that requires registration, a user has an account. Considering the

service/platform, this account may hold more or less information. The Account class implemented in the

marketplace is a very simple object, containing the email, password and state. Besides the constructor

(create a new Account) one can perform tasks such as: log in, log out and change a password. Figure

3.4 presents the Account class and implemented methods. An Account is integrated in a Customer

class, since by itself it cannot perform any operation as regards the marketplace. The Customer class

will be explained in detail, below.

Wallet

21

Figure 3.4: Class Account and implemented methods.

In order to purchase or sell a customer must have a virtual wallet. The Wallet implemented in the

marketplace contains solely the customer’s current balance. As with the Account class, the Wallet is

also integrated in the Customer class. This class allows a customer to check his current balance, to

add funds to his wallet. The Marketplace can change the balance when a data or control transaction is

performed.

Transaction

When two customers exchange data or control of a sensing device for currency, a record of that

exchange must be kept and associated with both consumer and supplier. The Transaction class imple-

mented in the marketplace is the record of an exchange. It stores two unique IDs, one for the consumer

and one for the supplier, the value (in terms of currency) of the operation and a timestamp for when the

transaction occurs. This class is also integrated in the Customer class. However, to create a log, the

Customer class contains a list of Transaction objects. To create a registry using a list of instances from

the class Transaction, may not be the most efficient manner, nevertheless is a simple and somewhat

reliable way to achieve it. Figure 3.5 illustrates classes Wallet and Transaction. Wallets implements

methods to manage the balance and Transaction implements a constructor only.

Figure 3.5: Classes Wallet and Transaction.

Data

Data is an abstract class that represents the data measured by a sensor. This abstract class stores

the Universal Unique Identifier (UUID) of the sensor to which it belongs. Moreover, it should be imple-

mented (subclassed) for each type of data measured by the sensors in the marketplace. If the market-

place integrates a sensor that measures temperature, humidity, air pressure, light intensity or localization

there should be an implementation of each data type.

Temperature

As discussed above, Temperature is a type of data measured by a sensor. To enable trade of this

data it is required for the marketplace to have a class Temperature that implements the abstract class,

Data. Temperature stores the sensor’s precision, the minimum and maximum temperatures it is capable

of measuring and the unit of measurement. Other types of data should mirror this implementation.

22

Figure 3.6 presents abstract class Data and Temperature and inheritance relationship.

Figure 3.6: Abstract class Data and Class Temperature (a subclass of Data).

Device

The marketplace will have different types of sensors, from temperature sensors to web cameras or

remote-controlled UAVs. A web camera will not have the same characteristics as a temperature sensor.

Device is an abstract class that contains the common features of every device. It stores the UUID of the

sensor/device and the ID of its owner (customer ID). Figure 3.7 illustrates abstract class Device. The

methods implemented in Device are available to its subclasses.

Figure 3.7: Abstract class Device and the methods every subclass will inherit.

Sensor

Sensor is a subclass of Device and represents every sensor that participates in the marketplace and

that is connected to the AWS IoT. This class stores an instance of class Data, associating the sensor

to the type of data it measures, and two components from the AWS IoT: the sensor’s name and topic.

If the name or the topic are changed in the AWS IoT, they must also be changed in the marketplace,

otherwise it will not be possible for the marketplace to communicate with the sensor or for a buyer to

receive the data he/she purchased. This class implements its normal constructors, getters and setters,

plus a constructor to initiate the topic and the super class fields, only. This constructor is invoked when a

customer intends to buy real-time data (listen to a sensor’s topic for a time interval) via AWS IoT. There

is no real device involved from the buyer’s side; this customer, only has a topic associated with a virtual

device in the AWS IoT. Figure 3.8 illustrates the implementation of Sensor.

VideoDevice

VideoDevice, a subclass of Device, is also an abstract class. Since a video device may or may

not be controllable, or even be a mobile video device, VideoDevice is a more comprehensive class,

23

Figure 3.8: Class Sensor (a subclass of Device) and the methods implemented.

that should be subclassed for more specific devices. This class contains the IP and port of the video

stream, the IP and port of the control module and the width and height of the video resolution. If the

device has no control module, the control’s IP and port should not be set. For a stationary camera,

the implemented subclass could store its GPS location and the direction where is the camera pointing.

Figure 3.9 illustrates abstract class VideoDevice.

Figure 3.9: Abstract class VideoDevice is the super class of all video-related implemented classes.

Camera

As the designation suggests, this class represents the characteristics of a real web camera. This

class should be applied to every web camera that offers the consumer the ability to purchase its video

stream and to control the camera. This class should store additional information. For example, if a

camera has specific characteristics, such as, pan and tilt movement, zoom, and IR sensor or night

vision capabilities. Figure 3.10 presents class Camera.

Catalog

For a customer to find a specific temperature sensor or a controllable web camera, every item avail-

able in the marketplace must be stored in a list to allow an item to be removed, searched and ordered.

Catalog is an abstract class that contains a list of objects and a set of abstract methods. Hence, rep-

24

Figure 3.10: Class Camera implemented considering the functionalities of a controllable camera.

resents a catalog that a customer can use to find a sensor. Considering that there will be two types

of items in the marketplace, data and devices, each type should have its own catalog, i.e., a specific

implementation of Catalog. These subclasses are the final catalog where a customer will search for a

specific sensor or where the marketplace will store a new device submitted by a supplier. Figure 3.11

illustrates abstract class Catalog and its relationship to DeviceCatalog and DataCatalog classes.

DeviceCatalog

DeviceCatalog is a subclass of Catalog. It stores a list of Devices thus, is a catalog of Devices.

In addition, this subclass overrides some methods of Catalog, implementing the algorithms to find a

specific Device and retrieve it or add a new one to the catalog. As previously discussed, Sensor (a

temperature sensor) and Camera (a web camera) are subclasses of Device, thus the list may have

objects of different classes, each representing a type of sensing device. In present-day marketplaces

and stores, a customer searches for an item based on its characteristics. The current implementation of

the DeviceCatalog does not allow a customer to search for a web camera considering its characteristics,

if it has a specific video resolution, IR filter or is capable of zooming. Instead, the customer can only

search for a device via its UUID. The same issue occurs regarding the other sensing devices: a customer

cannot search for a temperature sensor considering its characteristics, if it measures in Celsius or Kelvin,

with a precision of a tenth of unit, for example.

DataCatalog

DataCatalog is very similar to the DeviceCatalog class. It is a sub class of Catalog and stores a list

of Sensors. However, this catalog is intended to store information of the buyer’s virtual device from AWS

IoT. This catalog does not store purchasable devices or data. On the contrary, when data is bought,

the marketplace will search through this catalog to find the buyer’s virtual device, so that the buyer can

receive the data. In the DataCatalog each Sensor is searched via its owner ID.

Product

The catalog lists all the devices in the marketplace, independently of the type of sensor, the cost or

if it is available for purchase. Nevertheless, a customer may temporarily choose not to sell its sensor,

making it unavailable to other customers. In addition, when one customer purchases the control of a web

camera during a time interval. It is important to guarantee that there is no collision with other customer’s

period. Product is a class that addresses these issues. Every item in the DeviceCatalog will have a

Product associated to it. Product stores the item’s UUID, price, availability and type, for example. To

25

Figure 3.11: Abstract class Catalog and the subclasses DeviceCatalog and DataCatalog.

verify if there will be any collision when purchasing the control of a web camera, or any other type of

controllable device, Product keep a list of timestamps, representing the start and the end of each time

interval. To confirm that there is no collision, a method to compare these time intervals is implemented.

As discussed before, the UUID allows for unique identification of each Product. Figure 3.12 presents

class Product.

Figure 3.12: Class Product implemented to act as a item in a stock inventory.

Stock

As the designation implies, this class represents the stock of products presented in the marketplace’s

catalog. Stock stores a list of Products and implements several methods to manage it. These methods

allow to add, retrieve or cancel (make a Product unavailable) a Product. One other method, checks if

a Product is in the list and invokes the time validation method if necessary. Figure 3.13 presents class

Stock.

In present-day marketplaces and stores, a catalog presents to the customer all (or a set of) the items

26

on sale. The stock is the inventory or record of the items in that catalog. One customer selects an

item in the catalog. The marketplace checks if there are items available to sell in the inventory. If there

are, the order is successful. If not, the customer cannot purchase the selected item. The Catalog and

Stock implemented in this marketplace were based on the catalog and stock described above. When a

customer attempts to purchase an item from the DeviceCatalog, the Marketplace will verify if the item

exists in the Stock and if it is available for purchase. If it is available, the order proceeds to the next step.

Figure 3.13: Class Stock developed to interact with a Product.

Customer

As already discussed, customer is a person or entity. A customer can be a supplier, a consumer or

both. The same customer can purchase data from a temperature sensor and sell to another customer the

ability to control his camera. Customer is an abstract class containing all the information of a customer.

This class stores a Wallet, an Account, a Transaction list and a Product list. In addition, it stores the

customer’s first and last name, and a unique ID. As presented above, Wallet represents the virtual

wallet of a customer, and, for every successful exchange the Marketplace will interact with this object to

perform the respective operation. This interaction is possible through the methods made available by the

Customer class. Customer performs a crucial role in a future marketplace. If by any means this class or

a similar is not correctly designed or implemented, the marketplace will most certainly fail or have serious

issues. Those issues could be security issues or simple performance faults. One customer could not

purchase data, because the system did not store all the necessary information, for example. Figure 3.14

illustrates the relationship between Customer and its subclasses. Buyer and Seller classes inherit the

implemented methods in Customer, thus being able to interact with Account, Product, Transaction and

Wallet classes.

Buyer

Buyer class is one of the two classes that implement Customer. This class, as the designation

suggests, is a class for all the buyers in marketplace. All the specific methods for buyers should be

implemented in this class.

Seller

Seller is the second implementation of Customer. This class represents each seller in marketplace.

All the specific methods for sellers should be implemented in this class.

27

Figure 3.14: Abstract class Customer and the subclasses Buyer and Seller.

ServerInterface

When a customer intends to purchase data or control of a sensing device, the Marketplace will

perform a set of standard verifications to the order itself, to the customer and to the sensing device.

After this, it attempts to communicate with the server or broker of the corresponding device. The class

ServerInterface acts as a representative of the Marketplace. This class implements a set of methods that

communicate with the outside via HTTP Requests. A URL path and a JSON object characterize each

request. The URL path is the URL of the server or database and the JSON object is the information to be

sent. The implemented methods can return one of the following: a JSON object, if the customer intends

to purchase the control of a sensing device or data stored in a database; a string, if the customer intends

to purchase data from a sensor, over a period of time. Figure 3.15 presents class ServerInterface and

the methods implemented to interact with the broker module.

Figure 3.15: Class ServerInterface is the intermediary between the Marketplace and the broker module.

Marketplace

As described before, a marketplace is an environment where one customer is able to exchange a

set of data or the control of a sensing device for currency. This characterization is valid considering

the scope of this project, whereupon the concept of Sensor Data Market for the Internet of Things is

analysed.

The classes, interfaces, variables and lists presented before are, essentially, the pieces of the mar-

ketplace. How one piece communicates and interacts with another is implemented in the marketplace

itself. All the pieces are put together in the class Marketplace. This class contains every variable and

implements every method required to run the marketplace. In greater detail, the concept of marketplace

presented here requires five elements:

• Customers, persons or entities that purchase and sell data;

28

• Sensing devices, to collect data or to be controlled;

• Catalogs, to store and present sensing devices to the customers;

• Stock, the inventory of each sensing device;

• Broker, the bridge between the sensing device and the customer/marketplace.

If one of these elements is not present, the marketplace will not succeed. Without sensing devices,

there is nothing to offer to the customers, no data to be collected and no device to be controlled. If it is

not possible to organize and store devices in a presentable and searchable way, the customers will never

find the device or data they wish. If a stock/inventory is not implemented, although it is possible to find

sensing devices (through the catalog), when a customer attempts to purchase data, the device might

be offline or not available at that moment. Thus, the client would pay for something that would never

receive. Even worse, the device, a controllable device, could already be in use by another customer,

resulting in two customers trying to control the same device at the same time. Without a communication

path between the marketplace and the sensing devices, the customer will never receive data collected

by a sensor or will never be able to control a device. Lastly, if there are no customers, the marketplace

will fail, for obvious reasons.

Analysing with greater detail the methods developed in the Marketplace class it is possible to ob-

serve a similar structure to the one previously presented. Firstly, a set of procedures to create a new

customer, Buyer or Seller, a new Device and a new Product. A Device will belong to a Customer and

a Product will be associated to a Device. Next, the methods to interact with the list of Customers and

with the catalogs, DataCatalog and DeviceCatalog are implemented. It is possible to add, retrieve or

delete elements, find a specific element or to present the catalogs. Considering the Stock’s structure

and purpose, Marketplace also implements a group of procedures to interact with the Stock. It is pos-

sible to add and retrieve a Product, verify if it is available to purchase or mark it as unavailable. With

the methods and variables already presented it is possible to create a marketplace. However, two cus-

tomers cannot exchange data for currency, yet. In order to exchange data or control of a sensing device

between customers, the Marketplace class implements specific methods and protocols via the ServerIn-

terface class. Figure 3.16 presents a purchase’s flowchart. The realized purchase methods require the

following: a UUID, the device’s unique identifier; two Customers, the Buyer and the Seller; an instant of

time (timestamp) or a period of time. As Figure 3.16 indicates, the Marketplace starts by verifying if the

UUID matches with an existing UUID in the DeviceCatalog and if the Product is available (in the Stock).

After these verifications, the Marketplace will attempt to retrieve the Seller, owner of the device that the

Buyer wishes, to proceed with the purchase order. If the Buyer has enough funds to fulfil the purchase

order, the procedure will create a sale specification, or sale spec, and send it to the broker. Depending

on the broker’s reply, the Marketplace will add a Transaction to both parties, Buyer and Seller, modify

their balance accordingly, and update the Product’s characteristics. Lastly, the Marketplace will present

the outcome of the purchase operation: a URL or the purchased data if successful, an error message

otherwise. Sale specification or sale spec, is the designation of a set of data crucial to perform any ex-

change in the marketplace. This information is in JSON format and is sent with the HTTP request to the

29

broker or to the database. Considering the item being purchased, the sale spec may contain the seller’s

name and topic, the buyer’s topic (associated with the AWS IoT) and the time interval. For another type

of item, it may contain the IP address of the video stream and of the control module, amongst other

information. The sale spec is critical. Without the information it contains, the broker will not be able to

connect buyers to sensors or send data to the buyer. A set of methods is implemented to create the

sale spec, send it to the broker via a ServerInterface, and handle the broker’s response. As previously

discussed, this information is sent in the HTTP request to the broker. An instance of the ServerInterface

is responsible for creating the request, sending it and receiving the reply.

Start

UUID,

Customer,

Timestamps

hasDeviceCatalog

Success

True

Fail

hasProductStock

True

True

purchase

False

UUID,

Buyer,

Seller,

Timestamps

Figure 3.16: Flowchart of a purchase, either for a Simple Sensor or for a Rich Sensor.

30

Figure 3.17 and Figure 3.18 present the exchange of messages, in a timeline model, between each

module. The marketplace must deliver the sale spec of the purchase to the broker. Without this step, the

broker cannot setup or establish a connection between the sensing device and the buyer. Figure 3.16

illustrates that when the marketplace finishes processing a purchase order (with success) it will send an

HTTP request to the broker. The broker handles the request and sends a reply. Then, the marketplace

will show the customer the response: a URL, for a video transaction and a setup reply message for a

data transaction.

:Actor:Actor :MarketPlace:MarketPlace :Broker:Broker

1: Purchase

2: Show Response

1.1: HTTP POST Request

1.2: HTTP Response

Figure 3.17: Messages exchanged between each participant regarding a data transaction.

:Actor:Actor :MarketPlace:MarketPlace :Broker:Broker

1: Purchase

2: Show URL


1.2: HTTP Response

Figure 3.18: Messages exchanged between each participant regarding a video transaction.

UML Specification

Figure 3.19 presents the complete UML of the marketplace implementation. It is possible to observe

the inheritance mechanism, from the abstract class Device to the sub class Camera, for instance. The

creation of abstract classes and the usage of inheritance ensures the possibility to integrate new types

of sensing devices and data in future works. If a sensor measures the humidity, the air pressure, or even

its geo-localization, Data can be sub-classed/extended to address the new data type. The same can

occur with the Device abstract class. If one wishes to add a new device, such as, a robot with a video

module and a control module, one can simply create a new class that extends Device or VideoDevice.

Figure 3.19 also illustrates how classes interact with each other, highlighting how the class Marketplace

interacts with the other classes. Marketplace acts as the hub, thus, it creates, stores and manages the

other components.

31

Figure 3.19: UML diagram of marketplace classes relationships.

3.3 Simple Sensor Products

The smartphone is a great example of a device containing multiple sensors, such as, temperature sen-

sor, GPS, accelerometer, microphone. Furthermore, a smartphone can be looked at as a device with

limited power resources and processing capability. It is possible to compare this equipment with an ag-

gregate of simple sensors. Considering a network of temperature sensors with a GSM/WiFi module and

a battery deployed throughout a city, one could create a temperature map, for instance. Every smart-

phone connected to the marketplace is also able to contribute, by taking measurements and sending

them.

Specification

Considering the two types of sensing devices defined in this project and their specific characteristics,

as discussed in Section 2.1.1, the development and implementation of the broker module was also

divided. This section presents the architecture and implementation for the simple sensors. In Figure 3.1

it is possible to observe that there is no direct communication path between the marketplace module

and the device module. The broker acts as the intermediary, thus ensuring a clear separation from

customers/Marketplace and the sensing devices. Moreover, the broker module is solely responsible

for creating and managing the communication link between the other modules. The broker’s module

implementation for sensing devices is composed of:

• AWS IoT platform, the foundation of the communication infrastructure;

• Data Broker, the entry point of communication;

• Data Handler (Simple mode), responsible for managing the customer-device connection in simple

data transactions;

• Data Handler (Long mode), responsible for managing the customer-device connection in long data

transactions.

32

Furthermore, the marketplace module accesses the purchased data via the Buyer Client, a AWS

IoT client, and the device module implements a gateway to establish a communication link between the

sensor and the AWS IoT platform.

As discussed in Section 2, the AWS IoT platform was selected, to be the infrastructure behind

the implementation for the simple sensors. The platform was designed around the MQTT protocol,

a lightweight publish and subscribe messaging protocol. Through AWS IoT platform each owner of a

simple sensor has a Thing, a virtual representation of the real sensor. However, a Thing can also be a

logical entity, i.e., it may not represent a real device. This characteristic is particularly useful to represent

a buyer, as it will be explained below. Every simple sensor is connected to the AWS IoT platform via

its Thing and publishes data to a topic, designated as topic S. Additionally, it is possible to associate

a Shadow to a Thing. The Shadow can be used to send control messages to the sensor, also via the

AWS IoT. Consider the following: a customer/buyer purchases all data measurements from a sensor for

a period of time; All data messages from the sensor are published to topic S and, since, the customer

cannot communicate directly with the sensor, the customer does not have access to topic S. As pre-

sented above, a Thing can also be a logical entity. By associating a Thing to the customer and creating

a topic, topic B, it is now possible for the buyer to receive all the data measurements published in topic S

without requiring access to that topic. Sensing devices publish data to their topic (topic S). Every buyer

has a virtual device and a topic (topic B) associated to that virtual device. The broker, namely the Data

Handler, will subscribe to topic S and copy the messages to topic B. This mechanism will be discussed

in greater detail below.

AWS IoT developed and published a SDK (Software Development Kit) meant to assist in building

applications that use the AWS IoT platform. The SDK [19] allowed to integrate the platform directly

in the broker module, namely in the Data Handlers and the Buyer Client. These components were

developed in Python [20]. Furthermore, the Data Broker is an HTTP server also developed in Python.

It assigns to each valid data request or control order a Data Handler. The Data Handlers are able to

publish messages, subscribe to topics and use a device’s Shadow, through the SDK.

Figure 3.20 exemplifies the sequence of interactions between each participant in a data purchase.

The customer initiates the whole process by making a purchase in the Marketplace. The Marketplace

communicates with the Data Broker, which in turn creates an instance of a Data Handler and replies back

to the Marketplace. At this point, the Data Handler subscribes to topic S and publishes any received

message to topic B. The Buyer Client allows the customer to receive every data measurement made by

the sensor. It should be highlighted the role of AWS IoT platform in steps 5 and 6. The platform provides

its communication infrastructure allowing the Data Handler to interact with the device and the client.

Device

As discussed above, the simple sensor implemented is a temperature sensor. This sensor is in fact,

the CPU temperature sensor of a Raspberry Pi. The Raspberry Pi is connected to the Internet in order

to be able to send the data it measures and to receive instructions. The described setup aims to simulate

a sensing device composed of a temperature sensor and a GSM/WiFi module. As pointed above, the

33

MarketPlace

2

4

Data Handler

3 5

1

Handler

1

2

3

Data Broker

6

Figure 3.20: Sequence of interactions between each participant. The AWS IoT platform is present insteps 5 and 6.

AWS’s library implemented communicates via the MQTT protocol. The device’s communication module

or gateway was also implemented by means of the same AWS library, thus ensuring total compatibility.

The MQTT was selected due to its characteristics regarding the energy and processing constraints of a

remote sensing device or a device not connected to the grid. The software implemented in the device

executes, recurrently, two tasks: temperature measurement and data (message) publication. Also, right

before publishing a new message, the device verifies if there is a new message for itself, such as a

restart command. If there is, it processes that message.

The device could not communicate directly with the AWS IoT platform using the MQTT protocol, as it

was not designed to perform that task. Thus, it was necessary to implement a script to allow the device

to interact with the AWS IoT platform. The software was developed in Python and acts as a gateway.

The device was configured to measure the CPU’s temperature periodically and is also capable of making

a measurement upon request, for instance, when the Data Handler sends a data request to the device

via the device’s Shadow.

Data Broker

The broker module is the intermediary between customer and device. The Data Broker is an HTTP

server designed to process every data transaction sent from the marketplace module. Two modes of

data transaction were defined, namely simple and long. A customer/owner wishes to send a command

or control message to a sensor, for instance a customer requests a device to switch on/off a LED or

the owner intends to remotely reboot his sensor. These situations are considered to be simple data

transactions. A customer requests a data measurement at a specific time-instant. For instance, a client

wants to collect the measured temperature in a particular location at 18h00. Again, this situation is

considered to be a simple data transaction, namely a simple data purchase. The presented scenarios

are characterized as being an isolated data request or order. By opposition, a long data transaction can

be described as a request that occurs for a time period. A customer wishes to collect all measurements

from a sensor starting at 13h00 and ending at 16h00, for instance. Hence, is it important to process

each transaction type separately. Thus, the HTTP server maps two URIs. As discussed above, the

34

salespec is adapted to each type of data transaction: for a simple transaction it contains the seller’s

device name, the seller’s device topic, the buyer’s topic and the timestamp for when the order/data

must be processed/measured; for a long transaction it is composed of the same first three data fields

of the simple transaction, plus the start time and the end time of the time-window associated to the

data purchase. An HTTP request containing the salespec is sent to the data broker only when the

Marketplace validates and concludes a data transaction. If the request is destined to one of the mapped

URIs, the broker attempts to process the salespec. The specification of a transaction is sent in the HTTP

request as a JSON object. Upon success the broker will create a sub-process of a Handler. Handler is

a Python script responsible for managing the actual transaction of measured data or control messages.

Finally, the Data Broker will resume and wait for a new HTTP request.

Data Handler (Simple mode)

As previously stated, AWS IoT created a SDK in Python enabling a direct communication with the

platform’s broker whilst using the MQTT protocol. The Data Handler for simple transactions was de-

signed to send a control message to a device or to request a data measurement and deliver the data

to the customer. According to the MQTT protocol, the sensor publishes data or messages in a topic,

designated as topic S. If someone subscribes to topic S, he/she will receive every message published by

the sensor. Consider the following scenario: a customer requests a data measurement at 16h00. The

marketplace could simply send to the customer the sensor’s topic, topic S. The client has now full access

to topic S. To ensure that the customer only receives the purchased data, AWS IoT would have to man-

age the subscriptions to topic S and cancel any subscription as soon as a client receives the purchased

data. This solution was not implemented, since it is not framed in the scope of this project. One possible

solution would be to delegate the responsibility to the sensing device. The device would be responsible

for creating a new topic for each customer and destroy it after completing the request. This scenario is

highly discouraged. The sensing device should never be responsible for managing data transactions.

Hence, the broker module should handle every data transaction, ensuring that the costumer can never

interact directly with a device, the sale specification of each purchase/request is enforced, and the de-

vice is only required to publish data measurements. The Data Handler is the intermediary between the

sensor and the customer. Bearing in mind the scenario described above, in which a customer requests

a data measurement: The Data Handler receives the sale specification from the Data Broker. After-

wards, it subscribes to topic S and sends the data measurement request or the control message to the

sensor via the sensor’s shadow topic, as previously discussed. Upon receiving a message from topic

S, the Data Handler verifies if the time of reception of the message satisfies a predetermined deviation

margin. If successful, the message is published to the buyer’s topic, topic B. Otherwise, when the instant

of reception surpasses the upper limit, an error message is published to topic B. However, if it does not

reach the lower limit, the Data Handler remains listening to topic S. Finally, after publishing to topic B the

Handler terminates itself.

Figure 3.21 illustrates the workflow for a Data Handler’s instance. It should be highlighted that the

Data Handler “sleeps” until the current time reaches the timestamp of the sale specification minus a

35

setup time of four seconds (this time interval was chosen based on visual inspection of the time required

to establish all connections). This difference is essential to create the necessary connections to AWS

IoT and ensuring the Data Handler is ready to send the data request or control message on time.

Start

Read Arguments

Sleep until current

time is the

scheduled time

Create a thread

Connect to AWS IoT

(Seller) & subscribe

topic S

Create a threadConnect to AWS IoT

(Device Shadow)

Update the Shadow

New message?

No

End

Yes

Connect to AWS IoT

(Buyer)

Send message to

topic B & signal to

exit

End

Signal?

0

1

End

Figure 3.21: Workflow of a Data Handler for a simple data transaction.

Data Handler (Long mode)

Similarly, to the Data Handler for simple transactions, the Data Handler for long transactions was

developed in Python and implements the AWS IoT SDK. This Data Handler was designed to subscribe

the seller’s sensor topic and publish any received message to the buyer’s topic. As discussed above,

to request AWS IoT to implement this very specific solution would be outside this project’s scope. As-

signing to the sensing device the responsibility to manage each publish and subscribe connection is

also highly discouraged. From a hardware perspective it would be a very demanding task. Moreover,

36

bearing in mind the architecture of the marketplace this solution would be inadequate. Consider the

following scenario: a customer wishes to receive every data measurement between 16h00 and 18h00.

The marketplace could simply send to the customer the sensor’s topic, topic S. However, as already

discussed there should be clear separation between customer and sensor. The Data Handler acts as

the intermediary in the above scenario. Thus, ensuring the customer does not have full access to topic S

nor to the sensor. Furthermore, the Data Handler is responsible to send every message published by the

sensor during the time-window sent by the Marketplace. According with the presented scenario, when

some customer purchases data measurements during a period of time, the Data Handler subscribes to

topic S. Upon receiving a new message, the Data Handler will publish that message to the buyer’s topic,

topic B. This interaction terminates when the current time surpasses the end time of the purchase. At

this moment, the Data Handler shuts down and the buyer will no longer receive messages, in topic B,

from sensor.

Figure 3.22 illustrates the workflow for a Data Handler’s instance. The Data Broker starts a Data

Handler and sends all the required information. The Data Handler initiates, processes the information

and sleeps until the schedule time. Upon resuming, it creates two AWS IoT clients, one in order to

subscribe to topic S and the second to publish to topic B. This Data Handler is alive for the duration of

the time-window purchased by the customer. When the end of the time-window is reached, the Handler

terminates itself. It should be highlighted that the Data Handler “sleeps” until the current time reaches

the start time of the sale specification minus a setup time, ten seconds. This difference is essential

to create the necessary connections to AWS IoT and ensuring the Data Handler is ready to publish

new messages to topic B. The setup time is longer, as this Data Handler creates to AWS IoT clients

consecutively, whereas the Data Handler, for simple transactions, used a thread to create one of the

clients. Thus, reducing the setup time.

Buyer Client

As previously stated, the Data Handler will publish messages from topic S to topic B. Nevertheless,

the customer is not able to directly read the messages in topic B, since, he/she does not have access to

the AWS IoT platform. One could argue that a valid solution is to give access to the AWS IoT platform.

Although a possible suggestion, this project addresses this situation in a different manner. Once again,

a direct-access policy is avoided by creating a broker, in this case between topic B and the customer.

The intermediary subscribes to topic B and stores every received message in a text file. This file could

be read by the Marketplace and shown to the customer. A different approach would be to store each

message in a database accessible to the Marketplace. Thus, providing close to real-time access to the

measurements collected by the sensor.

Data Stream

Figure 3.23 presents the simplest scenario of a data purchase. Topic S, the sensor’s topic, is sub-

scribed by a single Data Handler. The Data Handler belongs to the customer that purchased the data

measurements and will publish every message from topic S to topic B, the buyer’s topic. As previously

discussed, the customer is never directly connected to the device. The broker, specifically, the Handler

37

creates the link between sensor and customer by subscribing topic S and publishing in topic B. Figure

3.24 illustrates a scenario where the sensor’s measurements are consumed by multiples customers.

Each customer has a dedicated Data Handler, represented by a single Data Handler, subscribing the

same topic S. Nevertheless, each Handler publishes messages to its buyer’s topic. The sensing de-

vice will publish to only one topic, topic S, independently of the number of customers that purchased

access to the data measurements. Each Customer-Handler and Handler-Sensor connection is estab-

lished through the AWS IoT. The platform performs a crucial role in the scenarios illustrated by Figure

3.23 and Figure 3.24.

Start

Read Arguments

Sleep until current

time is the

scheduled time

Create a thread

Connect to AWS IoT

(Seller) & subscribe

topic S

New message?

No

Yes

Publish message to

topic B

Connect to AWS IoT

(Buyer)

EndTime>CurrentTime

No

End

Yes

Figure 3.22: Workflow of a Data Handler for a long data transaction.

38

Data Handler

Gateway

MQTT MQTT

Data Handler

Gateway

MQTT

MQTT

MQTT

MQTT

MQTT

MQTT

Figure 3.23: Single costumer interaction with a sensor.

Data Handler

Gateway

MQTT MQTT

Data Handler

Gateway

MQTT

MQTT

MQTT

MQTT

MQTT

MQTT

Figure 3.24: Multiple costumers consuming the data stream.

39

3.4 Rich Sensor Products

Throughout time, technological advances have enabled the construction and development of more so-

phisticated devices. Starting from simple radio frequency-controlled robots, such as radio-controlled

cars, to fully operational UAV (unmanned aerial vehicle) controlled via a satellite-link. As discussed in

Section 2.1.2, a rich sensor is a sensing device that, amongst other characteristics, can be controlled

via the Internet. This type of devices can create a live video stream, move themselves, etc. The rich

sensor implemented in this work is an IP controllable web camera. This device is both controllable and

able to create a video stream. Not integrating this type of sensing devices in a marketplace may have a

serious impact in its success, due to the ever-growing number of these rich sensors in the world.

Specification

As shown in Section 3.1 the marketplace module does not communicate directly with the device module.

The agent responsible to connect those modules is the broker. Moreover, after a successful purchase

of a time-slot on a rich sensor, the customer will communicate with the sensor via the broker module.

With this module, the buyer can control the device or receive any type of data stream, may it be video or

sound. The broker module is composed of:

• Video Broker, the entry point of communication;

• Handler, responsible for managing the customer-device connection;

• IP/TCP Tunnel, the line of communication;

• Web page, responsible for presenting the video stream and control mechanism.

Video Broker is the element, in the broker’s module, visible to the marketplace module. It is respon-

sible for receiving every request, related to a rich sensor, created by the marketplace. In greater detail,

the Video Broker is an HTTP server developed in Java. As discussed in Section 3.1, the communication

between each module should be implemented following a RESTful pattern. Thus, the Video Broker was

implemented according with this pattern. It exposes a URL, used by marketplace to send an HTTP

POST request to the broker module; its operation does not vary with previous requests.

The customer (through a web page) cannot communicate directly with the sensing device. Firstly,

since it would be necessary to install a client-side software to validate/authenticate the customer, to

setup the connection with the sensing device and to translate the requests made to the sensing device.

Secondly, the management of the connection would be client-side, which is not recommended. There-

fore, it is perfectly reasonable to implement a server-side management agent to all the connections

between customer and device. This agent is the Handler. In detail, the Handler must create, maintain

and destroy an IP/TCP tunnel, for the video stream, and interpret and send (to the device) any control

request issued by the customer.

Figure 3.25 presents a brief diagram for a new request. This diagram illustrates how each participant

acts and with whom it communicates, assuming each request is valid. When a customer purchases

40

a slot of time, the Marketplace will send an HTTP request to the broker, specifically the Video Broker.

The Video Broker will create an instance of Handler and send a reply so that the Marketplace is able

to create the URL. Once the customer opens the URL, the Handler created before will setup the video

IP/TCP tunnel. At this point the customer is able to watch the video stream. To control the web camera

the customer should press the control buttons. Each control button press will send an HTTP request to

the Handler, that is processed and relayed to the camera.

:Actor:Actor :MarketPlace:MarketPlace :Master:Master

1: Purchase

2: Show URL

:Handler:Handler


1.1.1: Create the Handler

1.2: Sleep1.3: Start server

1.3: HTTP Response

ViewView

3: Open URL

3.1: HTTP GET Request

3.1.1: Create Video tunnel

3.2: Video Stream

4: Control Button

4.1: HTTP GET Request

4.1.1: Process the request

4.2: HTTP Response

Figure 3.25: Message exchange - Purchase of video & control.

Figure 3.26 exemplifies the sequence of interactions between each participant in a video and control

purchase. The customer initiates the whole process by making a purchase in the Marketplace. The

Marketplace will communicate with the Video Broker, which in turn creates an instance of the class

Video Handler and replies back to the Marketplace. At this point, the customer will receive the URL on

which he/she will be able to watch the video stream and control the device. When the customer opens

the URL (inside the purchased time window), the Handler acknowledges that connection and interacts

with the video device. Step 6 and step 7 represent the continuous interaction between customer and

video device.

41

Handler

Video

CAM CONTROL

Video

HTTP

Handler

Video

CAM CONTROL

Video

HTTP

Video

Video

MarketPlace

2

4

Handler

3 7

1

Master

5

6

Figure 3.26: Sequence of interactions between each participant in a video & control purchase.

Device

As presented above, the rich sensor implemented is a IP controllable web camera. This device can

create a video stream and associate it to an IP and a port by creating a server. As it is controllable, the

control module has an IP and port. The separation between video and control is essential. Ensuring

a division between the two prevents a customer, which only purchased the visualisation and not the

control, to use the video IP/TCP connection to attempt to control the camera. The device’s control

module is a program developed in C. This program manages a TCP socket to receive control messages

from the Handler’s server, translate them and move the camera.

Video Broker

The broker module is the connecting link between customer and device. The Video Broker, is a class

that creates the HTTP server exposed to the Marketplace. When invoked, it associates the HTTP server

to an inner class designated Register. This class handles each HTTP request received by the server.

After determining if the request itself is valid, it will process the salespec sent by the marketplace. The

salespec is a JSON object with a predefined structure and content. If the JSON data inside the HTTP

request received by the server does not comply with the established structure or is missing any field,

the operation will stop and fail. The specification contains the following fields: the device’s UUID; the

video stream IP; the video stream port; the device’s control module IP; the device’s control module port;

the start time; the end time; an authentication string or hash. The salespec’s structure was designed

considering the device implemented. Different devices with different characteristics could require a new

salespec structure. Once processing is finished, the broker will create a file to store all the information

from the specification, create an instance of Handler and send a reply to the marketplace. The reply

contains the port associated to the HTTP server of the Handler previously created.

Figure 3.27 illustrates the workflow for an instance of Video Broker. Firstly, maps a URI path in the

HTTP server and binds it to a new object of Register. After that, it starts the HTTP server. Each valid

HTTP received request is processed. If successful, an instance of Handler will be created, otherwise

only a reply will be sent to the origin of the request. The HTTP server of the Video Broker will remain

alive until it is manually terminated.

42

Start

Map a URI path

and set an

HTTPHandler

New HTTP

Request?

Yes

Is the URI path

mapped?

Yes

Process the

Request

Create a

Handler’s

instance

Fail

Reply

End request

processing

No

No

Success

Start HTTP server

Figure 3.27: Workflow of the Video Broker.

Video Handler

Handler is a class that implements all the methods and a set of inner classes required to interact with

the customer and the device. When invoked the Handler will read the stored file created by the Video

Broker. After obtaining the necessary information, it will sleep until the beginning of the time window

bought by the customer. Once resumed, the program creates an HTTP server and waits for a request

from the customer. This server will not only process each control request sent by the customer, but

also manage the IP/TCP connections for the video stream and to control the device. The customer, by

opening the web page, activates a mechanism that sends an HTTP request to the Handler’s server. This

request triggers the creation of a connection (IP/TCP tunnel) between the customer’s address and the

video stream address. This connection acts as a link. From the customer’s perspective the address of

the video stream is the Handler’s server address, thus ensuring that the video stream address is never

43

shown to the customers. Once a customer sends a control request, via one of the buttons from the web

page, the server will authenticate the origin of the request. Upon a positive authentication, the request

is parsed and sent to the device’s control module. When the current time reaches the end of the time

window, each Handler’s instance will terminate the video IP/TCP tunnel, close the connection to the

control module, stop the HTTP server and terminate itself. Hence, each customer will no longer be able

to view the video stream nor control the device.

Figure 3.28 illustrates the workflow for an instance of a Data Handler.

Start

New HTTP

Request?

Yes

Is the URI path

mapped?

Yes

Process the

Request

If valid, create

video

connection

Reply

End request

processing

Map a set of URI

path to their

HTTPHandlers

Initialize the

Handler

Retrieve data

from stored

file

Sleep until current

time is start time

Start the HTTP

server

Set a timer for

termination

Curren t time

equal to end

time?

No

Yes

Close the video

connection

Close the control

connetion

End

No

If valid, send a

control

message

Video Request

Control Request

Figure 3.28: Workflow of a Data Handler instance.

44

Video Stream

As discussed in Section 3.1.4, for a customer to add a web camera or other video device with live

video stream capability, he/she must submit the IP and port of the video stream. Consider the following

scenario: The system is developed in such a way that every customer connects directly to the source

of the video stream, the web camera. The camera could have a WiFi or a cellular Internet connection.

Either way, if several customers purchase overlapping time-slots for this device, they would be viewing

the video stream simultaneously. This would cause a massive consumption of processing resources

and bandwidth in the device. Now, considering a different scenario: The system has an intermediary

between the customer and the video device. The customer will never be directly connected to the other

end of the stream and will never know the IP and port of the video device, assuming it is the same of the

video stream. Additionally, the number of customers, consuming the video stream, does not influence

the device’s resources. Considering the advantages and disadvantages of both scenarios, this project

implements the latter. The Handler acts as the intermediary, in the second scenario. Thus, employing a

separation of concerns, the customer simply opens a web page and sends HTTP requests; the Handler

manages both connections, supplies the processing resources and bandwidth. The IP/TCP tunnel is

once again implemented to avoid a direct connection between customer and device.

Figure 3.29 presents the simplest scenario of a video and control purchase. The camera’s video

stream is viewed by a single customer. In this situation, only one customer has interaction with the video

device. It is clearly visible the existence of two separate data streams: the video stream originated by

the camera and the camera control stream. As previously discussed, the customer is never directly

connected to the device. The broker, specifically the Handler, creates a IP/TCP tunnel to patch the

video stream to the customer. Also, it receives any HTTP (control) request and relays such request

to the device. Figure 3.30 illustrates a scenario where the camera’s video stream is consumed by two

or more customers. Nevertheless, only one customer is able to control the device. To the right of the

Handler only two data streams exist. On the other hand, to the left of the Handler it is possible to

observe four data streams. Of those, one is dedicated to control requests. The remaining are video

streams. The video device will create only one video stream, independently of the number of customers

that purchased that video stream. The Handler will, for each video customer (a customer that purchased

only the video stream), create a IP/TCP tunnel. Thus, the bandwidth required to transmit each stream

is allocated in the Handler’s network and not in the device’s network. Moreover, the device does not

need to create and process a video stream for each customer, therefore, not increasing the processing

requirements of the camera.

3.5 Summary

In this chapter we presented the architecture and each component of our solution. The Marketplace

is where costumers are able to share their devices and purchase sensor data or control of a device.

The Marketplace can be considered as the core of the solution. Rich and Simple Devices have specific

hardware and functional characteristics, as such, it was necessary to implement different approaches in

45

Handler

Video

CAM CONTROL

Video

HTTP

Handler

Video

CAM CONTROL

Video

HTTP

Video

Video

MarketPlace

2

4

Handler

3 6

1

Video Broker

5

Figure 3.29: Single costumer utilizing the controllable video camera.

order to create a structured communication channel between customer and device. Furthermore, within

the Simple Device development, simple and long types of data transactions required a dedicated design

and implementation. The AWS IoT platform was integrated into the implementation for Simple Devices

and acts as the foundation, enabling the communication between customers and devices. Rich Devices

can create a video stream and have a control mechanism, for example. These devices are considered

to be the more complex and demanding sensors and the implementation took into consideration the

fact that it is possible to have multiple customers consume the video stream, but only one being able to

control the sensor at any time.

46

Handler

Video

CAM CONTROL

Video

HTTP

Handler

Video

CAM CONTROL

Video

HTTP

Video

Video

MarketPlace

2

4

Handler

3 6

1

Video Broker

5

Figure 3.30: Multiple costumers consuming the video stream.

47

Chapter 4

Results and Evaluation

The platform was evaluated through quantitative and qualitative methods. The implementation of use

cases and the deployment of several sensors constitute the qualitative method. The platform must

be able to receive data from different types of sensors, ranging from simpler sensing devices (e.g.,

temperature sensors) to more complex sensors (e.g., video camera). The quantitative evaluation was

performed through various parameters, including the time delay between the instant from which the

sensor data is sent, to the access time of the consumer. Additionally, an evaluation was performed

regarding the overhead of finding the sensors and retrieving the sensor data from the marketplace

versus a direct interaction.

The hardware used in the following experiments is:

• Raspberry Pi 3 model B with Raspbian OS, a Quad Core 1.2GHz Broadcom BCM2837 CPU, 1GB

RAM and Internet access via WiFi;

• The temperature sensor (simple sensor) was the CPU temperature sensor of the Raspberry Pi 3

model B;

• The IP controllable Web Camera (rich sensor) was a Logitech Quickcam Orbit AF connected to

the Raspberry Pi 3 model B via USB;

• The IoT Marketplace was hosted on a Windows OS machine with a Quad Core Intel Core i5-

6300HQ CPU, 8GB RAM and Internet access via WiFi;

• The broker module (Video/Data broker and handler) was hosted on a regional (UK based) AWS

Elastic Compute Cloud virtual machine with Ubuntu OS, 1 core Intel Xeon virtual-CPU, 1GiB RAM

and Low-to-Moderate Network Performance.

4.1 Single Data Measurements

This experiment consists of scheduling three hundred measurements to the same sensor and receiving

the respective data. In greater detail, the assessment is divided in three series of one hundred mea-

surements, respectively. For each series, the Marketplace is set up with a pool of sensors, buyers and

49

sellers. One buyer and one sensor are selected and each data request is originated from the same buyer

to the same sensor. In the Marketplace, each request is generated with a one-second interval. However,

from the sensor’s perspective, two consecutive data measurements are separated by a sixteen second

interval, i.e., the sensor measures the temperature every sixteen seconds.

This test can be broken down into five stages. In the first stage, the Marketplace processes the

purchase request, checks if the sensor exists, if the scheduling is possible, and communicates the pur-

chase to the Broker (Data Broker). In the second stage, the Broker receives and stores the information

sent from the Marketplace and creates one Handler for this request. In the third stage, the Handler

resumes, sends a state update to the sensor via the Shadow Thing (AWS IoT). While sending the state

update, the Handler subscribes to the sensor’s topic. Fourth stage, the Handler detects a message (in

the subscribed topic) and replicates this message to the buyer’s topic. The fifth stage is the reception of

the sensor data. The buyer is already subscribed to its own topic in order to receive the sensor data.

To better evaluate the response time and performance (amongst other parameters) of the Broker and

the Handlers, the requests are created with one second interval from each other. Since every measure-

ment is set to occur in the future, the Handler will be inactive until such time is reached. Consider a set

of one hundred requests: a data measurement is taken every sixteen seconds; thus, the first measure-

ment is scheduled to happen sixteen seconds in the future and the second is scheduled to occur sixteen

seconds afterwards. After ninety-nine seconds all Handlers would have been created. A Handler is de-

stroyed shortly after taking a measurement. Therefore, in ninety-nine seconds, six Handlers have been

destroyed. So, at one point, there will be ninety-four instances with allocated memory simultaneously.

This setting approximates a possible real scenario in which a sensor is highly requested.

Figure 4.1 illustrates the number of allocated Handlers simultaneously, for the first three measure-

ments, and for how long. Since, the Handler is destroyed after publishing the message to the buyer’s

topic, it may stay alive for a few seconds more, regarding the scheduled instant.

0

|

#1

#2

#3

48

| 32

|

16

| asdasdat [seconds]

Figure 4.1: Number of allocated Handlers and their lifetime.

Figure 4.2 presents the time difference between the reception time and the scheduled time of each

measurement. These values have a high accuracy, since the Marketplace (which originates the schedul-

ing timestamp) and the client (which originates the reception timestamp) share the same clock: both

programs were running on the same machine. The values shown were collected throughout the third

series of one hundred measurements. In this series, the mean value for the time difference is 1.23 sec-

onds. While analysing the graph, it is possible to observe that a considerable number of samples are in

the interval from 1 to 1.5 seconds. Moreover, only six samples are above the 2 second mark, being that

the maximum in the series is 3.44 seconds.

50

0 20 40 60 80 100

0.5

1.0

1.5

2.0

2.5

3.0

3.5

Single Data Measurement

Index

Tim

e D

iffer

ence

(se

cond

s)

Figure 4.2: Temporal discrepancy between reception and schedule times.

To evaluate the performance of the proposed solution, it is crucial to assess the time required to

process each purchase order or each message replication. Figure 4.3 presents the proportion of pro-

cessing time to the total time, identifying its components using different colors. The Marketplace takes

more than fifty percent, the Handler more than forty percent and the Broker consumes no more than one

percent. These values are in accordance with what was stated in Section 3.3. The Marketplace must

validate all the information in the purchase order and communicate with the Broker. The Broker, upon

receiving a request from the Marketplace, stores the information and creates the Handler. The Handler

must communicate with the AWS IoT to activate the sensor, receive the measurement and then send a

copy to the client. Therefore, it is expected the Marketplace and Handler require more processing time

than the Broker.

Table 4.1 presents the overall statistical results calculated from the data collected throughout this

experiment.

Considering that the Marketplace only finishes processing when it receives a reply from the Broker

and the Client only receives a message after it is published by the Handler, one would expect that a

high processing time in the Broker or in the Handler could interfere with the processing time of the

Marketplace or delay a message to the Client. Nevertheless, this hypothesis is not verified. Analysing

the collected data, a higher time difference is not due to a higher processing time in the Handler. The

variation in the time difference, for instance, may be caused by the processing inside the AWS IoT

platform itself.

In Table 4.1, the mean time difference is 1.35 seconds. Picturing a real scenario where a client

51

0%

25%

50%

75%

100%

Average

Per

cent

age

Broker

Handler

MarketPlace

Time Usage

Figure 4.3: Average time usage for each participant regarding a Data transaction

schedules a measurement from a sensor, this value does not represent an issue to the feasibility of

this service. From a set of three hundred scheduled measurements, only one measurement was not

received by the client, which demonstrates a high success rate.

4.2 Long Data Measurements

This experiment is designed to analyse the performance of the implemented solution when it executes

one of the main operations. The procedure consists in purchasing real-time sensor data during a time

interval. In greater detail, the assessment is divided in three series of forty-five minutes, respectively.

For each series, the Marketplace is set up with a pool of sensors, buyers and sellers. Five buyers

Market Broker Handler Time Difference

Mean 1.05E-01 1.66E-03 8.68E-02 01.35

Standard Deviation 2.09E-02 5.20E-04 4.72E-03 00.63

95% Confidence Interval 2.37E-03 5.88E-05 5.34E-04 00.07

Maximum 3.38E-01 9.00E-03 1.00E-01 03.60

Minimum 9.87E-02 1.37E-03 8.00E-02 00.32

Table 4.1: Single Data Results: Processing time for each actor and overall time difference (seconds).

52

and one sensor are selected and each buyer generates a long data measurement request to the same

sensor. Those requests are created with a one-second interval. The sensor was configured to measure

and publish data to its topic every thirty seconds, even if there is no one listening. Considering that

the sensor does not have the ability to know if there are any subscribers to its topic, it will publish data

continuously. Each buyer purchases forty-five minutes of subscription to the sensor’s topic. Bearing in

mind the thirty-second halt at the sensor, it is expected that each buyer obtains ninety measurements.

To approximate this experiment to a potential real scenario, instead of having only one buyer listening to

the sensor’s topic, all five buyers are listening to the topic simultaneously.

This scenario can be broken down in four stages. In the first stage, the Marketplace processes the

purchase request by validating all the information from the buyer, the seller, the sensor and the request

itself. This stage culminates by sending the request to the Broker. In the second stage, the Broker

receives and stores the information, to subsequently create one Handler. In the third stage, the Handler

resumes its activity, then subscribes to the sensor’s topic and replicates any message to the buyer’s

topic. The last stage is dedicated to the reception of the sensor data through the subscription of the

buyer’s topic.

Throughout the experiment, each client, stores all received messages and generates a timestamp

for the instant of reception of those messages. As stated on Section 3.3, one message contains the

measured temperature and the timestamp of the measurement. The sensor is a Raspberry Pi, wherein

the time and date are configured with the default specifications. All five Clients were running on the

same machine, thus having the exact same clock. Bearing in mind that clients and sensor were on

different machines, therefore not being completely synchronized with each other, it is considered a

1 second maximum desynchronization. This 1 second is the maximum desynchronization between

the client’s clock and the sensor’s clock. Through visual inspection and some experimentation, the

desynchronization was always less than 1 second.

Even though it is not possible to guarantee total synchronization, Figure 4.4 presents the time differ-

ence between instant of reception (mean) and the instant of measurement.

These samples were collected during one of the three series from this experiment. It is possible to

observe that, on only two occasions, the delay between the measurement and reception is greater than

0.6 seconds, wherein the maximum time difference is 0.93 seconds. Also, more than 60 percent of the

samples are below the mean value. Even considering a desynchronization of 1 second (the sensor’s

clock is 1 second ahead of the actual time), the mean time difference, in that situation, is 1.47 seconds.

In a real scenario, a person intends to receive every measurement taken by the sensor in real-time, i.e.,

with a delay small enough to satisfy most applications.


experiment.

As discussed in Section 4.2, the results do not support a causality relationship between a higher

processing time and subsequent delays when receiving a message. In this experiment, the mean pro-

cessing time of the Marketplace and the Broker are similar to the values from Table 4.1. However, the

Handler has a smaller processing time when compared to the previous experiment. In this experiment,

53

0 20 40 60 80

0.5

0.6

0.7

0.8

0.9

Long Data Measurement

Index

Tim

e D

iffer

ence

(se

cond

s)

Figure 4.4: Temporal discrepancy between reception and schedule times.

the Handler subscribes the sensor’s topic and publishes to the buyer’s topic. Furthermore, the Handler

must also alert the sensor. This additional task explains the extra processing time.

4.3 Video and Control

This experiment is designed to analyse the performance of the implemented solution when it executes

one of the key operations. The scenario consists in watching a video stream from a video camera whilst

having the ability to control the camera in real-time. Contrary to previous experiments, this assessment

is aimed at more complex sensing devices, e.g., devices capable of creating a video feed and accepting

control requests. This assessment is divided in three series of one minute, respectively. In each series,

Market Broker Handler Time Difference

Mean 1.06E-01 1.44E-03 8.67E-05 00.51

Standard Deviation 7.36E-03 7.10E-05 1.01E-05 00.10

95% Confidence Interval 3.73E-03 3.59E-05 5.41E-07 00.01

Maximum 1.30E-01 1.63E-03 1.68E-04 01.49

Minimum 9.98E-02 1.37E-03 7.20E-05 00.42

Table 4.2: Long Data Results: Processing time for each actor and overall time difference (seconds).

54

the Marketplace is set up with a pool of sensors, buyers and sellers. One buyer and one video camera

are selected to perform this test. The buyer generates one video and control request directed to the

selected camera. This request contains the start time and end time, among other specifications. When

the time limit is reached, the video stream ends and the buttons no longer control the video camera.

This test can be broken down in five stages. In the first stage, the Marketplace processes the

purchase order, through the validation of all the information contained in the order. This stage culminates

by sending the specifications to the Broker (Video Broker). In the second stage, the Broker receives and

stores this information, then, creates one Handler to manage the request. Subsequently, the Broker

communicates with the Marketplace to send crucial information. In the third stage, the Handler resumes

its activity and waits for instructions. In the fourth stage, the client opens the URL generated by the

Marketplace. This action triggers a response in the Handler, which creates the video tunnel and opens a

line of communication with the camera’s control module. Finally, in the fifth stage, the client presses the

control buttons on the webpage, the web browser communicates with the Handler, and the latter sends

a movement order to the camera’s control module.

To evaluate the performance several timestamps, processing times and messages were collected

throughout this experiment. In order to increase the accuracy of this experiment, the agent that gen-

erates the message (to control the camera) and sends it to the Handler is replaced by an application

designated Postman. This application issues the control order, an HTTP request, to the Handler. Post-

man runs a set of four HTTP requests seventeen times. Two consecutive requests are separated by

five-hundred milliseconds. This configuration results in a total of sixty-eight instructions during the one-

minute test period.

For every control order issued, the Postman determined the response time, i.e. the time required to

receive the HTTP reply from the Handler, considered to be the total time. The Handler calculated the time

necessary to create and communicate the control message to the control module of the video camera,

designated TCP, and estimated the total time required to process the request, designated Processing.

This time includes a simple authentication verification, the TCP time and the HTTP reply to the client

(Postman). Figure 4.5 presents the total time and the fractions due to each component. Approximately

sixty percent of the total time is due to the Communication component. This element is considered to

be the time spent outside the Handler and the Postman. Analysing the Handler, the TCP component is

responsible for nearly ninety-eight percent.


experiment. The column Aggregate represents the total time registered in the Handler, i.e., the time

required by the TCP communication plus the processing time. The maximum values registered for

both the Postman and the Handler (Aggregate) occur in the first control order. According to [21] the

one-way latency for interactive video applications should be no more than 150 milliseconds. Hence,

two-way latency should be no more than 300 milliseconds. The latency measured corresponds to the

processing and reply of the control request, thus a two-way latency with a mean value of approximately

200 milliseconds. This result is acceptable, considering that the two-way latency should be less than

300 milliseconds. The mean value required for the Handler to fully process a request in approximately

55

0%

25%

50%

75%

100%

Total Time Handler Time

Average

Per

cent

age

Communication

Handler

Processing

TCP

Time Usage

Figure 4.5: Average time usage for each component regarding a Video and Control transaction.

Aggregate TCP Postman

Mean 5.73E-02 5.65E-02 2.01E-01

Standard Deviation 3.44E-03 1.61E-03 1.28E-02

95% Confidence Interval 4.72E-04 2.21E-04 1.76E-03

Maximum 8.23E-02 6.48E-02 2.84E-01

Minimum 5.39E-02 5.275E-02 1.93E-01

Table 4.3: Video and Control Results: Expended time in the TCP connection and overall two-way latency(seconds).

80 milliseconds, thus having some impact in the delay between instructing the camera to move and

observing that movement in the video stream.

4.4 Discussion

While developing and executing the two experiments for the Simple Device, the sensor sent more than

one thousand messages through the AWS IoT. Only five messages were not received by the client, a

0.005 fail ratio. Overall, the implemented solution proved to be: robust, when the customer attempted

to open the video URL in another computer or browser, the Video Handler detected this abnormal

behaviour and cancelled the request; effective, considering a 0.005 fail ratio in data transaction tests;

fast, regarding the low-delay visually observed during the video and control experiments and the delay

of the data transactions. The results are promising, the mean delay between measuring and receiving

is 1.54 seconds for the Single Measurements and 0.51 seconds (or 1.51 seconds, considering a 1

second desynchronization) for the Long Data Measurements. In the final experiment (Rich Device),

both the video stream and the control module proved to be responsive and accurate. The results were

56

obtained using a personal computer to run the Marketplace, a regional (UK based) virtual machine from

Amazon Web Services to host the broker module and Amazon Web Services IoT platform to support

the Device module. Configuring dedicated machines to host the marketplace module and the broker

module could improve the results. Optimizing AWS IoT platform to the specific requirements of the

Sensor Data Market may decrease the delay and the setup time necessary to establish the connections

to the platform. Nevertheless, the delay between receiving a data measurement and the instant of the

measurement itself may not be an issue for some applications or scenarios. For instance, a network of

constrained sensing devices (sensors with low processing power and limited connectivity) collects data

every two hours. The data is stored locally in the devices and uploaded once a day to a gateway. After

receiving the data measurements of all sensors, the gateway publishes it to the AWS IoT platform. In

this scenario the delay should not be an evaluation parameter.

57

Chapter 5

Conclusions

Through the research performed in Chapter 2 it was found there is no platform or service capable of

allowing two participants to exchange sensor data. We propose a Sensor Data Market for the Internet

of Things. This solution aims to be the intermediary between data suppliers and data consumers. Thus,

implementing a S2aaS model, where a sensor is available to any customer, in a service-like structure.

The Amazon Web Services Internet of Things platform was integrated in the proposed to solution,

due to its architecture and implemented protocols, such as MQTT and HTTP. The AWS IoT infrastructure

regarding the MQTT protocol is the foundation of the Simple Device implementation.

As discussed in Section 2.1, the devices were divided into two categories: Simple Device and Rich

Device. Two sets of tests were designed and conducted, one set for the Simple Device and the other for

the Rich Device. Nevertheless, both sets aimed to collect information in order to evaluate the proposed

solution.

Regarding the Simple Devices, the tests can be divided in two categories: Single Data tests, in which

a customer purchases an individual data measurement a at specific time instant; and Long Data tests, in

which a customer purchases all scheduled measurements during a window of time. Both tests scenarios

aim to replicate real circumstances. A Raspberry Pi 3 Model B was configured to collect the temperature

of its CPU, thus simulating an actual temperature sensor. Considering the Single Data tests, the results

show a mean time difference of 1.35 seconds between the schedule time and the instant of reception

and a maximum delay of 3.60 seconds. Thus, all received messages (ninety-nine out of one hundred)

were obtained in less than five seconds. The Long Data test results show a mean time difference of

0.51 seconds and a maximum of 1.49 seconds. Again, all received data were obtained in less than five

seconds.

Regarding the Rich Devices, the tests consisted in purchasing control of a sensing device during a

time interval. This test aims to replicate a real scenario in which a customer purchases both, the control

and video stream of the camera for period of time. A controllable camera was connected to a Raspberry

Pi 3 Model B, thus simulating an actual IP controllable camera. The results show mean two-way latency

of approximately two hundred milliseconds. Throughout this experiment there was no critical failures,

i.e., there was no interruption in the video stream nor regarding the control of the camera.

59

Considering the results presented above as well as the use cases described in Section 3.1, we

conclude that the platform designed and implemented in this work is a valid prototype/proof-of-concept

of a Sensor Data Market for the Internet of Things.

5.1 Future Work

Some points can be highlighted to improve the current solution:

• Study current business models and analyse possible trends to implement a successful business

model;

• The implemented solution cannot create a virtual representation of a real sensor in the AWS IoT

platform, namely a Thing. Upon the registration of a new device the platform must associate a

Thing to that device;

• As presented in Section 2, the security was considered to be an add-on in the scope of this solution.

Nevertheless, this issue is very important and should be addressed;

• Add privacy-preserving features, both to protect privacy of buyers and sellers, but also the privacy

of people potentially surveilled by such a system;

• Provide the customer with information regarding the performance of a device. Gather and provide

information, such as, the measurement’s accuracy or the time difference between the instant of

measurement and instant of reception of sensing data;

• Implementing more types of devices will widen the range of sensors that can be integrated in the

marketplace, thus strengthening a future Sensor Data Market;

• Design and implement a web interface to interact with the platform will enrich the current solution;

• Implement a database to store information of customers, accounts, transactions, devices. It could

also be used to store data collected from sensing devices;

• Creating and presenting new scenarios, such as the ones discussed in Section 1, may demon-

strate, even more, the importance and necessity of a Sensor Data Market. Furthermore, it may

attract clients and companies to join the platform;

• Consider the integration of a resell business model. For instance, a customer (customer X) pur-

chases all video images of a camera or all data measurements of a sensor in a time interval.

Afterwards, another customer (customer Y) intends to acquire the data sold to customer X. Thus,

the marketplace should verify if customer X wants to sell his data and perform a new data trans-

action. Furthermore, customer X purchases the right to all data measurements from a sensor in

a time window, i.e., no other customer is able to purchase data from that sensor during that pe-

riod directly from the owner of the sensor. Therefore, the marketplace should verify if customer X

60

wants to sell his data. If so, the data flow should be optimized to deliver data to every customer in

real-time;

• Integration with payment systems, both traditional currency, and crypto currencies.

61

Bibliography

[1] S. Haller, S. Karnouskos, and C. Schroth, The Internet of Things in an Enterprise Context.

Berlin, Heidelberg: Springer Berlin Heidelberg, 2009, pp. 14–28, Accessed: 3 July 2018. [Online].

Available: http://dx.doi.org/10.1007/978-3-642-00985-3 2

[2] C. Perera, A. Zaslavsky, C. H. Liu, M. Compton, P. Christen, and D. Georgakopoulos, “Sensor

search techniques for sensing as a service architecture for the internet of things,” IEEE Sensors

Journal, vol. 14, no. 2, pp. 406–420, Feb 2014.

[3] X. Sheng, J. Tang, X. Xiao, and G. Xue, “Sensing as a service: Challenges, solutions and future

directions,” IEEE Sensors Journal, vol. 13, no. 10, pp. 3733–3741, Oct 2013.

[4] P. J. Leach, T. Berners-Lee, J. C. Mogul, L. Masinter, R. T. Fielding, and J. Gettys,

“Hypertext transfer protocol – HTTP/1.1,” Accessed: 3 July 2018. [Online]. Available:

https://tools.ietf.org/html/rfc2616

[5] Z. Shelby, K. Hartke, and C. Bormann, “The Constrained Application Protocol (CoAP),” Accessed:

3 July 2018. [Online]. Available: https://tools.ietf.org/html/rfc7252

[6] T. Jaffey, “MQTT and CoAP, IoT Protocols,” Accessed: 3 July 2018. [Online]. Available:

https://eclipse.org/community/eclipse newsletter/2014/february/article2.php

[7] A. B. Zaslavsky, C. Perera, and D. Georgakopoulos, “Sensing as a service and big data,” CoRR, vol.

abs/1301.0159, 2013, Accessed: 3 July 2018. [Online]. Available: http://arxiv.org/abs/1301.0159

[8] C. Perera, A. Zaslavsky, P. Christen, and D. Georgakopoulos, “Sensing as a service model for smart

cities supported by internet of things,” Trans. Emerg. Telecommun. Technol., vol. 25, no. 1, pp.

81–93, Jan. 2014, Accessed: 3 July 2018. [Online]. Available: http://dx.doi.org/10.1002/ett.2704

[9] I. Corp., “Getting started with Watson IoT Platform,” Accessed: 3 July 2018. [Online]. Available:

https://console.ng.bluemix.net/docs/services/IoT/index.html

[10] Microsoft, “Azure solutions for Internet of Things,” Accessed: 3 July 2018. [Online]. Available:

https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-what-is-azure-iot

[11] A. W. Services, “What Is AWS IoT? - AWS IoT,” Accessed: 3 July 2018. [Online]. Available:

http://docs.aws.amazon.com/iot/latest/developerguide/what-is-aws-iot.html

63

http://dx.doi.org/10.1007/978-3-642-00985-3_2



https://eclipse.org/community/eclipse_newsletter/2014/february/article2.php

http://arxiv.org/abs/1301.0159

http://dx.doi.org/10.1002/ett.2704

https://console.ng.bluemix.net/docs/services/IoT/index.html

https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-what-is-azure-iot

http://docs.aws.amazon.com/iot/latest/developerguide/what-is-aws-iot.html

[12] N. Woolf. (2016) DDoS attack that disrupted internet was largest of its kind in history, experts say.

Accessed: 3 July 2018. [Online]. Available: https://www.theguardian.com/technology/2016/oct/26/

ddos-attack-dyn-mirai-botnet

[13] OpenStack. (2016) Open Source Cloud Computing Software. Accessed: 3 July 2018. [Online].

Available: https://www.openstack.org/

[14] OpenIoT. (2016) The Open Source Internet of Things Project. Accessed: 3 July 2018. [Online].

Available: https://github.com/OpenIotOrg/openiot

[15] C. Chang, S. N. Srirama, and M. Liyanage, “A service-oriented mobile cloud middleware frame-

work for provisioning mobile sensing as a service,” in 2015 IEEE 21st International Conference on

Parallel and Distributed Systems (ICPADS), Dec 2015, pp. 124–131.

[16] E. Montserrat and F. Xhafa, “Sensors as a service in the cloud,” Master’s thesis, Universitat

Politecnica de Catalunya, Catalunya, Spain, April 2015.

[17] Oracle, “Java programming language,” Accessed: 3 July 2018. [Online]. Available: https:

//docs.oracle.com/javase/8/docs/technotes/guides/language/index.html

[18] T. Bray, “The JavaScript Object Notation (JSON) Data Interchange Format,” Accessed: 3 July

2018. [Online]. Available: https://tools.ietf.org/html/rfc8259

[19] A. W. Services, “AWS IoT Device SDK - Amazon Web Services,” Accessed: 3 July 2018. [Online].

Available: https://aws.amazon.com/iot/sdk/

[20] P. S. Foundation, “Welcome to python.org,” Accessed: 3 July 2018. [Online]. Available:

https://www.python.org/

[21] T. Szigeti and C. Hattingh, End-to-End QoS Network Design: Quality of Service in LANs, WANs,

and VPNs. Cisco Press, 2004.

64

https://www.theguardian.com/technology/2016/oct/26/ddos-attack-dyn-mirai-botnet

https://www.theguardian.com/technology/2016/oct/26/ddos-attack-dyn-mirai-botnet

https://www.openstack.org/

https://github.com/OpenIotOrg/openiot

https://docs.oracle.com/javase/8/docs/technotes/guides/language/index.html

https://docs.oracle.com/javase/8/docs/technotes/guides/language/index.html


https://aws.amazon.com/iot/sdk/

https://www.python.org/

Sensor Data Market for the Internet of Things · Sensor Data Market for the Internet of Things...

Documents

Transcript of Sensor Data Market for the Internet of Things · Sensor Data Market for the Internet of Things...