DEGREE PROJECT, IN , FIRST LEVELMECHATRONICS

STOCKHOLM, SWEDEN 2015

Digital Lock

MÄNNISKODETEKTERANDE YTTERLÅS

ALEXANDRA TANG, JONATHAN HYTÖNEN

KTH ROYAL INSTITUTE OF TECHNOLOGY

SCHOOL OF INDUSTRIAL ENGINEERING AND MANAGEMENT

Digital Lock

Human Detecting Outdoor Lock

ALEXANDRA TANGJONATHAN HYTÖNEN

Bachelor’s Thesis in Mechatronics

Supervisor: Didem GürdürExaminer: Martin Edin GrimhedenApproved: 2015-05-27

TRITA MMK 2015-10 MDAB 063

AbstractThis thesis considers a human detecting outdoor lock which can be controlledfrom a distance with the use of a web browser. Focus was on optimizing detec-tion of possible visitors, i.e. eliminating non-human objects and people passingby with the use of a passive infrared sensor, an ultrasonic sensor and capturethem on camera. A proposal of an algorithm and an actual implementation ofit was made, taking the time a person is in the range of the sensors into account.Moreover, the position of the sensors were determined as well as the positionand angle of the camera which took a photograph when a visitor was detected.The results indicated on accurate detection of motionless bodies, includingnon-humans, and an accurate non-detection of bypassing objects. However,the process time varied due to measurement errors by the ultrasonic sensor.Thus in order to detect possible visitors more precisely, methods of eliminatingnon-human objects and ultrasonic sensor errors must be investigated further.

iii

SammanfattningMänniskodetekterande ytterlås

Arbetet syftar att utveckla ett ytterlås som upptäcker människor i dess närhet.Låset ska vidare kunna kontrolleras från godtyckliga avstånd med hjälp av enwebbläsare. Fokus under arbetet var att optimera upptäckandet av potentiellabesökare och på så sätt eliminiera förbigående och icke-mänskliga objekt medhjälp av en passiv infraröd sensor och en ultraljudssensor. Detta genomfördesgenom implementering och optimering av en algoritm som tog hänsyn till män-niskans position från sensorerna och antalet millisekunder han eller hon måstevara stillastående för att räknas som en besökare. Även sensorernas positionerbestämdes samt positionen och vinkeln på den kamera som användes för attfotografera den upptäckta människan. Resultatet tydde på ett korrekt upptäc-kande av stillastående objekt (dock även icke-mänskliga) och att förbigåendeobjekt inte upptäcktes. Tiden för att fullborda processen, från att en människastår framför sensorerna till att en bild visas på hemsidan, varierade på grundav felmätningar av ultraljudssensorn. Detta innebär att fortsatta studier påmetoder att eliminera icke-mänskliga objekt samt ultraljudssensorns felavläs-ningar måste utföras för att upptäckandet av potentiella besökare ska vara merkorrekt.

v

Preface

We want to thank our fellow classmates for valuable feedback on the report, StaffanQvarnström for providing us with materials, Sami Camacho for helping us with theserver and our supervisor Didem Gürdür for helping and guiding us from start tofinish.

Alexandra Tang, Jonathan HytönenStockholm, May, 2015

vii

Contents

Abstract iii

Sammanfattning v

Preface vii

Contents ix

Nomenclature xi

1 Introduction 11.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.4 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.4.1 Human Motion Detection . . . . . . . . . . . . . . . . . . . . 31.4.2 Photographing the Visitor . . . . . . . . . . . . . . . . . . . . 51.4.3 Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.4.4 Physically Controlling the Existing Mechanical Lock . . . . . 6

2 Theory 72.1 Using PIR Sensor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.2 Using Ultrasonic Sensor . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.2.1 Using Dual-Technology Detectors . . . . . . . . . . . . . . . . 8

3 Demonstrator 113.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.2 Electronics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

3.2.1 Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.2.2 Camera Module . . . . . . . . . . . . . . . . . . . . . . . . . 143.2.3 Server on Raspberry Pi . . . . . . . . . . . . . . . . . . . . . 143.2.4 Servo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.3 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.3.1 Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

ix

3.4 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.5.1 Sensor Placement and Time Variable x . . . . . . . . . . . . . 203.5.2 Human Detection Test . . . . . . . . . . . . . . . . . . . . . . 203.5.3 Camera Angle and Position . . . . . . . . . . . . . . . . . . . 213.5.4 Measurements of Process Time . . . . . . . . . . . . . . . . . 22

4 Discussion and Conclusions 234.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234.2 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

5 Recommendations and Future Work 275.1 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Bibliography 29

Appendices

A Code for the Human Motion Detection Algorithm 33

B Case Study 1: Results 35

C Draft of Servo Case 37

x

Nomenclature

SymbolsSymbols Descriptionγ Heat capacity ratio [Dimensionless]kHtz KilohertzkΩ KiloohmM Molar mass [kg/kmol]m Metersms MillisecondR Gas constant [J/mol·K]x Time a person has to stand in front of the sensors to be

counted as a visitor [milliseconds]

AbbreviationsAbbreviation DescriptionGPIO General Input/OutputIDE Integrated Development EnvironmentLE Low EnergyNFC Near Field CommunicationPIR Passive Infrared or Pyroelectric infraredPWM Pulse-Width Modulation

xi

Chapter 1

Introduction

In this section, the background, purpose, scope and method of the report will bedescribed.

1.1 Background

Automation is defined as making a process operate by machines or computers in or-der to reduce the amount of work done by humans [Cambridge Dictionaries Online,2015]. With respect to door locks, automation is generally reserved for the purposeof providing accessibility for people, eliminating or reducing the need of a physicalkey.

Being able to lock or separate belongings from others is an essential necessity. Thelock industry is currently heading towards a digitalization [Pulford, 2007], whereways of identification are being explored to follow the trending concept of smarthomes and the ’Internet of Things’ [Greenough, 2014].

HID Global, a division of the Swedish-owned manufacturer of mechanical locksAssa Abloy, have been working on the development of digital key software aimedfor smart phones, unlocking doors with NFC (Near Field Communication), a short-range wireless technology. HID Global completed a month-long trial of the tech-nology in collaboration with 32 students at Arizona State University. The studyrevealed that there was a need for digital keys. However, the potential expenseof the keys caused by the cost of owning, maintaining a smartphone and licensingthe necessary software made the replacement of mechanical keys with digital onesdifficult in practice [Woyke, 2011].

Several concepts of a wireless lock have been released for the consumer market.Yale, an Assa Abloy group brand, has developed Yale Doorman, a door lock enablinglock-control via key tab, code or by using Wi-Fi through the smart lock module andmobile application provided by Verisure [Assa Abloy, 2015, Verisure, 2015]. Other

1

CHAPTER 1. INTRODUCTION

developed digitalized locks include Bolt by Lockitron [Lockitron, 2015], AugustSmart Lock by August [August, 2015], Goji Smart Lock [Bielet Inc, 2015] and KevoLock by Kwikset [Kwikset, 2015], shown in Figure 1.1.

Figure 1.1. Current smart locks. From left: Yale Doorman, Bolt, August, Goji andKevo. Taken from the homepages of each lock [2015].

The locks mentioned above use different wireless technologies such as Wi-Fi, NFC,Bluetooth LE (Low Energy) and RFID (Radio-frequency identification) to commu-nicate with the lock, often using several of them in combination.

The Goji Smart Lock has another feature added, making it stand out among theothers. It has a camera built into the front end of the lock that automatically takesphotographs and sends them to the user [Bielet Inc, 2015].

The possibility to unlock from distance can be applied in various areas, fulfilling awide range of needs. For example it can be used as an outdoor lock by the averageperson enabling others to get access to the accommodation even though the homeowner is not at home. It could also be used by bedridden or disabled people, allowingthem to unlock for visitors without help. However, this raises the importance ofbeing able to know whom you are unlocking for.

1.2 PurposeThe purpose of this thesis is to control a mechanical lock from anywhere in theworld using a web browser. To know when to open the door and for whom, themodelled unit will be able to detect a possible visitor, capture a photograph of thatperson and send it to a web browser. Thus, this thesis incorporates server-clientcommunication, servo, sensor technology and camera function.

The research aspect of this thesis is on optimizing detection of house-visitors. Thiswill be carried out by the implementation of an algorithm involving a PIR (PassiveInfrared) sensor and an ultrasonic sensor and by determining the optimal position ofthe sensors and the camera. Emphasis will be on detecting and capturing a pictureof possible house-guests and eliminating by-passers and non-human objects.

2

1.3. SCOPE

1.3 Scope

During the development of the unit, the durability of the mechanical lock is notconsidered since focus is on the concept of a digitalized lock and human detection.The digital security aspect of protecting the system from hackers and fail-safe so-lutions are also excluded due to the same reason.

Methods to achieve human motion detection are limited to a PIR sensor and anultrasonic distance sensor only, thus no other electronic hardware or software is in-cluded to solve this matter. When optimizing detection of a human being, focus ison analysing placement and angle of camera, placement and software implementa-tion of the sensors in use. These are then tested, evaluated and determined by realenvironment studies. The studies are conducted with one participant at the timewithin the range of the sensors. This result could be different with several objectsin front of the sensors and it is therefore not valid in that case. No modificationsin the hardware is dome.

The human motion detection methodology is designed for detecting human beingsat age seven and above. Children below seven years and non-human living objects,such as animals, are not included due to the unlikeliness of them wanting to entera residence on their own.

The digital lock is developed with a targeted area of use consisting of outdoor locksonly.

1.4 Method

In this section, the four main functions and the approach used to achieve them areexplained.

1.4.1 Human Motion Detection

A human motion detection system has to be developed to answer the research ques-tion. The human motion detection system consists of a PIR sensor, an ultrasonicsensor and a camera. The PIR sensor detects changes in infrared radiation, possi-bly indicating human motion, thus activating the ultrasonic sensor measuring thedistance to the object. By adding a motion sensor, the unlikeliness of detecting apassing non-human object e.g. a passing car, might decrease. This order of thesensors was chosen to save energy since the PIR sensor is a passive sensor whilstthe ultrasonic is an active.

3

CHAPTER 1. INTRODUCTION

Defining a Visitor

To separate people passing by from actual visitors, the sensors are programmed toreact to people of a certain height standing within a specific distance from the doorfor a certain amount of time. Statistics show that the mean length for men in Swe-den in 2010 was 1.79 m from age 16 and above and 1.66 for women. The same studyshows that 90 % of the men were shorter than 1.88 m [Statistiska Centralbyrån,2013]. The mean length of children of age seven in Sweden was 1.24 m [Werneret al., 2006]. To have a safety margin as well as taking the people who are deviatingfrom the average length into account, the sensors are positioned and programmedto react to objects with the height of 1.0 to 2.0 m.

Testing was done to determine at which distance visitors stand from the door. Thesensors will then ignore objects beyond that distance. Five people were instructedto knock on a door without further instructions. This was carried out ten timeseach to get a range of distances since a person will most likely not stand at theexact same position all of the times. The distance from the test person to the doordiffered and were approximately within a range of 0.5 to 1.0 m. Following thisresult, the distance was set to 0.0 to 1.0 m. This and the detection ranges of thePIR and ultrasonic sensor gives us a detection area as illustrated in Figure 1.2.

Figure 1.2. The detection area (red color within 1 meter) defined by the range ofthe PIR sensor (triangle in green), the ultrasonic sensor (rectangle in blue) and themaximum detection distance.

A by-passer is anybody passing by the door without the intent of going in.

Detecting Human Motion

To analyze the human motion detection system, a real environment study was con-ducted in which the proposed algorithm was evaluated and modified to minimizethe time x ms a person had to stand still in front of the sensors in order to countas a house-guest. The tests also helped to determine the placement of the sensors.

4

1.4. METHOD

Two test persons of approximately 1 m and 2 m were part of this study, as well asa non-human object and a human crawling to resemble a person of 0.5 m. It wascarried out indoors with the sensors facing the windows, 2.5 m from the sensors tosimulate an outdoor environment. Each and every participant was performing thefollowing tasks:

1. Stand at a distance within the detection range (0.0-1.0 m) for x seconds

2. Walk past the sensors at a, according to them, normal walking speed

At first, every participant performed the tasks ten times each to give an idea ofwhich values of x were appropriate regarding the ability to separate visitors fromby-passers without leading to a relatively large process time. These values were thentested 100 times per participant to get a rate on the accuracy of ignoring by-passers.

1.4.2 Photographing the Visitor

The camera in this thesis is activated when the sensors detect a house-guest. A pho-tograph of the environment is then sent to the web browser through an establishedserver. This emphasizes the importance of accurately detecting possible visitorsand excluding by-passers as explained in Section 1.2, as well as taking a picture inwhich the face of the visitor is fully visible.

To identify the position and angle of the camera, a second study was conducted inwhich participants of the approximate heights of 1 m, 1.5 m and 2 m were askedto stand at a, according to them, natural distance in front of the door. This wascarried out once per participant for every variation in the position and angle of thecamera. These heights were chosen to cover the range in which the camera wassupposed to take a picture. The taken photographs were graded in a scale from 1to 3 according to the following:

1. No face in picture

2. Parts of face in picture

3. Fully visible face in picture

1.4.3 Server

Wi-Fi is used as wireless communication since it is desired to be able to control thelock from an arbitrary distance. To control it over Internet a server is set up to sendand receive data. If an already established server were to be chosen for uploadingthe taken photograph, i.e. Dropbox, it would not be possible to control the servowith the same server.

5

CHAPTER 1. INTRODUCTION

1.4.4 Physically Controlling the Existing Mechanical LockThis is made by a servo attached to the lock house and mounted on the door witha case for the servo.

6

Chapter 2

Theory

To date, various methods have been developed and introduced to detect human mo-tion. Some of the most used methods regarding sensors are described in this chapter.

2.1 Using PIR Sensor

PIR sensors detects sudden changes in radiation at wavelengths within the infraredspectrum, thus indicating motion in its vicinity. Bodies emit some low level radi-ation, the hotter the body, the more radiation. To increase its detection range afresnel lens is mounted on top of the sensor [Parallax Inc, 2012].

The sensor is composed of two elements, each of which is made of a crystallinematerial sensitive to IR. When a warm body passes by, e.g. a human, the emittedradiation is first intercept by one of the elements, causing a positive differential inthe sensor, turning into negative when the body leaves the area of which the sensorsare active. These pulses are used for detecting motion [Adafruit, 2014]. When thePIR sensor is detecting motion, a high signal is sent through its output pin whichcan be read by a micro-controller [Parallax Inc, 2012]. When the sensor is inactive,both elements are detecting the same amount of IR and thus not generating a pulse[Adafruit, 2014].

2.2 Using Ultrasonic Sensor

Ultrasonic sensors measure distances by dispatching a high frequency sound wave,usually around 40 kHz and registering the echo when it hits an object and bouncesback. By measuring the time it takes for the echo to reach the sensor the distancefrom the sensor to the detected object can be calculated with

distance = time · velocity of sound2 , (2.1)

7

CHAPTER 2. THEORY

where the denominator 2 is due to the fact that the echo signal has travelled backand forth [Monk, 2014].

The velocity of sound in an ideal gas varies depending on the temperature as seenin the following equation

v =√γRT

M, (2.2)

where γ is the ratio of heat capacity (1.4 for air), R is the gas constant (8.3145J/mol·K), T is the temperature in Kelvin and M is the molar mass (28.97 kg/kmol)[Young, 2013]. If air is considered ideal, the velocity of sound in 20 °C is 343 m/s.If the wrong temperature is used to calculate the distance this can lead to a relativeerror calculated with

Relative error = sound of speed used− real sound of speed

real sound of speed. (2.3)

This gives a relative error of 9 % and 3 % for -20 C respectively 40 C when thesound of speed in 20 C is used.

The amount of reflected ultrasonic sound from a person may vary depending onthe person’s clothing. A person wearing a flat hard belt buckle will most likely bedetected at further distances than a person with sound absorbent material, e.g. athick wool jacket [Bonar, 2012]. Observations have shown that occasional distancereadings may be incorrect due to the soft target nature of humans, even though themajority of reported readings are accurate [Bonar, 2012]. Non-smooth surfaces mayalso affect the results of measuring [Elec Freaks, nd].

Former research on human detection have been using ultrasonic sensors to detectmotion by continuously measuring distances to nearby objects thus detecting mo-tion when a sudden change in distance is measured [Faltpihl, 2012].

Some older intruder alarm systems uses a separate transmitter and receiver to de-tect motion. A transmitter sends pulses and a receiver listen and picks up thesepulses which have travelled through the area under protection. If there is somethingbetween the transmitter and the receiver, e.g. a moving object, a shift in the fre-quency will occur, also known as the Doppler effect, initiating an alarm condition[Raj et al., 2012].

2.2.1 Using Dual-Technology Detectors

Many current intruder alarms systems are often using PIR sensor combined withan ultrasonic sensor sensitive to Doppler frequency effect, as first presented by R.

8

2.2. USING ULTRASONIC SENSOR

McMaster [McMaster, 1987]. Hence, an alarm is triggered when the equipment de-tects a moving heat source and a moving object.

9

Chapter 3

Demonstrator

This chapter describes the process of developing the demonstrator and the results ofthe conducted real environment studies.

3.1 Problem FormulationSolutions are to be found to the following problems based on the desired functionsof the Wi-Fi controlled lock unit mentioned in Chapter 1.

• Optimize the PIR motion sensor and ultrasonic distance sensor (from hereon referred to as "the sensors" while mentioned together) in order to detecthuman motion within a range of 1.0 m.

• Activate a camera when the sensors have detected human motion.

• Set up a server.

• Create a web page from which the user can view the image and control theservo by communicating with the server.

• Control a servo with input from the server.

These problems are structured and narrowed down into three areas as shown inFigure 3.1.

Part 1 concerns the sensors and camera. The aim is to take a picture when thesensors detect a human. Part 2 consists of posting the picture to the server anddisplaying it to the user. Part 3 is the process of sending a signal to the servo whenuser input is entered through the browser.

11

CHAPTER 3. DEMONSTRATOR

Figure 3.1. Flowchart of the components, Part 1 in blue, Part 2 in red and Part 3in green.

3.2 ElectronicsThe Wi-Fi controlled lock has the following electronic components:

• Raspberry Pi B for controlling the sensors, camera and servo

• PIR sensor and ultrasonic sensor for human motion detection.

• Camera module for taking picture of the detected human.

• Raspberry Pi B+ hosting a server for incoming and outgoing information.

• Servo controlling the mechanical lock.

A Raspberry Pi B is used for providing the required logic for the sensors, cameraand servo.

At first the lock unit was modelled with the micro-controller Arduino Uno but thememory provided by it was insufficient to support all of the functions mentionedabove [Richardson and Wallace, 2012]. Therefore, a Raspberry Pi was chosen andused for controlling and operating the modeled lock unit in this thesis.

Raspberry Pi B is a computer with 512 GB RAM, a Broadcom BCM2835 processor,two USB ports, HDMI output and an Ethernet port. Because of its processor power

12

3.2. ELECTRONICS

and RAM it has no problems handling the operations needed for this project. It alsohas 17 GPIO (General-purpose input/output) pins out of a total of 26 pins, whichare used to connect the Raspberry Pi to the sensors and servo. To make the unitwireless an Edimax EW-7811Un wireless adapter is inserted in one of the USB ports.

3.2.1 Sensors

To optimize detection of a human being as described in Section 1.2, a Parallax PIRsensor (Rev A) is combined with an Ultrasonic Ranging Module HC-SR04.

PIR Sensor

To get a continuous trigger output mode where the sensor output remains HIGHwhen it is re-triggered repeatedly, i.e. as long as motion is detected, the jumper isset to H [Parallax Inc, 2012].

The PIR sensor is connected to Raspberry Pi according to Figure 3.2. The positivepin is connected to the 3.3 V power supply on pin 1 and the negative to ground onpin 6. The digital out pin on the PIR sensor is connected to GPIO 27 on pin 13.When the PIR sensor detects a change of infrared radiation, this pin is set to high.

Figure 3.2. Circuit of the PIR sensor.

13

CHAPTER 3. DEMONSTRATOR

Ultrasonic Sensor

Since the ultrasonic sensor requires a power supply of 5 V (pin 2) [Elec Freaks, nd]and the GPIO on the Raspberry Pi only handles voltage levels of at most 3.3 V, a1 kΩ resistor is placed between the echo pin on the sensor and the GPIO 18 on pin12 (see Figure 3.3). The trigger pin is connected to GPIO 17 on pin 11.

Figure 3.3. Circuit of the ultrasonic sensor.

3.2.2 Camera Module

To take a picture, a camera module for Raspberry Pi was chosen because it was easyto connect and control. It has a horizontal field of view of 53.50 +/- 0.13 degreesand a vertical field of view of 41.41 +/- 0.11 degrees and connected to the CSI buson the Raspberry Pi [Raspberry Pi Foundation, 2015]. The sensor is a OmniVisionOV5647 with 5 megapixel which is high enough resolution to capture details.

3.2.3 Server on Raspberry Pi

The server is created on a Raspberry Pi B+ since it is easy to set up and allowsprogramming in the same environment as the modelled lock unit.

The client, in this case the Wi-Fi Lock, will send the picture to the server whichwill display the picture in a browser.

14

3.2. ELECTRONICS

3.2.4 ServoA Parallax Standard Servo is used in this thesis. A servo is a motor with integratedgears and shaft that can rotate and be positioned with precision, most often be-tween 0-180 degrees. The position is controlled by pulses and is set by the lengthof every pulse. If a pulse is high for 1.0 ms the servo angle is set to 0 degrees, if itis high for 1.5 ms the angle is set to 90 degrees and if it is high for 2.0 ms the angleis set to 180 degrees. The servo expects to receive a pulse in an interval of at least20 ms [Monk, 2014].

This is suitable for controlling the mechanical lock since locking and unlocking isexecuted by rotating the lock to initial angle position (0 degrees) to end position(180 degrees) of the servo, as illustrated by Figure 3.4. In this thesis, rotating toend position signifies an unlocking movement.

Figure 3.4. The positions of the servo used for locking and unlocking.

The complete circuit of the hardware components is built as shown in Figure 3.5.

15

CHAPTER 3. DEMONSTRATOR

Figure 3.5. Schematics of all the electronic components.

3.3 Software

The Raspberry Pis used in this thesis both use Raspbian Kernel version 3.18 as theoperating system.

The chosen platform for programming the lock and the server was Node.js. Itmakes it possible to use JavaScript on the server-side as well which simplifies thecode [Joyent, Inc, 2015]. It is also possible to add libraries and import scripts byimporting modules [Haverbeke, 2011].

Several modules are used in this thesis in order to facilitate the programming;Socket.io, to send data bi-directionally in real-time between the lock and server,onoff, to control the GPIO pins, Pi-Blaster to control the servo and r-pi-usonic tocontrol the ultrasonic sensor.

16

3.3. SOFTWARE

The server hosts a web site written in HTML and CSS. The site displays the picturetaken by the camera as well as a button to control the servo. It is made accessiblefrom any network by port forwarding the IP address of the server.

3.3.1 AlgorithmsThe implementations of the human motion detection system and the establishmentof the server-client communication on the lock unit developed in this thesis aresolely written in JavaScript. The following algorithms were developed to ensurethat it would perform the necessary task.

Human Motion Detection

In order to send a picture of an actual visitor, the modelled lock unit must accu-rately distinguish a visitor from a passer-by in its vicinity. Not only does it have toeliminate non-human objects, but also people passing by.

In this thesis, a possible visitor is defined as a human who have stood in front of thedoor within a distance of 1.0 m (see Section 1.4.1) for at least x ms. The variablex affects the time it takes for the system to initiate a detection-alarm, the higherthe value, the more time. However, if it is too low, non-human objects might passas human. x will be selected according to the results of the studies that will beconducted with the modelled prototype, presented in Section 3.5.

When the PIR sensor outputs high from the GPIO input, i.e. probably sensinghuman motion, the ultrasonic sensor is triggered to measure the distance to thedetected object. The ultrasonic sensor transmits a pulse with an interval of at least60 ms to prevent incorrect results [Elec Freaks, nd] from GPIO output, thus sendingout p pulses within x seconds according to

p = x ms

60ms. (3.1)

A value of x below 120 ms will not be evaluated in this thesis as it will result in lesserthan two pulses, meaning that the developed algorithm for controlling whether aperson is standing still would not have any effect because one pulse would not beenough to determine if the object is passing by or not.

The receiver connected to the GPIO input listens for the echo and times how longit has been high, with which the distance can be calculated according to Equation2.1. If the detected object is within the required range, each and every one of thesepulses must indicate a distance of < 1 m, otherwise we will return to the PIR sensor.If these p pulses are within a distance of 1 m, the camera function is called and aphotograph is taken and sent to the server. The camera will thereafter stop taking

17

CHAPTER 3. DEMONSTRATOR

additional photographs if the ultrasonic sensor still measures an object within 1.0m. The idea is that multiple photos of the same person will not be taken. Theflowchart of the logic is described in Figure 3.6 below and the coding is presentedin Appendix A.

Figure 3.6. Flowchart of the logic.

Controlling the Lock

The lock is controlled through a button on the website. Depending on the buttonsstate, when clicked, the server will send input to the Raspberry Pi B to rotate theservo to unlock or lock the door.Changing servo angle is made by use of PWM (Pulse-Width Modulation) to controlthe width of the pulses. The PWM frequency is set to 50 Hz, resulting in pulsesevery 20 ms. The duty cycle is then calculated according to the following

duty cycle = desired width

20ms , (3.2)

18

3.4. HARDWARE

giving us a duty cycle equal to 5 % to achieve a servo angle of 0 degrees and 10 %to achieve 180 degrees. Hence, when an input 0 is received, the duty cycle changesto 0.05 and 0.10 when the input is 1.

3.4 Hardware

The final mechanical construction of the demonstrator was built using plywoodto resemble a door. Due to the difficulty to build and handle a real-sized door asmaller prototype in the size of 40x60x4 cm was built. A key was fastened to theservo which was fastened outside the door with a case made of polyurethane (fordraft see Appendix C). A mortice lock was installed in the door and holes weredrilled for the sensors, camera and key as seen in Figure 3.7.

Figure 3.7. View of the mechanical construction.

3.5 Results

In this section, the results of the implemented human motion detection method willbe presented, as well as the results of the modelled system in its whole.

19

CHAPTER 3. DEMONSTRATOR

3.5.1 Sensor Placement and Time Variable x

The ultrasonic sensor and PIR sensor where both positioned 1.0 m above the floorsince the ultrasonic sensor’s limited field of view only made it possible to measurethe distance of objects standing directly in front of it. This led to a variation in theobject width because it sometimes measured the lower body and other times thetop of the head. The PIR sensor on the other had a wide field of view but becauseit was limited by the ultrasonic sensor it was placed right on top of it at the centerof the door to get an even detection.

3.5.2 Human Detection Test

The results obtained from the analysis of the motion detection as a function of thedistance from the sensors and human height x are presented in Appendix B.

The study showed that the sensors activated the camera for all humans and non-human objects above the height of 1.0 m and within the detection range of 1.0m. By-passers and objects further away than 1.0 m from the sensors where notdetected at all regardless of the walking speed. Within 1.0 m the only by-passersdetected was when the ultrasonic was set to react to objects standing still for 120 ms.

Further statistical test with the six lowest values for x show that when the timewas set to 180 ms, equal to 3 pulse-samples by the ultrasonic sensor, 16 % of theby-passers where falsely detected as standing in front of the door. When increasingx to 240 ms (4 pulses) the false-positives decreased to 6 %, 300 ms to 3 % and360 ms to 2 %. At 120 ms, which was the lowest value of x as defined in Section3.3.1, the system interpreted 46 % of the participants passing as visitors. 100 %non-detection rate was reached with x set to 420 ms.

All of the participants in the tests walked with varying speeds and it was noticedthat it greatly affected the detection especially when x was in the lower range.

The time a person is directly in front of the sensor depending on the walkingspeed can be calculated with,

time in front of ultrasonic sensor = width of object

walking speed, (3.3)

The PIR sensor falsely detected motion repeatedly, which was corrected by the ul-trasonic sensor all of the cases (measured object was not within 1.0 m).

The camera took one photograph per person the majority of times, however, it didtake two or more than two a couple of times. When this occurred, the ultrasonicsensor stated that the object was more than 30 m away.

20

3.5. RESULTS

3.5.3 Camera Angle and Position

Figure 3.8 presents the summary statistics for the grading of the photographs.

Figure 3.8. Statistics on picture satisfaction when varying camera position andangle.

Figure 3.8 indicates that the most satisfactory pictures taken of people with a heightof 1-2 ms is when the camera is mounted 1.9-2.0 ms above ground and when theangle is set to 50 as shown in Figure 3.9, taking the large margin in the defineddetection height into account.

Figure 3.9. Position and angle of the camera.

21

CHAPTER 3. DEMONSTRATOR

3.5.4 Measurements of Process TimeThe time it took for the ultrasonic to detect a visitor and upload a picture on theweb browser after that the PIR sensor detected motion, was approximately thesame as the preset value of x in all test cases, with a standard deviation of a fewmilliseconds. However, the time it took for the sensors to actually detect a partici-pant, from the second it stood in front of the sensors, varied by approximately onesecond and increased with increasing x but without any apparent pattern. In allof the cases when the process was notably delayed, the ultrasonic claimed that theobject was further than 30 m away.

The servo reacted immediately when getting inputs from the web browser. Thereaction time was affected by the time the servo could switch from one position toanother rather than the system when continuously getting different inputs.

22

Chapter 4

Discussion and Conclusions

In this chapter, the results presented in Section 3.5 are discussed and summarized.

4.1 Discussion

The modelled system for detecting house-visitors worked as expected in terms ofdiscovering objects between approximately 1-2 m standing still within the prede-fined detection range and ignoring objects walking by. Due to the sensitivity of thePIR sensor, not only did it detect human objects but also non-human. In additionto this, the sensitivity may be troublesome in outdoor environments, false-positivelydetecting motion due to heat changes in the surroundings. This phenomenon wasnoticed in the test studies when it was facing the window and frequently indicatedmotion even though no object was put in the range of the sensor.

The measurement errors by the ultrasonic sensor were apparent in the test cases,affecting the system notably. It caused the camera to take more than one pictureper visitor at times. It also made the reaction time vary, thus making the processtime unpredictable. The instability of the sensor is prominent due to the fact thatthe algorithm implemented is depending on accurate measurements. Since the sys-tem developed in this thesis approves distances within a specified interval withoutcontrolling if there were any measurement errors within the interval, e.g. an object0.98 m from the sensors might have been measured as 0.51 m away, the instabilityof the ultrasonic sensor might have been more prevalent than shown by the resultsof the case studies.

Since the ultrasonic sensor is active and continuously sends out sound waves, prob-lems may occur when several are near each other. Thus possibly making it unreliablewhen installed in a neighbourhood, especially when two doors are facing each other.

The time the distance to the detected object had to be within 1.0 m to ensurethat the object was not passing by the sensor’s range could be minimized to 180

23

CHAPTER 4. DISCUSSION AND CONCLUSIONS

ms. However the rate of accurately eliminating people passing by was only 84 %,meaning that in 16 cases of 100 a person walking by will be interpreted as a visitor.Increasing the time lead to higher rate of accuracy, being 98 % at 360 ms (6 controlpulses by the ultrasonic sensor) and 100 % at 420 ms (7 pulses). The test resultsrevealed that the execution time for the whole process increased with increasing x,probably due to measurement errors since the system has to recount when it hap-pens. If 360 ms was to be chosen, the process time would decrease at the price ofdecreased rate of accuracy. Yet, since the lock unit will be operated for years wheninstalled, 2 % error rate will be apparent enough to make the whole system visiblyunstable and therefore less trustworthy. To minimize the risks for false-positives,the value of x was chosen to 420 ms.

Further research of the effects of walking speed and width of the person on the ul-trasonic detection revealed that the mean human walking speed for all pedestrianswas 1.44 m/s [Fitzpatrick et al., 2006] and mean waist circumference for men over20 years in the US was 100.9 cm [Fryar et al., 2012]. By dividing the circumferenceby π it gives us an approximate width of a person, 0.32 m. Using Equation 3.3 withabove statistics gives us a time of 0.223 ms. This means the ultrasonic sensor willinterpret the person as a visitor when set to react to objects standing in front ofit for 220 ms, leading to a false-positive. A possible explanation for the differencebetween this theoretically calculated value and the one seen in the results is thedifference in side length of a person. The waist is usually more elliptical than roundand this leads to a shorter width, also, some of the test persons where women withthinner bodies which explains why the time could be set as low as 180 ms.

Temperature changes were not taken into account during the tests since the testswhere done indoors but since detection range does not necessarily have to be within1.0 m, the errors mentioned in Section 2.2 may not be of great significance. If anexact range would be needed a temperature sensor could be used in combinationwith the ultrasonic sensor to determine the correct sound of speed.

4.2 Conclusions

The purpose of thesis was to optimize a human detection system on a Wi-Fi con-trolled outdoor lock and analyze if it was reliable detecting possible visitors. Thesystem was developed for the purpose of enabling the user to know whom it isunlocking for since it can be made from any distance. The system consisted of aPIR sensor, an ultrasonic sensor and a camera which was activated when a visitorwas detected. The proposed algorithm was taking the time a person was standingwithin 1.0 m in the range of the ultrasonic sensor into account to determine whetherthe person was a visitor or a person passing by.

The results of the conducted tests showed that the modelled human motion detec-

24

4.2. CONCLUSIONS

tion system worked well in terms of separating bodies standing still from bodiespassing by the predefined range of 1.0 m. However, the task to separate humanbeings from non-human objects was not accomplished. The camera position wasdetermined to 1.9-2.0 m above ground and was able to capture a photo of objectswith the height of 1.0-2.0 m. The ultrasonic sensor was determined to be at a dis-tance of 1.0 m above the ground and in the center of the door and the PIR sensoron top of it.

25

Chapter 5

Recommendations and Future Work

Recommendations and further improvements are highlighted in this chapter.

5.1 Recommendations

In order for the human motion detection method implemented in this thesis to fullyfunction as desired, ways of reducing the measurement errors by the ultrasonic sen-sor and reducing the sensitivity of the PIR sensor needs to be explored.

Further research is needed to evaluate if an ultrasonic sensor alone would be enoughto detect human motion if the sensitivity of the PIR sensor would turn out to benonadjustable to detect human beings only. Using the camera as a motion detectorin itself, possibly using OpenCV, could also be explored. This would minimize thestruggles of eliminating non-human objects and also allow for face recognition.

The system developed in this thesis should be tested with groups of people. If thelock were to be commercialized, it will most likely be mounted in varying environ-ments, e.g. sparsely-populated areas or crowded cities, being exposed to differentamounts of people.

Another possible area of future research would be to investigate the object’s angle infront of the ultrasonic sensor since it could affect the accuracy of the measurements.

5.2 Future Work

Instead of a mechanical lock and a servo the possibility to use an electric lock ora module that can be mounted to a current lock should be evaluated. The firstsolution would minimize the size of the lock body and make the servo unnecessarywhile the second solution has the benefit of making the digital lock compatible with

27

CHAPTER 5. RECOMMENDATIONS AND FUTURE WORK

current locks.

Another problem with the digital lock in this thesis is the lack of notification to theresident. In the future ways to notify the user should be found, e.g. through email,text message or app notification. A possibility would be to let the visitor activatethe camera by themselves with a button.

Lastly following the discussion in Section 4.1 future work should assess if thereactually is a need for a explicit human detection or if object detection is enough.

28

Bibliography

[Adafruit, 2014] Adafruit (2014). How PIRs Work. Available from: https://learn.adafruit.com/pir-passive-infrared-proximity-motion-sensor/how-pirs-work [cited 2015-03-21].

[Assa Abloy, 2015] Assa Abloy (2015). Yale Doorman. Available from: http://www.yale.se/sv/yale/yalese/yale-doorman/ [cited 2015-04-21].

[August, 2015] August (2015). Welcome to August Smart Lock. Available from:http://august.com/ [cited 2015-02-20].

[Bielet Inc, 2015] Bielet Inc (2015). Goji Smart Locks for the Home. Available from:http://www.gojiaccess.com/home.html [cited 2015-02-20].

[Bonar, 2012] Bonar, T. (2012). Ultrasonics and People Detection. Available from:http://www.maxbotix.com/articles/037.htm [cited 2015-03-21].

[Cambridge Dictionaries Online, 2015] Cambridge Dictionaries Online (2015). Au-tomation. Available from: http://dictionary.cambridge.org/dictionary/british/automate?q=automation [cited 2015-03-12].

[Elec Freaks, nd] Elec Freaks (n.d.). Ultrasonic Ranging Module HC - SR04. Avail-able from: http://users.ece.utexas.edu/~valvano/Datasheets/HCSR04b.pdf [cited 2015-03-21].

[Faltpihl, 2012] Faltpihl, P. (2012). Ultrasonic sensing design and implementationfor detecting and interacting with human beings in an ai system.

[Fitzpatrick et al., 2006] Fitzpatrick, K., Brewer, M. A., and Turner, S. (2006). An-other look at pedestrian walking speed. Transportation Research Record: Journalof the Transportation Research Board, 1982(1):21–29.

[Fryar et al., 2012] Fryar, C. D., Gu, Q., and Ogden, C. L. (2012). Anthropometricreference data for children and adults: United states, 2007-2010. Vital and healthstatistics. Series 11, Data from the national health survey, (252):1–48.

[Greenough, 2014] Greenough, J. (2014). The ’Internet of Things’ WillBe The World’s Most Massive Device Market And Save Companies Bil-lions Of Dollars. Available from: http://uk.businessinsider.com/

29

BIBLIOGRAPHY

how-the-internet-of-things-market-will-grow-2014-10?r=US [cited 2015-03-02].

[Haverbeke, 2011] Haverbeke, M. (2011). Eloquent Javascript. No Starch Press,San Franscisco, USA, 1st edition.

[Joyent, Inc, 2015] Joyent, Inc (2015). Node.js. Available from: http://nodejs.org/ [cited 2015-04-30].

[Kwikset, 2015] Kwikset (2015). Meet Kevo. Available from: http://www.kwikset.com/kevo/default.aspx#intro [cited 2015-02-20].

[Lockitron, 2015] Lockitron (2015). Bolt - Unlock your door without keys. Availablefrom: https://lockitron.com/ [cited 2015-02-20].

[McMaster, 1987] McMaster, R. L. (1987). Dual technology intruder detection sys-tem. US Patent 4,660,024.

[Monk, 2014] Monk, S. (2014). Raspberry Pi Cookbook, Software and HardwareProblems and Solutions. O’Reilly Media Inc, Sebastopol, California, 1st edition.

[Parallax Inc, 2012] Parallax Inc (2012). PIR Sensor (555-28027). Avail-able from: https://www.parallax.com/sites/default/files/downloads/910-28027-PIR-Sensor-REV-A-Documentation-v1.4.pdf [cited 2015-03-21].

[Pulford, 2007] Pulford, G. (2007). High-Security Mechanical Locks: An Encyclo-pedic Reference. Butterworth-Heinemann, Oxford, United Kingdom, 1st edition.

[Raj et al., 2012] Raj, B., Kalgaonkar, K., Harrison, C., and Dietz, P. (2012). Ul-trasonic doppler sensing in hci. IEEE Pervasive Computing, (2):24–29.

[Raspberry Pi Foundation, 2015] Raspberry Pi Foundation (2015). Camera mod-ule. Available from: https://www.raspberrypi.org/ [cited 2015-04-30].

[Richardson and Wallace, 2012] Richardson, M. and Wallace, S. (2012). GettingStarted with Raspberry Pi. O’Reilly Media Inc, Sebastopol, California, 1st edition.

[Statistiska Centralbyrån, 2013] Statistiska Centralbyrån (2013). Längd, vikt ochBMI 2010-11. Percentiler. Available from: http://www.scb.se/ulf/#c_li_354235 [cited 2015-05-07].

[Verisure, 2015] Verisure (2015). Smarta tjänster för hemmet och familjen. Avail-able from: http://www.verisure.se/smarta-hem.html [cited 2015-04-21].

[Werner et al., 2006] Werner, B., Bodin, L., and Bremberg, S. (2006). Data onheight and weight from school health records as a national public health surveil-lance tool: The case of sweden. Scandinavian journal of public health, 34(4):406–413.

30

[Woyke, 2011] Woyke, E. (2011). Smartphone ’Digital Keys’coming to Campuses, Offices, Homes. Available from:http://www.forbes.com/sites/elizabethwoyke/2011/09/14/smartphone-digital-keys-coming-to-campuses-offices-homes/ [cited2015-02-23].

[Young, 2013] Young, H. (2013). University Physics with Modern Physics Technol-ogy Update. Pearson, San Francisco, USA, 13th edition.

31

Appendix A

Code for the Human Motion DetectionAlgorithm

The coding for the proposed human motion detection algorithm is presented on thenext page. See Section 3.3.1 for further explanations.

33

//Importing modules

var usonic = require('r-pi-usonic');

var Gpio = require('onoff').Gpio,

pir = new Gpio(27, 'in', 'both');

//Deploying the PIR sensor and setting it to watch for heat changes. If it is detected, the

ultrasonic sensor will be called trough the init() function.

pir.watch(function(err, value)

if (err) exit();

switch (value)

case 1:

console.log('Motion detected by PIR');

init();

case 0:

break;

);

console.log('Pi deployed successfully!');

//If the PIR sensor was not deployed successfully, the process will be exited.

function exit()

pir.unexport();

process.exit();

//Measuring the distance every 60 milliseconds to the nearest object and storing them to an

array if is measured to be within 1.0 meters, or else it will exit and go back to the PIR

sensor.

var init = function()

var sensor = usonic.createSensor(18, 17, 1000);

var distance;

var distances = [];

(function measure()

distance = sensor();

if (distance <= 100)

distances.push(distance);

verifyMotion(distances);

else if (!distances || distance > 100)

console.log('Ultrasonic sensor did not agree');

console.log('Distance: ' + distance.toFixed(2) + ' cm');

return;

setTimeout(function()

measure();

, 60);

());

;

//If the detected object has stood in front of the sensors within 1.0 meter for 420 ms (7

pulses by the ultrasonic sensor), the detected object is interpreted as a visitor. A picture

is then taken.

var verifyMotion = function (distances)

var lastDistance = distances[distances.length -1];

if (distances.length === 7)

console.log('Motion verified by ultrasonic sensor');

console.log('Distance: ' + lastDistance.toFixed(2) + ' cm');

takePicture();

setTimeout(function() StillAPerson();, 1500);

;

//No more than two pictures will be taken of the same person.

function StillAPerson()

var sensor = usonic.createSensor(18, 17, 1000);

var distance2;

var distances2 = [];

distance2 = sensor();

while (distance2 <= 100)

distances2.push(distance2);

if (distances2.length === 1)

console.log('Still a person nearby');

distance2 = sensor();

distance2 = sensor();

Appendix B

Case Study 1: Results

The results of the study in which the ability to detect visitors depending on thetime variable x was evaluated and are shown on the next page.

35

Test Case - Bachelor ThesisTEST OBJECTSIdentity Type of object Height [m]

1 Human 12 Human 0,53 "Non-Human": Wheelchair 1,25

TaskTask 1 Standing still in front of the door

Task 2 Passing by in walking speed

TASK 1 Identity 1: 1 mDistance[m]\Variable x[ms] 120 180 240 300 360 420 480 540 1020 1980

0,5 73 139 205 299 343 423 467 532 1073 21811 78 152 223 272 348 418 484 574 1168 2301

1,5 - - - - - - - - - -

Identity 2: 0.5 mDistance[m]\Variable x[ms] 120 180 240 300 360 420 480 540 1020 1980

0,5 - - - - - - - - - -1 - - - - - - - - - -

1,5 - - - - - - - - - -

Identity 3: 1.25 mDistance[m]\Variable x[ms] 120 180 240 300 360 420 480 540 1020 1980

0,5 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes1 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

1,5 - - - - - - - - - -

TASK 2 Identity 1: 1 mDistance[m]\Variable x[ms] 120 180 240 300 360 420 480 540 1020 1980

0,5 Yes - - - - - - - - -1,5 - - - - - - - - - -

Identity 2: 0.5 mDistance[m]\Variable x[ms] 120 180 240 300 360 420 480 540 1020 1980

0,5 - - - - - - - - - -1,5 - - - - - - - - - -

Identity 3: 1.25 mDistance[m]\Variable x[ms] 120 180 240 300 360 420 480 540 1020 1980

0,5 Yes - - - - - - - - -1,5 - - - - - - - - - -

Statistics on 180 ms, task 2 Visitor detection rate within range 16 %100 test cases, walking speed Visitor detection rate outside range 0 %

Statistics on 240 ms, task 2 Visitor detection rate within range 6 %100 test cases, walking speed Visitor detection rate outside range 0 %

Statistics on 300 ms, task 2 Visitor detection rate within range 3 %100 test cases, walking speed Visitor detection rate outside range 0 %

Statistics on 360 ms, task 2 Visitor detection rate within range 2 %100 test cases, walking speed Visitor detection rate outside range 0 %

Statistics on 420 ms, task 2 Visitor detection rate within range 0 %100 test cases, walking speed Visitor detection rate outside range 0 %

Appendix C

Draft of Servo Case

The draft on the next page shows the servo case.

37

APPENDIX C. DRAFT OF SERVO CASE

38

TRITA MMK 2015-10 MDAB 06

www.kth.se