Smarten up a Pi Zero Security Camera with Amazon Web Services and

Node.jsMark West, Bouvet Oslo

Who Am I?•Manager at Bouvet Oslo.

• Focus on Java, JavaScript, AI, Cloud & the Internet of Things.

• JavaZone organiser.

@markawest

Talk Outline

@markawest

Motivation and

Requirements

Pi Zero Camera

Version 1

Pi Zero Camera

Version 2Conclusion

Motivation and RequirementsInput from Stakeholder

@markawest

Motivation and Requirements

Pi Zero Camera Version 1

Pi Zero Camera Version 2 Conclusion

Motivation

@markawest

Project RequirementsFunctional

•Monitor activity in the garden.

• Send warning when activity detected.

• Live video stream.

Non-functional

• In place as soon as possible.

• Low cost.

• Portable.

@markawest

1. Activity in garden

2. Camera detects

movement

3. Camera sends alert email with

snapshot

Event Flow to Implement

@markawest

Pi Zero Camera Version 1

Hardware and Activity Detection

@markawest

Motivation and Requirements

Pi Zero Camera Version 1

Pi Zero Camera Version 2 Conclusion

PiZero Starter Kit: 300kr

Camera Module (NoIR): 250kr

Camera Adapter: 50kr

ZeroView: 80kr

Total Cost: 680kr

Hardware Shopping List

@markawest

Assembled CameraRear View Front View

@markawest

1. Activity in garden

2. Camera detects

movement

3. Camera sends alert email with

snapshot

Event Flow

@markawest

Motion (https://motion-project.github.io)

• Open Source motion detection software.

• Excellent performance on the Raspberry Pi Zero.

• Built-in Web Server for streaming video.

• Detected activity or ‘motion’ triggers events.

• Works “out of the box”. No need for additional programming.

@markawest

How Motion Works

@markawest

1 2 3 4

Demo

Pi Zero Camera

Version 1

@markawest

Project Requirements : Evaluation

Functional

•Monitor activity in the garden.

• Send warning when activity detected.

• Live video stream.

Non-functional

• In place as soon as possible.

• Low cost.

• Portable.

@markawest

Example Pictures from Alert Emails

@markawest

Examples of False Positives

@markawest

The Motion Software focuses on the

amount of changed pixels, and not the

cause of the changed pixels!

@markawest

Pi Zero Camera Version 2

Addressing false positives

@markawest

Motivation and Requirements

Pi Zero Camera Version 1

Pi Zero Camera Version 2 Conclusion

Project Requirements ReloadedFunctional

•Monitor activity in our garden.

• Send warning when activity detected.

• Live video stream.

Non-functional

• In place as soon as possible.

• Low cost.

• Portable.

@markawest

Project Requirements ReloadedFunctional

•Monitor activity in our garden.

• Send warning when human activity detected.

• Live video stream.

Non-functional

• In place as soon as possible.

• Low cost.

• Portable.

@markawest

Finding an Image Analysis Solution

OpenCV

• Use Face Detection to find out if a human was in the snapshot.

• Problem: What if the subject was facing away from the camera, or wearing a mask?

TensorFlow

• Train and use a Neural Network to find humans in the snapshot.

• Rejected by stakeholder as this solution “would take too long to implement”.

@markawest

@markawest

AWS Rekognition

@markawest

• Officially launched in November 2016.

• Part of Amazon Web Services (AWS) suite of products.

• Image Analysis as a Service, offering a range of API’s.

• Built on Deep Neural Network models.

• Competitors: Google Vision, MicroSoft Computer Vision and Clarafai.

AWS Rekognition Demo

@markawest

Limitations of AWS Rekognition

• Black box:• Not always intuitive.• Which labels are we looking for?

• 85% hit rate:• Noisy or blurred images give

worse results.• Possible solution - send multiple

snapshots for each single motion detected event.

@markawest

Updated Event Flow

@markawest

Amazon Web Services (AWS)Trigger Email if

snapshot contains a

person

Analyse snapshot from

Motion software

Tweaked Motion configuration file to provide multiple snapshots for each

detected activity

Demo

Pi Zero Camera

Version 2

@markawest

AWS IAM

AWS Rekognition

AWS Simple Email Service

AWS s3(storage)

Amazon Web Service FlowOverview

AWS Step Function(workflow)

s3 Upload Trigger

@markawest

1

2

4

5

6

calls

3

uses

uses

AWS Software Developer Kit (SDK)

• SDK’s exist for a range of platforms – including Python, Ruby, .NET and Java.

• Provide an API for consuming AWS Services.

• For this project I used the Node.js AWS SDK.

• Upload from PiZero Camera to AWS s3 handled by the AWS Node.js SDK.

@markawest

AWS IAM

AWS s3(storage)

AWS Lambda FunctionsCode Building Blocks or Microservices

AWS Step Function(workflow)

s3 Upload Trigger

@markawest

calls

AWS Rekognition

AWS Simple Email Service

uses

uses

AWS Lambda Functions• Units of code, based on Java, C#, Python or Node.js

• Serverless.

• Stateless, allowing for rapid scaling as needed.

• High availability.

• “Pay as you go” model, with a generous free tier.

• AWS SDK natively available.

@markawest

AWS Lambda Demo

@markawest

AWS IAM

AWS s3(storage)

AWS Step FunctionsOrchestation of Lambda Functions (Microservices)

AWS Step Function(workflow)

s3 Upload Trigger

@markawest

calls

AWS Rekognition

AWS Simple Email Service

uses

uses

AWS Step Functions• Launched in December 2016.

• Coordinate and orchestrate Lambda Functions into a Workflow or State Machine.

• Defined via JSON files, displayed as visual workflows.

• Provide the same benefits as AWS Lambda (High Availability, Scalable, “Pay as you go” pricing model).

@markawest

Pi Zero Camera v2 : Step Function

@markawest

Error Handler (sends Error Email

via AWS SES)

Sends snapshot to AWS Rekognition

Evaluate AWS Rekognition response

Send Alert Email?

Send Alert Email via AWS SES

Archive image in AWS s3

1

2

3

4

5

61

2

3

4

5

6

AWS Step Function Demo

@markawest

Estimated AWS CostFirst 12 months

0 kr *

After first 12 months

Less than 50 kr a month *

@markawest

* Estimated cost based on access to AWS Free Tier for first 12 months, with 50 processed images per day

Project Requirements : Evaluation

Functional

•Monitor activity in the garden.

• Send warning when human activity detected.

• Live video stream.

Non-functional

• In place as soon as possible.

• Low cost.

• Portable.

@markawest

Lessons Learned • Hobby projects are still projects.

• Iterative development allows for changing requirements.

• Find creative solutions to cost and time constraints.

• Use The Cloud to push past hardware constraints.

• AWS provides a cheap and easy way to experiment with Cloud Computing, but beware vendor lock-in!

@markawest

Next StepsNight vision Automated deployment

• Round trip for development and test is clunky, especially for Zip Files.

• Look into easing this process.

@markawest

Bouvet Labs

@markawest

Thanks for listening!

markwest1972/smart-security-camera

@markawest

utbrudd.bouvet.no