Creating a Face-recognising Security Cam With a Raspberry Pi Part 1

7/28/2019 Creating a Face-recognising Security Cam With a Raspberry Pi Part 1

1/5

Through the Interface

September 12, 201 2

Creating a face-recognising security cam with a Raspberry Pi Part 1

This series of posts builds upon the mini-series on building a motion-detecting sec urity cam based around the

Raspberry Pi. Once y ou have y our motion detecting security c am up and running, y ou should be able to mov e

on to the next stage: enabling that system to reco gnize faces that it has been trained against.My specific project (which Im c alling the Facecam, although I havent applied for a trademark;-) pulls data

do wn from Faceboo k and uses that to train the face recognition system, but thats far from being a requirement

its also very possible to train the database in other ways.

Well go into detail on the various parts ov er the coming weeks (probably once a week, as I want to continue

blogging about other to pics, too), but lets start by introducing the projec t and the system architecture.

Heres the elevator p itch: Facec am is a security camera that recognises a residents Faceboo k friends when they

come to their front door and allows for tailored co mmunication both to the resident and the visitor.

Thiswould basically mean that instead of the resident rec eiv ing a not ificatio n email say ing Y ou have a v isitor

(as in the case of the motion-detection sy stem we saw previously), the email would read John Smith visitedyou. A nd Jo hn c ould potentially receiv e a c ustomized wer e out message while standing at the doo r and eve

a post o n his Faceboo k wall thanking him for the v isit.

A lot of this is clearly contingent o n the accuracy of the facial rec ognition sy ste m something well lo ok at muc

more c losely later on. In case the accurac y isnt adequate, it would be nice at least to prov ide some feedback to

the v isitor, say ing Welco me, John! (even if its actually Megan at the do or ;-).

Right, now lets dive into the sy stem architecture . Heres a quick diagram of the Facecam sy stem:

Heres a look at the new components (in addition to the Motion detection box we saw previously):

1. A componen t to download information on the residents Facebook friends (.NET desktop

application)

This component will build a friend database (stored in a file named facedata.xml) to be used at runtime by the

security cam. The database will be trained with the friends photos as downloaded from Facebook. So the

co mponent will need to access for each friend both their names and the photos in which they have been
http://through-the-interface.typepad.com/.a/6a00d83452464869e2017d3bfcb797970c-pihttp://through-the-interface.typepad.com/.a/6a00d83452464869e2017d3bfcb797970c-pihttp://through-the-interface.typepad.com/.a/6a00d83452464869e2017d3bfcb797970c-pihttp://through-the-interface.typepad.com/.a/6a00d83452464869e2017d3bfcb797970c-pihttp://through-the-interface.typepad.com/.a/6a00d83452464869e2017d3bfcb797970c-pihttp://through-the-interface.typepad.com/.a/6a00d83452464869e2017d3bfcb797970c-pihttp://through-the-interface.typepad.com/.a/6a00d83452464869e2017d3bfcb797970c-pihttp://through-the-interface.typepad.com/.a/6a00d83452464869e2017d3bfcb797970c-pihttp://through-the-interface.typepad.com/.a/6a00d83452464869e2017d3bfcb797970c-pihttp://through-the-interface.typepad.com/.a/6a00d83452464869e2017d3bfcb797970c-pihttp://through-the-interface.typepad.com/.a/6a00d83452464869e2017d3bfcb797970c-pihttp://www.facebook.com/http://www.raspberrypi.org/http://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-motion-detecting-security-cam-with-a-raspberry-pi-part-2.htmlhttp://through-the-interface.typepad.com/through_the_interface/2012/08/creating-a-motion-detecting-security-cam-with-a-raspberry-pi-part-1.htmlhttp://through-the-interface.typepad.com/through_the_interface/


2/5

Creating a face-

recognising

security cam with a

Raspberry ...

Creating a motion-

detecting security

cam with a

Raspberry ...

Creating a motion-

detecting security

cam with a

Raspberry ...

Creating a face-

recognising

security cam with a

Raspberry ...

tagged (as well as the tag information, of course).

Using the tag information, the co mponent will use OpenCV to attempt to ex tract the friends face (by chec king

the list of detec ted faces against the tag lo cation, to get the closest match), and if it finds it it will resize, c rop

and equalize the grey scale intensity o f the image before saving it to disk for later use in the database training

process.

This is ultimately an offline, CPU-intensive o peration we will simply need the database transferred acro ss to

the Raspberry Pi from time to time so the approach well take is to c reate a .NET desktop application that

performs this task on a higher-powered dev ice.

2. A com ponent to analyse frames saved by th e motion-detection sy stem, checkin g for friends

(face recognition)

This is the guts of the sy stem: the runtime compo nent that will again use OpenCV to process images captured

from the webcam, detect faces and then check the results against the facial reco gnition database on the

Raspberry Pi. This needs to be pretty efficient as the feedback should be in clo se to real-time which may be

challenge: given the relatively low power o f the Raspberry Pi CPU, well have to jump through some interesting

hoops to get this working well enough.

3. A com ponent to provide feedback to the v isitor (LED messaging)

If were going to tell the v isitor that they have b een recognised (or in some way warn unwanted visitors that

their photo has been sent to the pro perty s residents), then were going to need so me kind of screen. The

Raspberry Pi has both HDMI and component v ideo outputs, but Iv e dec ided to go the way o f a USB-powered

LED message-board. More o n this in due co urse.

Right, thats it for this introductory post. Well look in more detail at each of these co mponents ov er the c oming

weeks, starting with the .NET desktop applic atio n to download friend information.

You m igh t also like:

LinkWithin

Posted at 07 :45 AM in Raspberry Pi | Permalink

Comments

Jeremy Tammik dijo...

Hi Kean,

Sounds neat and clear and interesting.
http://thebuildingcoder.typepad.com/http://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.htmlhttp://through-the-interface.typepad.com/through_the_interface/raspberry-pi/http://www.linkwithin.com/learn?ref=widgethttp://www.dreamcheeky.com/led-message-boardhttp://opencv.org/http://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-2.htmlhttp://through-the-interface.typepad.com/through_the_interface/2012/08/creating-a-motion-detecting-security-cam-with-a-raspberry-pi-part-1.htmlhttp://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-motion-detecting-security-cam-with-a-raspberry-pi-part-2.htmlhttp://through-the-interface.typepad.com/through_the_interface/2012/10/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-4.html


3/5

I really wo nder what kind of data is going to go into the face data XML file. I would not put pixel data there , but

store that in external indiv idual image files, and reference those .

Some of my friends boy co tt internet-based social networking and thus hopefully cannot be identified by

facebook, so I would enable private non-facebo ok photos from the local system to be used as well.

Cheers, Jeremy.

Responder September 12, 201 2 at 09:11 A M

Kean Walmsley dijo en respuesta a Jere my Tammik...

Hi Jeremy ,

We'll of course talk about the XML format (although it's something this projec t has inherited), which uses an

algorithm called Eigenfaces to greatly r educe the amount of data stored.

And y es - I 'm pr obably not ev en going to share the Faceboo k-spec ific c ode (for once), but will certainly ex plain

more abo ut the file/folder structure y ou'd need to build to train the database in some other way .

Cheers,

Kean

Responder September 12, 201 2 at 09:15 A M

AlexFielder dijo...

Hi Kean,

This loo ks really interesting!

An API I 'm not familiar with (but which I b eliev e might prove useful here) is the Goo gle Picasa Web API; As I

say, I o nly know there ex ists an API from Google which (in some way ) ties in with the Picasa Web o ffering fromthe big G.

In case y ou aren't aware, Picasa (the desktop application) offers y ou the ability to tag faces o f the people it finds

within y our picture librar y , and will (with y our permission) sy nc them to your Picasa Web Pho tos albums. I 've

tagged all my family with it, so if it's possible to use this API instead o f Facebook, that might be another avenue

that would likely get aro und the friends-not-tagged-on-Facebo ok issue that Jeremy mentions.

Here is the best link I co uld find that will be of help:

Retrieving face tag data from the Picasa Web API (which in turn links to this page: Google Groups Picasa issues

page

Cheers,

Alex.

Responder September 12, 201 2 at 04:52 PM

Kean Walmsley dijo en respuesta a Alex Fielder...

Hi Alex ,

That's interesting... it should certainly be possible to swap o ut the Facebo ok-oriented piece to pull down data
http://twitter.com/AlexFielderhttp://blogs.autodesk.com/through-the-interfacehttp://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html#comment-6a00d83452464869e2017c31d0245a970bhttp://void%200/https://code.google.com/p/gdata-issues/issues/detail?id=751&sort=-stars%20id&colspec=API%20ID%20Type%20Priority%20Stars%20Summaryhttp://www.klick.com/health/news/blog/retrieving-face-tag-data-from-the-picasa-web-api/http://twitter.com/AlexFielderhttp://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html#comment-6a00d83452464869e2017c31ce9048970bhttp://void%200/http://thebuildingcoder.typepad.com/http://blogs.autodesk.com/through-the-interfacehttp://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html#comment-6a00d83452464869e2017744ac4fb8970dhttp://void%200/


4/5

from Picasa.

I'll be describing the process I've used to isolate and extract faces to train the database next week.

Cheers,

Kean

Responder September 12, 201 2 at 04:56 PM

Philipp Wagner dijo...

This will probably help you: http://docs.opencv.org/trunk/modules/contrib/doc/facerec/index.html.

Responder September 12, 201 2 at 1 0:39 PM

Philipp Wagner dijo en respuesta a Philipp Wagner. ..

Especially the part o n Face Recognition in Videos might be interesting to yo u:

* http://doc s.opencv.org/trunk/modules/contrib/doc/ facerec/ tutorial/facerec_video_recognition.htmlResponder September 12, 20 12 at 10:45 PM

Kean Walmsley dijo en respuesta a Philipp Wagner...

Thanks, Philipp - that's very helpful.

I'm pretty much do ne with that side of things, but I'll certainly rev isit the co de onc e I've loo ked at the tutorial

more closely.

Kean

Responder September 12, 201 2 at 1 0:58 PM

Hlytin dijo...

That was an interesting thing,security camera purpo se is really a lot specially o n giving a security a shop and

home.In Finland country many of people are still having doubt on putting security cameras howev er most of th

already did.Any way this is really ex cellent one.

Responder November 07 , 2012 at 10:27 AM

Pyry l dijo...

Hi Kean, hope y ou are still following this thread. I'm wondering whether to start a face reco gnition pro ject. My

co ncept would be based on linkedin profile pics, thus only 1 re ference pic per face.

Based on your ex perience is reco gnizing a face with only one reference pic available from a photo by any chanc

possible? Did you encounter in your ex periment cases where the user had only one pic in fb?

Responder March 25 , 201 3 at 03 :22 PM
http://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html#comment-6a00d83452464869e2017d424741a3970chttp://void%200/http://twitter.com/pyrylhttp://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html#comment-6a00d83452464869e2017d3d5f6744970chttp://void%200/http://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html#comment-6a00d83452464869e2017c31d19b3f970bhttp://void%200/http://www.bytefish.de/http://blogs.autodesk.com/through-the-interfacehttp://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html#comment-6a00d83452464869e2017c31d18f37970bhttp://void%200/http://docs.opencv.org/trunk/modules/contrib/doc/facerec/tutorial/facerec_video_recognition.htmlhttp://www.bytefish.de/http://www.bytefish.de/http://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html#comment-6a00d83452464869e2017744af50af970dhttp://void%200/http://docs.opencv.org/trunk/modules/contrib/doc/facerec/index.html.http://www.bytefish.de/http://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html#comment-6a00d83452464869e2017744adea69970dhttp://void%200/


5/5

Kean Walmsley dijo en respuesta a Pyry l...

Hi Pyry l,

Based on my experience the problems would be:

* 1 photo is way to o few, at least for the system I tried to use .

* Profile photos aren't representative enough of how people loo k when they co me to y our doo r (or sit at theirPC, if somehow recognising people's faces v ia their webcam).

Cheers,

Kean

Responder March 25 , 201 3 at 04:00 PM

Pyry l dijo...

Thanks Kean! I think yo u just saved me a week of hacking towards a dead end.

Responder April 01, 20 13 at 12:44 PM

Comment below o r sign in with TypePad Faceboo k Twitter and more...
http://www.typepad.com/sitelogin?uri=http%3A%2F%2Fthrough-the-interface.typepad.com%2Fthrough_the_interface%2F2012%2F09%2Fcreating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html&fp=9658899d2e71abe8d5c8188ddc835a8c&view_uri=http%3A%2F%2Fprofile.typepad.com%2F&via=blogside&service=openid&post_uri=http://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.htmlhttp://www.typepad.com/sitelogin?uri=http%3A%2F%2Fthrough-the-interface.typepad.com%2Fthrough_the_interface%2F2012%2F09%2Fcreating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html&fp=9658899d2e71abe8d5c8188ddc835a8c&view_uri=http%3A%2F%2Fprofile.typepad.com%2F&via=blogside&service=twitter&post_uri=http://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.htmlhttp://www.typepad.com/sitelogin?uri=http%3A%2F%2Fthrough-the-interface.typepad.com%2Fthrough_the_interface%2F2012%2F09%2Fcreating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html&fp=9658899d2e71abe8d5c8188ddc835a8c&view_uri=http%3A%2F%2Fprofile.typepad.com%2F&via=blogside&service=facebook&post_uri=http://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.htmlhttp://www.typepad.com/sitelogin?uri=http%3A%2F%2Fthrough-the-interface.typepad.com%2Fthrough_the_interface%2F2012%2F09%2Fcreating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html&fp=9658899d2e71abe8d5c8188ddc835a8c&view_uri=http%3A%2F%2Fprofile.typepad.com%2F&via=blogside&post_uri=http://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.htmlhttp://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html#comment-6a00d83452464869e2017d4270a3bd970chttp://void%200/http://twitter.com/pyrylhttp://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html#comment-6a00d83452464869e2017c3818455f970bhttp://void%200/http://twitter.com/pyrylhttp://blogs.autodesk.com/through-the-interface

Creating a Face-recognising Security Cam With a Raspberry Pi Part 1

Documents

Transcript of Creating a Face-recognising Security Cam With a Raspberry Pi Part 1