License Plate Recognition (LPR) Auto-gate System

April 14th, 2009

Danny Choi (President and CEO)Jaehoon Shin (CMO and CFO)

Wook Sun Shim (CTO and COO)

• Introduction• Motivation• System Overview• Hardware

– Metal Detector– Microcontroller

• Communication Module• LPR System

– Camera Module– License Plate Locator– Letter Recognition

• Finances• Timeline• Final Thoughts• Questions and Answers

License Plate Recognition System Convert the license plate number to ASCII

245 AFC

License Plate uniquely identifies vehicle. Various police forces

Electronic toll Collection Pay per use roads Monitoring traffic activity

Auto Theft◦ On an average day in B.C.

37 vehicles are stolen 62 vehicles are broken into

One third occurred in the private residences◦ Underground parking lots with Auto gate

“Stop until the gate close” No physical barrier !

Auto-gate Parking Lot◦ Must carry key entry devices

Could be lost, stolen, or no battery

◦ RF signals can be stolen RF (radio frequency) remote control analyzer

Following after vehicles Walk-ins

No Physical barrier !

LPR Auto-gate system: Virtual Barrier◦The system recognizes the vehicle and

checks for verification◦Following-after vehicles or walk-ins are

controlled ◦Alerting a trespassing and save the image

for further crime investigation and enforcement.

Conveniences◦Eliminating the use of key entry device.◦Simply “come and enter”

No need to wait for the gate to close!◦Registering for guest vehicles:

Type in the plate number

Block Diagram

Metal Detector

LPR System

MicrocontrollerBoard (LCD & LED)

Logic Diagram

Capture&

ImageProcessing

GreenLED

Start

Vehicle? Verified?

Green?RedLED

RedLED & Alarm

YY

Y N

N

N

The Concept of Mutual Inductance

Main Components◦ TL072 (Dual Operational Amplifier), NE555

(Timer), a couple of Inductors

Microcontroller (Atmel ATmega168)◦ High density of various features (ADC, PWM,

LCD driver, and etc)◦ GNU Complier Collection (GCC) supported

Voltage Regulator (L7805)◦ 9V battery to 5V

LCD◦ 2x24 characters display◦ HD44780 protocol

Outputs (piezoelectric buzzer and LEDs)

Vehicle detection◦ Road tube, Magnetometer sensor, Weigh-In-

Motion (WIM) , Piezoelectric sensor, and etc Microcontroller

AtmelATmega168

ZilogZ86E3416VSC

Speed 20MHz 16MHz

Memory 16K 16K

Ram 1Kb 512b

I/O pin 23 24

ADC Yes No

Package DIP-28 DIP-28

Price $4-5 $11

RS232 serial I/O port◦ Cheaper and Simpler solution than USB!

Use CSerial source code

Baud Rate 115200

Data Bits 8

Parity None

Stop Bits 1

Flow Control None

Brain of the overall system. OpenCV – Computer Vision library by Intel. Module:

◦ Camera Module◦ License Plate Locator◦ (Skew Correction Module)◦ Letter Recognition Module◦ Verification Module

Grab a frame from a webcam Refresh the frame every 0.5 seconds If signal comes in from microcontroller, save

current frame.

Erosion:

Dilation:

Original:

Erosion after Dilating the image:

Dilation:

Original:

Dilation:

Grey Scaled:

Erosion after Dilation:

Dilation:

Subtract:

Threshold and Noise Filter

Canny Edge Detection

Not integrated with the overall system.◦ Not every input image had detectable edges.◦ Assumption: the camera is in its fixed position,

almost perfectly parallel to the ground.◦ System can also recognize the letters from license

plate with tilt angle of ~10°

Other Possible Solutions:◦ Hough Transform◦ Radon Transform

Localized Plate Invert Image Paint out margins Binary Image

Cropped plate

Character Segmentation

Character Segmentation

Template Matching !

Not a character !

Ratio is not valid

~

~

Template Matching

I, O, Q, U, Y, Z

Identification

…SIX

Identification

…B!

616BBL_

EquipmentEquipment Estimated CostEstimated Cost Actual CostActual Cost

CameraCamera $100 $0

Microcontroller & development kitMicrocontroller & development kit $300 $20

Metal detectorMetal detector $50 $0

Miscellaneous (LCD, cable, case, alarm, Miscellaneous (LCD, cable, case, alarm, etc)etc)

$50 $80

TotalTotal $500 $100

EquipmentEquipment Market priceMarket priceProject Project

CostCost

CameraCamera $150 $0

Microcontroller & development kitMicrocontroller & development kit $20 $20

Metal detectorMetal detector $800 $0

Miscellaneous (LCD, cable, case,Miscellaneous (LCD, cable, case,alarm, etc)alarm, etc)

$400 $80

TotalTotal $1370 $100

We learned:◦ Team dynamics◦ Digital Image Processing techniques◦ Programming in C/C++◦ Circuitry◦ Microcontroller

Challenges◦ A group of three members ◦ Localization

Possible Future Works◦ Improve skew correction◦ Implement with actual gate (or motor)◦ Finer resolution camera◦ User Interface◦ More accurate LPR algorithm

“The complexity of each of these subsections of the program determines the accuracy of the system” (en.wikipedia.org/wiki/Automatic_number_plate_recognition)

Dr. Parvaneh Saeedi Jun Kim Steve Whitmore Patrick Leung, P. Eng Jamie Westell Jason Lee

Insurance Corporation of British Columbia (ICBC). 2008. Stop Auto Crime in Multi-Residential and Mixed-Use Buildings.

Insurance Corporation of British Columbia (ICBC). 2007. Auto Crime and Fraud.

Mia Dauvergne. Statistics Canada. 2008. Motor Vehicles Theft in Canada, 2007.

Statistics Canada. 2008. Motor Vehicle Theft. G. Baridski and A. Kaehler. 2008. Learning OpenCV:

Computer Vision with the OpenCV Library. C. WU, L. C. ON, C. H. WENG, T. S. KUAN, K. NG.

August 2005. A Macao License Plate Recognition System.