October 5, 2006. 1Memorial University of Newfoundland Faculty of Engineering & Applied Science...

October 5, 2006. 1Memorial University of Newfoundland

Memorial University of Newfoundland

Faculty of Engineering & Applied Science

Engineering 6806

Electrical & Computer Engineering

Design Project

INTRODUCTION TO MACHINE VISION II

Prof. Nick Krouglicof


Presentation Outline

1. Machine vision systems for mechanical metrology

• A systematic approach to the calibration of machine vision systems for industrial metrology

• An automatic, machine vision based system for robot calibration

2. Industrial Applications of Machine Vision

• High speed, line scan camera-based inspection system for the food processing industry

• Vision based inspection of liquid crystal display (LCD) modules

3. CPLD based interfaces for sensors and actuators


A systematic approach to the calibration ofmachine vision systems for industrial metrology

• In the field of machine vision, camera calibration refers to the experimental determination of a set of parameters which describe the image formation process for a given analytical model of the machine vision system.

• Ideally, camera calibration is performed without specialized optical equipment, without modifications to the hardware, and without a priori knowledge of the vision system.

• Most calibration techniques are based on the observation of planar (2D) targets with a large number of control points.



The machine vision parameters which must be identified include :

a) The scale factor b) The frame buffer coordinates of the image center c) The effective focal length of the lens-camera assemblyd) The radial lens distortion coefficiente) The pose (position and orientation) of the camera

Parameters a) through d) are classified as intrinsic, e) as extrinsic.



In general, most camera calibration techniques fail because:

•They fail to account for the mathematical dependency between the image center and the X-Y position of the camera.

•The determination of the intrinsic parameters assumes a priori knowledge of the extrinsic parameters (i.e. position and orientation of the camera). Therefore, the extrinsic parameters are generally determined first.

•In order to accurately evaluate the intrinsic parameters, the extrinsic parameters must be know to a degree of accuracy which is practically unattainable (6 significant digits).



In order to compensate for the deficiencies of existing calibration techniques, a special fixture was designed to evaluate the critical parameters (i.e. image center and effective focal length) before the remaining parameters.


• To locate the edge with sub-pixel accuracy, the gradient in the vicinity of the edge is reconstructed using a continuous Gaussian function.

• The coordinates of the pixel nearest to the edge are denoted by i and j. The location of the edge with sub-pixel accuracy is determined by calculating the offsets x and y which maximize the function s(x,y) below over a 5 by 5 kernel :

en)+j m,+G(i = y)s(x, )n-(y+)m-(x2

1-

2

-2=n

2

-2=m

22



An automatic, machine vision basedsystem for robot calibration

Objective: To develop a low cost, non-contact robot calibration system that can be applied on a routine basis without significantly affecting production schedules.

Based on a CCD camera mounted on the robot’s wrist. The camera is used to determine the position and orientation of a fixed, passive target in six degrees-of-freedom.

The system employs the Hayati and Mirmirani modeling convention for closed kinematic chains and non-linear least squares analysis to determine the real kinematic parameters specific to a particular robot.


Industrial Machine Vision Applications


High Speed, Line Scan Based Inspection System


•Objective: To remove defects (I.e. visible, dark particle larger than 0.007”) from apple sauce

•System must be able to handle 12 metric tons per 8 hour shift

•System must remove 95% of visible defects

Line Scan Based Inspection System: Specifications


•2 distinct challenges:

•Detection

•Removal



•Detection system based on a high performance line scan camera; 4096 pixels per line at 4800 lines per second.

•Image acquisition and processing functions implemented on a Complex Programmable Logic Device (CPLD) as opposed to a microprocessor or Digital Signal Processor (DSP).

•The objective is to implement image processing functions with dedicated logic gates (i.e. hardware) for real-time performance.

Line Scan Based Detection System


What are CPLDs?

•Complex Programmable Logic Devices (CPLDs) are a class of programmable logic device that are commonly used to implement complex digital designs on a single integrated circuit.

•Applications of CPLDs in the field of computer engineering include the implementation of bus controllers, address decoders, priority encoder and state machines


Line Scan Camera-Based Detection System



Typical Particle

Typical section of apple sauce recorded with an area scan camera


LINESCAN DATA

20

40

60

80

100

120

1 101 201 301 401 501 601 701

Pixel Number

Lig

ht

Lev

el



Removal System


Removal System: Flow Characterization

•Rheological nomenclature and associated velocity profiles for steady flow through tubes with circular cross section.


Viscosity Measurement


Viscosity Measurement

Velocity profile can be characterized by power law!


Flow Profile of Apple Sauce


Particle Removal Window


System Timing Diagram


Aspiration Valve Characterization


Particle Detection Rate Versus Flowrate


High Speed, Line Scan Based Inspection System:Current Status

•Industrial partner is currently developing the production version of the system.

•Packaging of the principle components (i.e., lenses, cameras, electronics, light sources) remains a major challenge given the environment.

•One possible solution is to integrate all the electronics in the camera enclosure.

•Partner is anxious to explore applications in the pulp and paper industry.


Vision Based Inspection of LiquidCrystal Display (LCD) modules

•Objective: to automate the inspection of LCD modules in order to improve quality control

•One step in the implementation of a Six-Sigma Program (“3.4 defects per million opportunities”)

•The inspection must be completed within 30 seconds for 10 predetermined LCD patterns

•System can “learn” new LCD modules without modifying software


Vision Based Inspection of LCD modules: System Components

•Pulnix camera with macro lens

•High frequency fluorescent light sources

•Coreco Bandit integrated image acquisition and VGA accelerator

•Software developed using with WiT graphical programming environment in combination with Microsoft VB


Original Image Showing Error in Alignment



Thresholding Operation – Image Subtraction with respect to an image with no segments

illuminated



Blob Analysis – Reference Points are Identified



Image Rotation and Translation



Pixel by Pixel Image Subtraction from Reference Image – Thinning Operator



Blob Analysis



Blob Analysis

Vision Based Inspection of LiquidCrystal Display (LCD) modules: Conclusions

•Inspection system was installed at BAE Systems Canada Ltd. where it was used to test between 200 and 600 LCD displays per day.

•Number of defective modules that passed inspection was basically reduced to zero.

•Occasional “false positives” proved to be technical problems with the devices that previously went unreported.

•Applications for this technology are numerous given the number of LCD displays produced annually.


CPLD based interfaces for smart

sensors and actuators


Introduction

•Several novel “mechatronic” applications for CPLDs are presented.

•CPLD basically provides reconfigurable digital I/O that permits the implementation of interfaces for smart sensors and actuators.

•Applications that will be discussed include:

•Stepper motor controller

•Quadrature decoder/counter for optical encoder

•Pulse-Width Modulation (PWM) motor drives

•By incorporating a CPLD that supports In-System Programmability (ISP) the target device can be reprogrammed by the user for a variety of applications without removing it from the host system.


Mechatronic Applications of CPLDs – RSG System


Design Objectives of RSG System

To develop a low cost, microcontroller-based system that allows the user to interface a variety of sensors and actuators to a laptop computer. In addition to being “user- friendly”, the system must be capable of functioning as either:

1. A portable data acquisition system,2. An autonomous, stand-alone controller in support of

mechatronic design projects,3. A data logger that can acquire and store data at

regular intervals over extended periods of time.


Design Philosophy

Embedded Controller• System incorporates an 80C52-BASIC

Microcontroller• Simplifies the management of the various

onboard peripherals (Analog-to-Digital converter, LCD display, serial interface)

• Incorporates a powerful, imbedded BASIC interpreter

• Can also be programmed in assembler or C (Small Device C Compiler: SDCC)

• Assembled/compiled code can be “called” from Basic


Design Philosophy (continued)

Programmable Logic• System incorporates a user configurable

Complex Programmable Logic Device (CPLD)• CPLD supports in-system programmability by

the user• Mechatronic applications envisioned for the

CPLD on the UC2 system include quadrature decoder/counter interfaces for optical encoders, stepper motors controllers, PWM motor drives, sequential controllers, event counters/timers, Serial Peripheral Interface (SPI), etc.


80C52-BASICMICROCONTROLLER

32K X 8 EEPROM

32K X 8 SRAM

12 / 16 BIT ATOD CONVERTER

COMPLEX PROGRAMMABLE LOGIC

DEVICE

16 CHARACTERLCD DISPLAY

RS232PORT

JTAGPORT

DIGITAL I/O(18 LINES)

ANALOG IN(4 CHANNELS)

DATA BUS

ADDRESS BUS

CONTROL LINES

Block Diagram of RSG System


Stepper Motor Controller: Synchronous Counter Design


Stepper Motor - Full-Stepping


Stepper Motor – Half Stepping


Stepper Motor Half-Step Switching Sequences

STEPTRANSISTORS

1 2 3 4

1 ON OFF ON OFF

2 ON OFF OFF OFF

3 ON OFF OFF ON

4 OFF OFF OFF ON

5 OFF ON OFF ON

6 OFF ON OFF OFF

7 OFF ON ON OFF

8 OFF OFF ON OFF


Truth Table for Half-Stepping Switching Sequence


Stepper Motor: Synchronous Counter Design


Schematic of stepper motor controller

Stepper Motor: Synchronous Counter Design


Quadrature Decoder/Counter Interface for Optical Encoder: State Machine


Optical Encoder


Optical Encoder - Decoding


Optical Encoder – State Diagram


Optical Encoder – Edge Detection Circuitry


Summary of Quadrature Decoding Logic

State TransitionEncoder Signals

Flipflop outputs: enc_decX.q

PastState

PresentState

CountChan

AChan

B0 1 2 3

1 2 UP 1 1 1 1 0

2 3 UP 1 0 1 1 1

3 4 UP 0 0 0 0 1

4 1 UP 0 1 0 0 0

1 4 DOWN 0 0 1 0 0

4 3 DOWN 0 0 0 1 0

3 2 DOWN 1 1 0 1 1

2 1 DOWN 1 1 1 0 1

Note: and indicate a rising and falling edge respectively.


Truth Table Implementation of Quadrature Decoder

TABLEenc_dec[3..0].q => dwn_cnt, up_cnt;B"0000" => 0, 0;B"0001" => 0, 1;B"0010" => 1, 0;B"0011" => 0, 0;B"0100" => 1, 0;B"0101" => 0, 0;B"0110" => 0, 0;B"0111" => 0, 1;B"1000" => 0, 1;B"1001" => 0, 0;B"1010" => 0, 0;B"1011" => 1, 0;B"1100" => 0, 0;B"1101" => 1, 0;B"1110" => 0, 1;B"1111" => 0, 0;

END TABLE;


AHDL Implementation of Counter Module

IF up_cnt THENencoder[].d = encoder[].q + 1;

ELSEIF dwn_cnt THEN

encoder[].d = encoder[].q - 1;ELSE

encoder[].d = encoder[].q;END IF;

END IF;


Pulse-Width Modulation: VHDL/AHDL Programming


AHDL Implementation of PWM Controller

totaltime[].clk = clk;totaltime[].clrn = VCC;totaltime[].prn = VCC;totaltime[].ena = io_space_2 & (add_latch[3..0].q == B"0110") & !write/; totaltime[].d = add_data[];

lowtime[].clk = clk;lowtime[].clrn = VCC;lowtime[].prn = VCC;lowtime[].ena = io_space_2 & (add_latch[3..0].q == B"0111") & !write/;lowtime[].d = add_data[];


cntr[].clk = clk;cntr[].clrn = !(cntr[].q == totaltime[].q);cntr[].prn = VCC;cntr[].ena = VCC;cntr[].d = cntr[].q + 1;

switch.clk = clk;switch.clrn = VCC;switch.prn = VCC;switch.ena = VCC;pwm_out = switch.q;switch.d = (cntr[].q < lowtime[].q);

AHDL Implementation of PWM Controller


Pedagogical Applications


Conclusions

•CPLDs are well suited to the implementation of interfaces for smart sensors and actuators.

•Incorporating a CPLD that supports In-System Programmability (ISP) into a data acquisition system provides reconfigurable digital I/O that allows the target system to be reprogrammed by the user for a variety of applications without hardware modifications.

•This is particularly useful for those applications that require hardware interfaces for several different sensors simultaneously.


QUESTIONS

?

October 5, 2006. 1Memorial University of Newfoundland Faculty of Engineering & Applied Science...

Documents

Transcript of October 5, 2006. 1Memorial University of Newfoundland Faculty of Engineering & Applied Science...