MARKETING OF NON-PROFIT ORGANIZATIONS Ing. Mgr. Radim Bačuvčík, Ph.D. U44 - 207, [email protected] .
FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image...
Transcript of FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image...
![Page 1: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/1.jpg)
ni.com
FPGA for image processing
Radim ŠTEFAN www.ni.com/vision
![Page 2: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/2.jpg)
2 ni.com
National Instruments - What We Do
NI combines graphical programming software with modular hardware, leveraging the latest technologies.
Low-Cost Modular Measurement and Control Hardware
Productive Software Development Tools
Highly Integrated Systems Platforms
![Page 3: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/3.jpg)
3 ni.com
Graphical System Design
A Platform-Based Approach for Measurement and Control
![Page 4: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/4.jpg)
4 ni.com
NI Vision Platform
![Page 5: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/5.jpg)
5 ni.com
Field Programmable Gate Array (FPGA)
•A semi-conductor device containing many gates (logic devices)
•A wiring list downloaded to the FPGA determines the gate connections and the functionality
5
CONFIGURABLE LOGIC BLOCK (CLB)
PROGRAMMABLE INTERCONNECT
I/O BLOCK
Source: Xilinx
![Page 6: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/6.jpg)
6 ni.com
FPGAs
• Latency
• Jitter
• Compute power
• Pipelining
• Security
• Weight / Power / Heat
• Complexity
• Raw Clock Rates
• Limited Floating Point support
![Page 7: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/7.jpg)
7 ni.com
LabVIEW FPGA VHDL
LabVIEW FPGA - Abstraction to the Pin
![Page 8: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/8.jpg)
8 ni.com
FPGA Image Processing
• High-speed control
• Image pre-processing
Optical Coherence Tomography
Web & Surface Inspection
Laser Alignment & Tracking
High-Speed Sorting
![Page 9: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/9.jpg)
9 ni.com
Pixel Processing Example - Threshold
Threshold
![Page 10: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/10.jpg)
10 ni.com
From Algorithm Design to Deployment
![Page 11: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/11.jpg)
11 ni.com
NI Vision FPGA Development Platform
• LOW LATENCY
• FULLY PIPELINED
• SCTL
• MULTIPLE TARGETS
• IN-LINE AND CO-PROCESSING
• SEAMLESS ACUISITION
• PORTFOLIO OF ESSENTIAL IP
• MULTIPLE IMAGE TYPES
• CUSTOMIZABLE
• RAPID PROTOTYPING & SIMULATION
• RESOURCE & TIMING ESTIMATION
• CODE GENERATION
Algorithm Engineering
Vision IP
Performance Deployment
![Page 12: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/12.jpg)
12 ni.com
Algorithm Engineering using Vision Assistant
• Rapid prototyping and functional simulation
• Ability to combine IP to design and test a Vision algorithm before compiling the code
• Achieved by: • Using Vision Assistant for fast prototyping and code generation
o FPGA Resources Estimation
o Automatic parallelization of code
o Synchronization of parallel streams (latency balancing etc.)
• Enforcing that all FPGA IP produces the same result as the corresponding VDM function call on the host
NEW IN 2014
![Page 13: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/13.jpg)
13 ni.com
Vision Assistant FPGA Code Generation
![Page 14: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/14.jpg)
14 ni.com
Vision FPGA IP („Intellectual Property“ = functions)
• Efficient Host <-> Target image
transfer Vis
• Preprocessing functions
• Support for multiple image types
• Four-wire handshake protocol for streaming
• Synchronization Express VI to support branching and merging
• Partially customizable using IP Builder
![Page 15: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/15.jpg)
15 ni.com
Vision FPGA IP
• Image Management • Cast • Vision FPGA Sync
• Color Utilities • Extract Color Plane • Bayer to RGB • RGB to Color • Color to RGB • Integer to Color
• Processing • BCG Lookup • Inverse • Threshold
• Filters • Convolute • Low pass • Nth order • Edge Detection (Sobel, Prewitt,
Roberts, Differentiation, Sigma, Gradient)
• Morphology • Binary (Dilate, Erode, Open,
Close) • Gray (Dilate, Erode, Open,
Close)
• Color Processing • Color Histogram • Color Threshold
![Page 16: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/16.jpg)
16 ni.com
Vision FPGA IP
• Operators
• Add
• Subtract
• Multiply
• Divide
• Absolute Difference
• MulDiv
• Modulo
• And
• Or
• Xor
• LogDiff
• Mask
• Compare
• Analysis
• Histogram
• Quantify
• Centroid
• Linear Averages
• Image transfer
• FIFO to Pixel bus
• Pixel bus to FIFO
![Page 17: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/17.jpg)
17 ni.com
Other Sources for Processing Algorithms
• IPNet (ni.com/ipnet)
• Repository for image processing and many other functions
• Xilinx CORE Generator Libraries
• 15+ blocks for image scaling, color space conversion, noise removal, etc.
• Integrated within LabVIEW FPGA environment
• IP Integration Node
• Import custom VHDL to create IP blocks with standard LabVIEW IO interfaces
![Page 18: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/18.jpg)
18 ni.com
Image Processing Functions
FPGAs not suitable for certain high-level algorithms
• Object-level vision functions – Pattern matching
– OCR/OCV
– Some geometric measurements
– Classification
![Page 19: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/19.jpg)
19 ni.com
Vision Assistant FPGA Code Generation
![Page 20: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/20.jpg)
20 ni.com
FPGA VI
CPU to FPGA FPGA to CPU
Image Processing
(VI automatically generated by NI Vision Assistant)
![Page 21: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/21.jpg)
21 ni.com
Host (CPU) VI
Configuration VIs
Host to Target Transfer VI
Target to Host Transfer VI
(VI automatically generated by NI Vision Assistant)
![Page 22: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/22.jpg)
22 ni.com
• Processing loop rate limited by longest delay in loop
• Reduce delays by splitting operations into multiple cycles
• Increases speed, but also latency
Morphology
Threshold
Acquire
Single Cycle Timed Loop + Pipelining
![Page 23: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/23.jpg)
24 ni.com
• FPGA Families supported: Zynq™, Kintex, Virtex 5, Spartan 6
• Co-Processing Targets (DMA capabilities, USB/GigE support) • NI cRIO (9111, 9112, 9113, 9114, 9116, 9118, 9033, 9034, 9038, 9066,
9067, 9068, 9076, 9081, 9082)
• NI sbRIO 9606, 9626, 9636, 9651
• NI myRIO 1900, 1950
• Compact Vision Systems NI CVS 1457RT, 1458RT and 1459RT
• Inline Processing Targets (Framegrabbers) • NI PCIe-1473R-LX50
• NI PCIe-1473R-LX110
Supported Targets
![Page 24: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/24.jpg)
25 ni.com
You Might Want to Use an FPGA for Vision…
• If latency or jitter is critical
• If power consumption is critical
• If you have to speed up throughput
• If you can pipeline your algorithms
• If you have to reduce the amount of data or aggregate multiple high-speed channels
![Page 25: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/25.jpg)
26 ni.com
Performance Examples
Functions (Number of instances)
CPU (fps)
FPGA (fps)
Speedup Resources on cRIO – 9068
Convolution Filter (1), Threshold (1), Binary Morphology (1)
23.82 98 4x LUT – 25% BRAM -17%
Convolution (10), Operators (4), Morphology (1), Threshold (1)
1.56 98 60x LUT – 70% DSP – 80% BRAM – 75%
Image Resolution: 640 x 480
![Page 26: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/26.jpg)
27 ni.com
Demo – Rotational Speed control, up to 100fps
![Page 27: FPGA for image processing - Technical University of Liberec · · 2017-05-23FPGA for image processing Radim ŠTEFAN . ... (DMA capabilities, ... FPGA (fps) Speedup Resources on](https://reader031.fdocuments.in/reader031/viewer/2022022514/5af184ee7f8b9ac62b9019cd/html5/thumbnails/27.jpg)
ni.com
Více informací: czech.ni.com/nidays
Zveme Vás na NIDays 2014, největší konferenci National Instruments v České republice, Praha 22.10. 2014
Více informací a registrace: czech.ni.com/nidays