GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of...
Transcript of GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of...
![Page 1: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/1.jpg)
GPU Computing in Tomorrow’s Automobiles
GPU Computing in Tomorrow’s Automobiles
Pinar Muyan-Özçelik *Vladimir Glavtchev * †
*UC Davis, †BMW Group Technology Office in Palo Alto
San Jose, CA
![Page 2: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/2.jpg)
GPU Computing in Tomorrow’s Automobiles
automotive apps need compute
• Today’s cars provide many features with compute requirements.– e.g. BMW 7 Series have over a hundred
digital systems
• Future apps will require even more compute!
![Page 3: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/3.jpg)
GPU Computing in Tomorrow’s Automobiles
Some of the current advanced tech:
– active cruise control
today we have…
![Page 4: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/4.jpg)
GPU Computing in Tomorrow’s Automobiles
today we have…Some of the current advanced tech:
– active cruise control– adaptive headlights
![Page 5: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/5.jpg)
GPU Computing in Tomorrow’s Automobiles
today we have…Some of the current advanced tech:
– active cruise control – adaptive headlights– lane departure warning system
![Page 6: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/6.jpg)
GPU Computing in Tomorrow’s Automobiles
today we have…Some of the current advanced tech:
– active cruise control – adaptive headlights – lane departure warning system– blind spot information system
[courtesy of Volvo]
![Page 7: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/7.jpg)
GPU Computing in Tomorrow’s Automobiles
today we have…Some of the current advanced tech:
– active cruise control – adaptive headlights – lane departure warning system– blind spot information system – night vision
![Page 8: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/8.jpg)
GPU Computing in Tomorrow’s Automobiles
[courtesy of Volkswagen]
today we have…Some of the current advanced tech:
– active cruise control – adaptive headlights – lane departure warning system– blind spot information system – night vision
– parallel park assistance
![Page 9: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/9.jpg)
GPU Computing in Tomorrow’s Automobiles
today we have…Some of the current advanced tech:
– active cruise control – adaptive headlights – lane departure warning system– blind spot information system – night vision
– parallel park assistance– head-up display
![Page 10: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/10.jpg)
GPU Computing in Tomorrow’s Automobiles
[courtesy of Audi – from
Jean-François Mainguet’s site]
today we have…Some of the current advanced tech:
– active cruise control – adaptive headlights – lane departure warning system– blind spot information system – night vision
– parallel park assistance – head-up display– fingerprint
entry/ignition/personalization
![Page 11: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/11.jpg)
GPU Computing in Tomorrow’s Automobiles
today we have…Some of the current advanced tech:
– active cruise control – adaptive headlights – lane departure warning system– blind spot information system – night vision
– parallel park assistance – head-up display– fingerprint
entry/ignition/personalization– infotainment & telematic apps
![Page 12: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/12.jpg)
GPU Computing in Tomorrow’s Automobiles
tomorrow we will have…Some of the upcoming tech:– Internet connection
![Page 13: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/13.jpg)
GPU Computing in Tomorrow’s Automobiles
tomorrow we will have…Some of the upcoming tech:– Internet connection– e-mailing with speech
recognition/synthesis
![Page 14: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/14.jpg)
GPU Computing in Tomorrow’s Automobiles
tomorrow we will have…Some of the upcoming tech:– Internet connection– e-mailing with speech
recognition/synthesis– driver fatigue recognition
[courtesy of Mercedes-Benz]
![Page 15: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/15.jpg)
GPU Computing in Tomorrow’s Automobiles
tomorrow we will have…Some of the upcoming tech:– Internet connection– e-mailing with speech
recognition/synthesis– driver fatigue recognition– car-to-car communication
![Page 16: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/16.jpg)
GPU Computing in Tomorrow’s Automobiles
tomorrow we will have…Some of the upcoming tech:– Internet connection– e-mailing with speech
recognition/synthesis– driver fatigue recognition– car-to-car communication
In the long run:– active pedestrian avoidance
![Page 17: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/17.jpg)
GPU Computing in Tomorrow’s Automobiles
tomorrow we will have…Some of the upcoming tech:– Internet connection– e-mailing with speech
recognition/synthesis– driver fatigue recognition– car-to-car communication
In the long run:– active pedestrian avoidance– active crash prevention and stabilization– communication with infrastructure:
• Traffic lights• Cameras
• Speed signs
![Page 18: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/18.jpg)
GPU Computing in Tomorrow’s Automobiles
…and even maybe…
…get a ridewith Knight Rider’s KITT, which can:
- see, learn, think - communicate- have a personality
![Page 19: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/19.jpg)
GPU Computing in Tomorrow’s Automobiles
in “terms” of Computer Science:• computer vision/pattern recognition
speed limit recognition, lane departure warning system, blind spot information system, parallel park assistance, fingerprint apps, driver fatigue recognition, active cruise control, active crash avoidance/stabilization
• signal processingspeech recognition, night vision
• graphicsinfotainment apps, head-up display
• networkingcar-to-car communication, telematic apps, communication with infrastructure, Internet connection
![Page 20: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/20.jpg)
GPU Computing in Tomorrow’s Automobiles
GPUs are good fit for…• computer vision/pattern recognition
speed limit recognition, lane departure warning system, blind spot information system, parallel park assistance, fingerprint apps, driver fatigue recognition, active cruise control, active crash avoidance/stabilization
• signal processingspeech recognition, night vision
• graphicsinfotainment apps, head-up display
• networkingcar-to-car communication, telematic apps, communication with infrastructure, Internet connection
naturally good at
with GPU computing, also good attasks with data-level parallelism
not suited for
![Page 21: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/21.jpg)
GPU Computing in Tomorrow’s Automobiles
• Today's vehicles use dozens of:– Automobile Specific Integrated Circuits (ASICs)– Application Specific Standard Products (ASSPs),– Programmable Logic Devices (PLDs)– Field Programmable Gate Arrays (FPGAs)
GPU vs others
[courtesy of Actel]
![Page 22: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/22.jpg)
GPU Computing in Tomorrow’s Automobiles
• Today's vehicles use dozens of:– Automobile Specific Integrated Circuits (ASICs)– Application Specific Standard Products (ASSPs),– Programmable Logic Devices (PLDs)– Field Programmable Gate Arrays (FPGAs)
• GPUs allow consolidation:– simplify vehicle design
GPU vs others
![Page 23: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/23.jpg)
GPU Computing in Tomorrow’s Automobiles
• Today's vehicles use dozens of:– Automobile Specific Integrated Circuits (ASICs)– Application Specific Standard Products (ASSPs),– Programmable Logic Devices (PLDs)– Field Programmable Gate Arrays (FPGAs)
• GPUs allow consolidation:– simplify vehicle design
• Due to economies of scale:– adding a GPU to the production line is cheap
• With its programmability, GPUs offer ability to:– rapidly prototype– improve functionality with software updates
GPU vs others
![Page 24: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/24.jpg)
GPU Computing in Tomorrow’s Automobiles
our project
• We focus on three tasks:– computer vision techniques for recognizing
traffic signs– augmented display techniques for
enhancing the visual aids– speech recognition techniques for
controlling the body electronics
=> GPUs are good fit for above
• Will use CUDA, NVIDIA GPU and a host CPU.
![Page 25: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/25.jpg)
GPU Computing in Tomorrow’s Automobiles
speed sign recognition
• We are investigating the use of parallel algorithms to detect speed signs with GPUs.
• Here are details from Vladimir, who worked on this project over the summer at BMW Group Technology Office in Palo Alto…
![Page 26: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/26.jpg)
BMW GroupTechnology Office Palo Alto Innovation at the BMW Group.
BMW Technology Office - Palo Alto CA
EU Speed Limit DetectionVladimir Glavtchev
![Page 27: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/27.jpg)
BMW GroupTechnology Office Palo Alto
EU Speed Limit Sign Recognition. The Problem
Can a G86 be used as an embedded processor in a vehicle to perform speed limit sign recognition in real-time?
What algorithms are best suited for parallelization (e.g. can be done in CUDA)?
![Page 28: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/28.jpg)
BMW GroupTechnology Office Palo Alto
EU Speed Limit Sign Recognition. Challenges
Real-time constraints
Embedded hardware requirements
Robustness
![Page 29: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/29.jpg)
BMW GroupTechnology Office Palo Alto
EU Speed Limit Sign Recognition. Built-in GPU hardware advantages
Line interpolationImage resizing
Triangle rasterizationLarge available memory – for storing vertex arrays
Tons and tons of compute power!
![Page 30: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/30.jpg)
BMW GroupTechnology Office Palo Alto
Digit Recognition
Radial Symmetry
Thresholding
Sobel
Live Image from Camera
EU Speed Limit Sign Recognition. A GPU-oriented solution
Pre-Processing
Detection
Recognition
![Page 31: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/31.jpg)
BMW GroupTechnology Office Palo Alto
EU Speed Limit Sign Recognition. Project Structure
• Project interface:• Integrate into CUDA VisionWorkbench (CVWB)
from John Roberts / Joe Stam• Use CVWB graphic user interface• Use the existing Sobel and add simple
thresholding
Credit: nVidia (John Roberts, Joe Stam, James Fung)
![Page 32: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/32.jpg)
BMW GroupTechnology Office Palo Alto
EU Speed Limit Sign Recognition. Original Image: EU speed limit sign
![Page 33: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/33.jpg)
BMW GroupTechnology Office Palo Alto
EU Speed Limit Sign Recognition. Pre-Processing: Sobel with thresholding
![Page 34: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/34.jpg)
BMW GroupTechnology Office Palo Alto
EU Speed Limit Sign Recognition. Detection: Radial Symmetry
Algorithm developed by: Vladimir Glavtchev, Joe Stam, James Fung
![Page 35: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/35.jpg)
BMW GroupTechnology Office Palo Alto
EU Speed Limit Sign Recognition. Detection: Radial Symmetry voting
![Page 36: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/36.jpg)
BMW GroupTechnology Office Palo Alto
EU Speed Limit Sign Recognition. Detection: Region of interest isolation
![Page 37: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/37.jpg)
BMW GroupTechnology Office Palo Alto
EU Speed Limit Sign Recognition. Recognition: Digit recognition
![Page 38: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/38.jpg)
BMW GroupTechnology Office Palo Alto
EU Speed Limit Sign Recognition. Recognition: Digit isolation
![Page 39: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/39.jpg)
BMW GroupTechnology Office Palo Alto
EU Speed Limit Sign Recognition. Identification: Digit probability output
Image and Concept Credit: Cheng-Kai Chen and Shi Yan
![Page 40: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/40.jpg)
GPU Computing in Tomorrow’s Automobiles
standardization efforts
• No APIs exist for apps we focus on
• We would like to create a common platform:– have data-parallel embedded computing
interface for automotive apps– allow multiple vendors to compete and give
auto manufacturers more choice– allow reuse of the algorithms we developed
![Page 41: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/41.jpg)
GPU Computing in Tomorrow’s Automobiles
challenges
• Allowing the GPU to multitask between the three tasks we focus in our project
• Delivering real-time performance guarantees
• Dealing with embedded system constraints
• Integrating CPU-GPU
![Page 42: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/42.jpg)
GPU Computing in Tomorrow’s Automobiles
our group
• IDAV at UC Davis : – Pinar Muyan-Özçelik, Vladimir Glavtchev,
Dr. John D. Owens
• BMW Group Technology Office in Palo Alto: – Vladimir Glavtchev, Jeff Ota
• NVIDIA Corporation in Santa Clara : – Joe Stam
![Page 43: GPU Computing in Tomorrow ’s Automobiles Computing in Tomorrow ’s Automobiles in “terms” of Computer Science: • computer vision/pattern recognition speed limit recognition,](https://reader031.fdocuments.in/reader031/viewer/2022013015/5b3060937f8b9ad76e8e0b80/html5/thumbnails/43.jpg)
GPU Computing in Tomorrow’s Automobiles
Thank you!Questions?