tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC...
Transcript of tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC...
![Page 1: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:](https://reader034.fdocuments.in/reader034/viewer/2022042405/5f1c7066ae81ad2d9c634e55/html5/thumbnails/1.jpg)
tinyML meetup kick-offEnabling ultra-low power Machine Learning at the Edge
Evgeni Gousev, Qualcomm AI ResearchSanta Clara
June 27, 2020
![Page 2: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:](https://reader034.fdocuments.in/reader034/viewer/2022042405/5f1c7066ae81ad2d9c634e55/html5/thumbnails/2.jpg)
Motivation: Problem Statement and Opportunity
1. Growing (urgent) need to drive tinyML acceleration and adoption throughout the whole ecosystem
• Use-cases – Apps – SW – Tools – Algos – HW – ASIC – Device - Fabs
2. TinyML is real and will be huge• Many pieces of the big puzzle are “popping up”. Someone (who if not us !)
needs to put them together
• More innovations and breakthroughs ahead
![Page 3: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:](https://reader034.fdocuments.in/reader034/viewer/2022042405/5f1c7066ae81ad2d9c634e55/html5/thumbnails/3.jpg)
What is tinyML ?
• (for now) tinyML is broadly defined as machine learning architectures, techniques, tools and approaches capable of performing on-device analytics for a variety of sensing modalities (vision, audio, motion, environmental, human health monitoring etc.) at “mW” (or below) power range targeting predominately battery operated devices (IoT, bioelectronics, …)
![Page 4: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:](https://reader034.fdocuments.in/reader034/viewer/2022042405/5f1c7066ae81ad2d9c634e55/html5/thumbnails/4.jpg)
What is tinyML ?
• Capable of performing on-device analytics for a variety of sensing modalities (vision, audio, environmental, human health monitoring etc.) at “mW” power range
• battery operated devices (IoT, bioelectronics, …)
• HW – algorithms – SW – Use cases/applications
![Page 5: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:](https://reader034.fdocuments.in/reader034/viewer/2022042405/5f1c7066ae81ad2d9c634e55/html5/thumbnails/5.jpg)
Why tinyML ?Data is a new oil(electricity) and ML is a way to produce it
Cloud ML
•DNN on the cloud
•HW: TPU, FPGA, GPU, CPU
Edge ML
•Optimized algos and CNN-light
•SoC (with NPUs/NSP accelerators)
Tiny ML
•CNN-micro
•MCU w/ HW accelerators
Data Sources:
Storage and sharing
User provided:1. Pics2. Audio3. Clicks/likes4. GPS, Location based
Real-time in the physical world
CMOS cameras
IRcameras
IMUs Audiomicsb
Environ/chemical
Temperature Optricalsensors
1%
4%
95%
![Page 6: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:](https://reader034.fdocuments.in/reader034/viewer/2022042405/5f1c7066ae81ad2d9c634e55/html5/thumbnails/6.jpg)
tinyML is “good enough” NOW… and more enhancements coming in the near future
SW
AlgosHWHW accelerators (digital) Quantization, compressionSmaller models (100s kB)
$ initial tinyML applications
- Compute in memory- Analog compute- Neuromorphic
- Novel algos/networks - 10s kB models
$$$ More tinyML apps and value creation
Enabling technologies: ULP sensors, novel memories, 3D, energy scavenging, ULP radio
![Page 7: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:](https://reader034.fdocuments.in/reader034/viewer/2022042405/5f1c7066ae81ad2d9c634e55/html5/thumbnails/7.jpg)
![Page 8: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:](https://reader034.fdocuments.in/reader034/viewer/2022042405/5f1c7066ae81ad2d9c634e55/html5/thumbnails/8.jpg)
tinyML Summit-2019
![Page 9: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:](https://reader034.fdocuments.in/reader034/viewer/2022042405/5f1c7066ae81ad2d9c634e55/html5/thumbnails/9.jpg)
tinyML Summit 2019
• March 21-22 – hosted by Google
• 160 attendees
– Over 100 additional people on the interest list• unable to attend due to space constraints
• 17 technical presentations & 2 panels
• 29 posters
• 15 demos
www.tinymlsummit.org 9
![Page 10: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:](https://reader034.fdocuments.in/reader034/viewer/2022042405/5f1c7066ae81ad2d9c634e55/html5/thumbnails/10.jpg)
tinyML-2019
Committee
![Page 11: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:](https://reader034.fdocuments.in/reader034/viewer/2022042405/5f1c7066ae81ad2d9c634e55/html5/thumbnails/11.jpg)
ANNOUNCEMENT: tinyML Summit 2020
www.tinyMLsummit.org 11
Wei XiongSamsung
Samsung, San Jose
![Page 12: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:](https://reader034.fdocuments.in/reader034/viewer/2022042405/5f1c7066ae81ad2d9c634e55/html5/thumbnails/12.jpg)
tinyML-2020 Summit Objectives and Focus
• Continue to grow tinyML Global Ecosystem – 2x attendance size (wrt the 2019 Summit) while keeping the highest quality event
• Continue to build tinyML awareness
• Start connecting tinyML technologies to end-user products and applications
• More focus on: algorithms and end-user products and applications
• Bring more academic work of fundamental importance for tinyML ecosystem
• More organizational diversity
![Page 13: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:](https://reader034.fdocuments.in/reader034/viewer/2022042405/5f1c7066ae81ad2d9c634e55/html5/thumbnails/13.jpg)
Co-Chairs
Wei XiongSamsung
Technical Program Committee
Operations• Bette Copper• Ira Feldman
Organization
13
![Page 14: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:](https://reader034.fdocuments.in/reader034/viewer/2022042405/5f1c7066ae81ad2d9c634e55/html5/thumbnails/14.jpg)
tinyML Community “DNA”
• Highest Quality: prime tinyML global Community/Events
• Industry focused & driven, with strong academic participation & influence
• “Full stack”/E2E coverage: HW-SYS-Algo-SW-Apps
• Deeply technical with no marketing/sales pitches
• Diverse and collaborative (while respecting privacy)
• Building “tinyML Foundation”, a non-profit Org/Community
![Page 15: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:](https://reader034.fdocuments.in/reader034/viewer/2022042405/5f1c7066ae81ad2d9c634e55/html5/thumbnails/15.jpg)
tinyML Meetup Committee
![Page 16: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:](https://reader034.fdocuments.in/reader034/viewer/2022042405/5f1c7066ae81ad2d9c634e55/html5/thumbnails/16.jpg)
“Let’s make tinyML BIG !”
16
![Page 17: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:](https://reader034.fdocuments.in/reader034/viewer/2022042405/5f1c7066ae81ad2d9c634e55/html5/thumbnails/17.jpg)
17
Objectives
1. Collaborative platform for the tinyML ecosystem• Networking and biz dev opportunities
2. Common tinyML “roadmap”• Shared vision
• Tech pipeline from pre-competitive R&D based on leading edge academic research
3. Growth engine• New use cases and applications
• Start-ups and VCs
4. Benchmarking and standards• To make sure we all speak the same language (e.g. open datasets for benchmarking)
• Esp important as the ecosystem pie grows; scale is impossible w/o standards
5. Workforce dev’t• Training
• Collaboration w/ academia