IoT Meets the Cloud Ali Ghodsi UC Berkeley & KTH & SICS [email protected].
-
Upload
nora-ramson -
Category
Documents
-
view
220 -
download
0
Transcript of IoT Meets the Cloud Ali Ghodsi UC Berkeley & KTH & SICS [email protected].
![Page 2: IoT Meets the Cloud Ali Ghodsi UC Berkeley & KTH & SICS alig@cs.berkeley.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062303/5517c0dd5503461b658b4791/html5/thumbnails/2.jpg)
Cloud Computing?
• Larry Ellison, CEO of Oracle Corporation
“The computer industry is the only industry that is more fashion-driven than women's fashion. Maybe I'm an idiot, but I have no idea what anyone is talking about. What is it? It's complete gibberish. It's insane. When is this idiocy going to stop?”
• Richard M. Stallman, President of FSF
“It’s stupidity. It’s worse than stupidity: it’s a marketing hype campaign. Somebody is saying this is inevitable — and whenever you hear somebody saying that, it’s very likely to be a set of businesses campaigning to make it true.”
• My claim:– Cloud computing is inevitable for the Internet-of-Things
![Page 3: IoT Meets the Cloud Ali Ghodsi UC Berkeley & KTH & SICS alig@cs.berkeley.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062303/5517c0dd5503461b658b4791/html5/thumbnails/3.jpg)
Mobile Applications
Most of the Computation on the Cloud Already!
![Page 4: IoT Meets the Cloud Ali Ghodsi UC Berkeley & KTH & SICS alig@cs.berkeley.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062303/5517c0dd5503461b658b4791/html5/thumbnails/4.jpg)
Do we need the cloud for IoT?
• Device deluge– 3 billion smart phones – Another 40 billion IoT devices
• Devices will be challenged– Limited storage– Limited processing– Limited communication – Limited energy
Clouds needed for IoT, just as for phones and desktops
![Page 5: IoT Meets the Cloud Ali Ghodsi UC Berkeley & KTH & SICS alig@cs.berkeley.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062303/5517c0dd5503461b658b4791/html5/thumbnails/5.jpg)
What is the cloud?
• Datacenter Computing– Thousands of servers– Co-located storage– Routers and switches– Backup power
supplies– Cooling
![Page 6: IoT Meets the Cloud Ali Ghodsi UC Berkeley & KTH & SICS alig@cs.berkeley.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062303/5517c0dd5503461b658b4791/html5/thumbnails/6.jpg)
Why do we need datacenters?
• Multi-core Computing– Processing speed stagnation– Increased parallelism– Supercomputer not sufficient
• Parallel computing quintessential to cloud computing– Request-level parallelism – Parallel algorithms
(MapReduce, Indexing …)
![Page 7: IoT Meets the Cloud Ali Ghodsi UC Berkeley & KTH & SICS alig@cs.berkeley.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062303/5517c0dd5503461b658b4791/html5/thumbnails/7.jpg)
Why do we need datacenters? (2)
• Economy of scale– Reduce server cost– Reduce cooling cost– Reduce power cost
• Clouds are efficient– PUE = total_facility_power/
equipment_power ~ 1.2– Energy economy-of-scale– Commodity servers– Workload consolidation
![Page 8: IoT Meets the Cloud Ali Ghodsi UC Berkeley & KTH & SICS alig@cs.berkeley.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062303/5517c0dd5503461b658b4791/html5/thumbnails/8.jpg)
Workload Consolidation
• Data replicated over commodity machines– Pioneered by Inktomi
• Interactive and latency sensitive jobs– User facing applications
e.g. search queries, tweets, …– Millisecond SLOs
• Batch-jobs– Building search indexes …– Analytics of trends, business data …– AV/spam filtering …
![Page 9: IoT Meets the Cloud Ali Ghodsi UC Berkeley & KTH & SICS alig@cs.berkeley.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062303/5517c0dd5503461b658b4791/html5/thumbnails/9.jpg)
Workload Consolidation (2)
• Interactive and batch on same machines– Virtualization of computation
e.g. migration, hardware agnosticism
– Isolation of workloadse.g. meet SLO guarantees
– Automatic fault-handling e.g. through replication
![Page 10: IoT Meets the Cloud Ali Ghodsi UC Berkeley & KTH & SICS alig@cs.berkeley.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062303/5517c0dd5503461b658b4791/html5/thumbnails/10.jpg)
Transformation of Computing
• Datacenter as a computer– Programs timeshare thousands
of servers
![Page 11: IoT Meets the Cloud Ali Ghodsi UC Berkeley & KTH & SICS alig@cs.berkeley.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062303/5517c0dd5503461b658b4791/html5/thumbnails/11.jpg)
Berkeley Vision
• Create an “Operating System Kernel” for the Datacenter Computer– First step with Mesos (mesosproject.org)
![Page 12: IoT Meets the Cloud Ali Ghodsi UC Berkeley & KTH & SICS alig@cs.berkeley.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062303/5517c0dd5503461b658b4791/html5/thumbnails/12.jpg)
Today’s Cloud Frameworks
• Frameworks simplify distributed programming– Programming models– Hide failures, synchronization, delay variance
Dryad
Pregel
Each framework runs on a dedicated cluster/partition
![Page 13: IoT Meets the Cloud Ali Ghodsi UC Berkeley & KTH & SICS alig@cs.berkeley.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062303/5517c0dd5503461b658b4791/html5/thumbnails/13.jpg)
One Framework Per Cluster Challenges
• Inefficient resource usage– E.g., Hadoop cannot use available
resources from IoT FW cluster– No opportunity for stat. multiplexing
• Hard to share data– Copy or access remotely, expensive
• Hard to cooperate– E.g., Not easy for IoT FW to use data
generated by Hadoop
Hadoop
IoT FW
Hadoop
IoT FW
Need to run multiple frameworks on the same cluster
![Page 14: IoT Meets the Cloud Ali Ghodsi UC Berkeley & KTH & SICS alig@cs.berkeley.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062303/5517c0dd5503461b658b4791/html5/thumbnails/14.jpg)
Solution: Mesos
• Common resource sharing layer – abstracts (“virtualizes”) resources to frameworks– enable diverse frameworks to share cluster
IoT FWHadoo
p
IoT FWHadoo
pMesos
Uniprograming Multiprograming
![Page 15: IoT Meets the Cloud Ali Ghodsi UC Berkeley & KTH & SICS alig@cs.berkeley.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062303/5517c0dd5503461b658b4791/html5/thumbnails/15.jpg)
IoT Framework Diversity
• Today’s frameworks tailored for specific application domains–MapReduce for indexing and filtering– Pregel for graph algorithms
• IoT problem domain highly diverse– Existing frameworks poor fit for IoT
![Page 16: IoT Meets the Cloud Ali Ghodsi UC Berkeley & KTH & SICS alig@cs.berkeley.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062303/5517c0dd5503461b658b4791/html5/thumbnails/16.jpg)
New IoT Frameworks for Clouds
• IoT framework requirements– Efficient device tag matching and filtering– Online stream processing of IoT data– Offline storage and batch processing of IoT
data
Goal: Build first cloud framework for IoT
![Page 17: IoT Meets the Cloud Ali Ghodsi UC Berkeley & KTH & SICS alig@cs.berkeley.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062303/5517c0dd5503461b658b4791/html5/thumbnails/17.jpg)
IoT Framework Applications
• Real time stream processing of data– Security, safety, health applications– Locating people, devices, objects
![Page 18: IoT Meets the Cloud Ali Ghodsi UC Berkeley & KTH & SICS alig@cs.berkeley.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062303/5517c0dd5503461b658b4791/html5/thumbnails/18.jpg)
IoT Framework Applications (2)
• Batch processing of big data– Learning trends, patterns, anomalies– Collaborative filtering/recommendation– Computing global device statistics
![Page 19: IoT Meets the Cloud Ali Ghodsi UC Berkeley & KTH & SICS alig@cs.berkeley.edu.](https://reader036.fdocuments.in/reader036/viewer/2022062303/5517c0dd5503461b658b4791/html5/thumbnails/19.jpg)
Summary
• Dichotomy: – Challenged IoT vs Powerful Clouds
• ”nerves”—sensors, actuators—collect and send data to the ”brain”—the datacenter
• Datacenter is the new super computer– Will need to multiplex between many IoT FW– Need IoT-tailored frameworks to aid IoT
services