A Decentralized Framework for Serverless Edge Computing in ...
Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way...
Transcript of Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way...
![Page 1: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/1.jpg)
CS 675: Distributed Systems (Spring 2020)Lecture 7
Yue Cheng
Some material taken/derived from: • Berkeley CS 262a (Spring ‘18) by Ali Ghodsi and Ion Stoica;• Tyler Harter’s HotCloud ’18 OpenLambda talk;• CSE 291 by Yiying Zhang.Licensed for use under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
Serverless Computing: Background & Trends
![Page 2: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/2.jpg)
Utility computing• “… computing may someday be organized as a
public utility just as the telephone system is a public utility… Each subscriber needs to pay only for the capacity he actually uses, but he has access to all programming languages characteristic of a very large system. … Certain subscribers might offer service to other subscribers. … The computer utility could become the basis of a new and important industry.”
- John McCarthy, speaking at the MIT Centennial in 1961
Y. Cheng GMU CS675 Spring 2020 2
![Page 3: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/3.jpg)
Utility computing
• It’s the 1960s, good computers are super expensive, almost no one has one
• But the expensive hardware stands idle most of the time
• Want to make the power of good computers available to everyone
• Technical issues around pricing, security, and management
Y. Cheng GMU CS675 Spring 2020 3
![Page 4: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/4.jpg)
Decline of utility computing
• Good computers get super cheap
• Even small organizations can run their own computing utilities
• Security and management issues are hard to overcome
Y. Cheng GMU CS675 Spring 2020 4
![Page 5: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/5.jpg)
Cloud computing
• Computing as a utility• Outsourced to a third party or internal organization
• Providers do more, tenants do less
Y. Cheng GMU CS675 Spring 2020 5
![Page 6: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/6.jpg)
Y. Cheng GMU CS675 Spring 2020 6
![Page 7: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/7.jpg)
#thecloudistoodamnhard
• What type of instances?
• How many to spin up?
• What base image?
• What price spot?
• And then wait to start…
Y. Cheng GMU CS675 Spring 2020 7
![Page 8: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/8.jpg)
Decision paralysis? Go for Serverless Computing!
Y. Cheng GMU CS675 Spring 2020 8
Microsoft Azure Functions
Google Cloud Functions
![Page 9: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/9.jpg)
What is serverless computing?
Serverless computing is a programming abstraction that enables users to upload programs, run them at (virtually) any scale, and pay only for the resources used
Y. Cheng GMU CS675 Spring 2020 9
![Page 10: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/10.jpg)
Who cares?
• Serverless computing is rapidly growing in popularity
Y. Cheng GMU CS675 Spring 2020 10
*: Serverless Computing: One Step Forward, Two Steps Back. Hellerstein et al., CIDR ‘19.
![Page 11: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/11.jpg)
Who cares?
• Serverless computing is rapidly growing in popularity
• Change the way we write applications and expose new challenges
• Predictions:“the future of AWS”
– Marvin Theimer, VP/Distinguished engineer at AWS
“will grow to dominate the future of cloud computing”– Paper “A Berkeley View on Serverless Computing”
Y. Cheng GMU CS675 Spring 2020 11
![Page 12: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/12.jpg)
Core capability1. (Provider) Manage a set of user-defined functions2. Take an event sent over HTTP or received from an
event source3. Determine function(s) to which to dispatch the
event4. Find an existing instance of function or create a
new one5. Send the event to the function instance6. Wait for a response7. Gather execution logs8. Make the response available to the user9. Stop the function when it is no longer needed
Y. Cheng GMU CS675 Spring 2020 12
![Page 13: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/13.jpg)
Basic architecture
Y. Cheng GMU CS675 Spring 2020 13
![Page 14: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/14.jpg)
A virtualization story
Y. Cheng GMU CS675 Spring 2020 14
Virtualizing the server
Virtualizing the cluster
Virtualizing the cloud
![Page 15: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/15.jpg)
Classic web stack
Y. Cheng GMU CS675 Spring 2020 15
Application
Server
Operating system
Hardware
RPCs
Weak virtualization
![Page 16: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/16.jpg)
1st generation: virtual machine (VM)
Y. Cheng GMU CS675 Spring 2020 16
Application
Server
Operating system
Hardware
RPCs
Virtual hardware
![Page 17: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/17.jpg)
1st generation: virtual machine (VM)
Y. Cheng GMU CS675 Spring 2020 17
Application
Server
OS
Hardware
RPCs
Virtual hardware
Application
Server
OS
![Page 18: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/18.jpg)
2nd generation: containers
Y. Cheng GMU CS675 Spring 2020 18
Application
Server
Hardware
RPCs
Virtual OS
Application
Server
Operating system
![Page 19: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/19.jpg)
3rd generation: serverless functions
Y. Cheng GMU CS675 Spring 2020 19
Application
Hardware
RPCsVirtual servers
Application
Operating system
Server
![Page 20: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/20.jpg)
3rd generation: serverless functions
Y. Cheng GMU CS675 Spring 2020 20
FnA
Hardware
RPCsVirtual servers
Operating system
Server
FnZ Fn0 Fn9… …
![Page 21: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/21.jpg)
Tradeoff
Y. Cheng GMU CS675 Spring 2020 21
Serverless functions(Lambdas) Containers VMs
Isolation??
Flexibility
Overhead(resources, startup, …)
![Page 22: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/22.jpg)
First serverless platform/app
• Google App Engine • Fully managed platform as a service
(PaaS) for developing and hosting web applications
• Google BigQuery• Fully managed data warehouse
• “Arbitrarily” large data and queries
• Pay per bytes being processed
• No concept of server or cluster
Y. Cheng GMU CS675 Spring 2020 22
![Page 23: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/23.jpg)
AWS Lambda
• Lambda functions: typically read/write data from/to S3
300 seconds 900 seconds (15 minutes)
single-core two-core
1.5 GB 3 GB memory
512 MB in /tmp
Python, Java, Node.js, Go, …
Pricing: • Sub-second billing: 100-millisecond billed duration
• $0.20 per 1M requests (invocations)
• $0.0000166667 for every GB-second
• 6,000 1 GB Lambda functions for one second: 10¢
Y. Cheng GMU CS675 Spring 2020 23
![Page 24: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/24.jpg)
Why is serverless computing desirable?
• Zero administration overhead• No need to handle server provisioning, failure, etc.
• Auto-scaling• Spin up / tear down functions quickly based on load
• Pay-per-use• Only pay for the resources used (CPU-mem bundle)
Y. Cheng GMU CS675 Spring 2020 24
![Page 25: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/25.jpg)
Limitations
• Question: Can developers really focus on the function development now?
Y. Cheng GMU CS675 Spring 2020 25
![Page 26: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/26.jpg)
Limitations
• Question: Can developers really focus on the function development now?
• In some sense, yes ?
• Tradeoff: but typically requires:• extra work
• slow performance
Y. Cheng GMU CS675 Spring 2020 26
![Page 27: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/27.jpg)
What’s missing?
• No efficient & explicit way of inter-function communication• No support for state sharing
• No explicit support for data transfer
• Current best practices heavily rely on cloud storage for facilitating inter-function communication
• Active research area• Wukong: https://github.com/mason-leap-lab/Wukong
Y. Cheng GMU CS675 Spring 2020 27
![Page 28: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/28.jpg)
How is communication a challenge?
Y. Cheng GMU CS675 Spring 2020 28
![Page 29: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/29.jpg)
Communication is the bottleneck
Y. Cheng GMU CS675 Spring 2020 29
0 200 400 600 800 1000
Enabled
Disabled
Time (seconds)
Redis I/O
Invoking Tasks
Serialization/Deserialization
Task Execution
Publishing Redis Pub/SubMessages
Aggregated time (seconds) breakdown for executing a complex DAG computation
Singular vector decomposition’s (SVD) DAG
![Page 30: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/30.jpg)
How about NAT hole punching?
Y. Cheng GMU CS675 Spring 2020 30
• An interesting research direction to investigate• Our current effort• Porting Wukong to use NAT
punching-based P2P communication• Challenging to coordinate
sender and receiver Lambda functions• Resulted in 10X +
performance degradation
Singular vector decomposition
![Page 31: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/31.jpg)
What’s missing?
• No good way of managing state• Rely on cloud storage for state sharing but cloud
storage is slow
• Active research area• InfiniCache: https://github.com/mason-leap-lab/infinicache
• Pocket:https://www.usenix.org/conference/osdi18/presentation/klimovic
Y. Cheng GMU CS675 Spring 2020 31
![Page 32: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/32.jpg)
What’s missing?
• Inflexible and poor resource configuration• Impossible to run large code, e.g., TensorFlow
• Unfriendly to resource-intensive code, e.g., ML/DL
Y. Cheng GMU CS675 Spring 2020 32
![Page 33: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/33.jpg)
What’s missing?
• Unpredictable performance
• Long cold startup latencies
Y. Cheng GMU CS675 Spring 2020 33
*: Peeking Behind the Curtains of Serverless Platforms. Wang et al., USENIX ATC ‘18
![Page 34: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/34.jpg)
What’s missing?
• Unpredictable performance
• Long cold startup latencies
Y. Cheng GMU CS675 Spring 2020 34
* Blog: Comparison of Cold Starts in Serverless Functions across AWS, Azure, and GCP, Mikhail Shilkov
![Page 35: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/35.jpg)
What’s missing?
• Unpredictable performance
• Long cold startup latencies• Long HTTP invocation latencies
Y. Cheng GMU CS675 Spring 2020 35
![Page 36: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/36.jpg)
What’s missing?
• Unpredictable performance
• Long cold startup latencies• Long HTTP invocation latencies
• Active research area• SAND: https://www.usenix.org/conference/atc18/presentation/akkus• SOCK: https://www.usenix.org/conference/atc18/presentation/oakes
Y. Cheng GMU CS675 Spring 2020 36
![Page 37: Serverless Computing · •Serverless computing is rapidly growing in popularity •Change the way we write applications and expose new challenges •Predictions: “the future of](https://reader035.fdocuments.in/reader035/viewer/2022071218/604fc8fd8604c2688712c6d8/html5/thumbnails/37.jpg)
Discussion
• Q1: Which of the “Limitations of Today’s Serverless Computing Platforms” does your project address, and how?
• Q2: How does your project’s direction correspond with the suggestions in “What Serverless Computing Should Become”?
• Project status: Give one question about your project you’d like class feedback about
Y. Cheng GMU CS675 Spring 2020 37