Serverless Logging with AWS Lambda and the Elastic Stack
-
Upload
edoardo-paolo-scalafiotti -
Category
Internet
-
view
168 -
download
1
Transcript of Serverless Logging with AWS Lambda and the Elastic Stack
![Page 1: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/1.jpg)
SERVERLESS LOGGING WITH ELKEdoardo Paolo Scalafiotti - Software Engineer @ Sainsbury’s
![Page 2: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/2.jpg)
![Page 3: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/3.jpg)
About me
Go, NodeJS
SPAs, WebComponents
Material Design
Serverless Architecture
Microservices
Machine Learning
FabLab, Meetups
![Page 4: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/4.jpg)
AgendaOVERVIEWWHY SERVERLESS?CHALLENGESUSE CASESDEMOQ & A
![Page 5: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/5.jpg)
WHAT IS SERVERLESS?
![Page 6: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/6.jpg)
TL;DRFaaS is about running back end code without managing your own server
systems or your own server applications
![Page 7: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/7.jpg)
applications that run in stateless compute containers that are event-
triggered, ephemeral and fully managed by a 3rd
party
![Page 8: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/8.jpg)
● 2 overlapping areas: BaaS and PaaS● run code without provisioning or
managing servers● horizontal scaling is completely
automatic, elastic, and managed by the provider
● responds to triggers
![Page 9: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/9.jpg)
![Page 10: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/10.jpg)
WHY SHOULD WE CARE?
![Page 11: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/11.jpg)
serverless is
HOT!
![Page 12: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/12.jpg)
Typical servers in business and enterprise data centers deliver between 5 and 15 percent of their maximum computing
output on average over the course of the year
![Page 13: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/13.jpg)
● Reduced operational cost● Good with irregular, high spike
traffic that cannot be handled by auto-scaling
● Promotes best coding practises
![Page 14: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/14.jpg)
WHAT ARE THE CHALLENGES?
![Page 15: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/15.jpg)
TL;DRnot for the faint-hearted... at the moment
![Page 16: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/16.jpg)
● Ugly hacks for keeping the function alive
● Vendor lock-in● Unit testing is ok, integration... another
story● Repetition of logic● Concurrent limit● Logging, Monitoring and Debugging
![Page 17: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/17.jpg)
USE CASES
![Page 18: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/18.jpg)
USE WISELYFaaS, containers and VMs are not
enemies
![Page 19: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/19.jpg)
FAAS is seen as a better choice for event-driven apps with few event types
per application component,while CONTAINERS are seen as a better
choice for synchronous-request driven components with many entry points
![Page 20: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/20.jpg)
VM Containers ServerlessConfigure machines, storage,
networking and OS
Run servers, configure
applications and control scaling
Run my code when it’s needed
![Page 21: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/21.jpg)
Use cases:● Real-time file processing (ingestion)● Real-time stream processing (mkt.
data)● Extract, Transform, Load● IoT Backends● Mobile Backends● RESTful APIs
![Page 22: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/22.jpg)
● Open Source alternative: IBM’s OpenWhisk
● API Orchestration: serverless.com and claudia.js
● Multiple PL support: Apex● Logging and monitoring: ELK
![Page 23: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/23.jpg)
LET’S GET TO THE CODE...
![Page 24: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/24.jpg)
THE MISSION...is to monitor a set of Lambda-based
functions that generates Game Of Thrones plot points
![Page 25: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/25.jpg)
![Page 26: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/26.jpg)
DEMO
![Page 27: Serverless Logging with AWS Lambda and the Elastic Stack](https://reader031.fdocuments.in/reader031/viewer/2022022414/587458bd1a28abd53e8b91cf/html5/thumbnails/27.jpg)
github.com/edoardo849/lot