END-TO-END HANDS-ON EXPERIENCE ON
Transcript of END-TO-END HANDS-ON EXPERIENCE ON
![Page 1: END-TO-END HANDS-ON EXPERIENCE ON](https://reader031.fdocuments.in/reader031/viewer/2022011813/61d4a36de2274d7c1658ca19/html5/thumbnails/1.jpg)
![Page 2: END-TO-END HANDS-ON EXPERIENCE ON](https://reader031.fdocuments.in/reader031/viewer/2022011813/61d4a36de2274d7c1658ca19/html5/thumbnails/2.jpg)
END-TO-END HANDS-ON
EXPERIENCE ON
MICRO/SERVERLESS
ARCHITECTURE WITH AWS
Liam Lai (賴怡誠)
2019/5/15
PPT: https://ppt.cc/fsEk3x
![Page 3: END-TO-END HANDS-ON EXPERIENCE ON](https://reader031.fdocuments.in/reader031/viewer/2022011813/61d4a36de2274d7c1658ca19/html5/thumbnails/3.jpg)
OUTLINE Conclusion
About me
MicroService – Architecture
Log journey
AWS cost optimization – ECS
Debug journey
Serverless use case and pain point
Conclusion
PPT: https://ppt.cc/fsEk3x
![Page 4: END-TO-END HANDS-ON EXPERIENCE ON](https://reader031.fdocuments.in/reader031/viewer/2022011813/61d4a36de2274d7c1658ca19/html5/thumbnails/4.jpg)
ABOUT ME
Optus Linux Administration (6m)
Vsphere
Puppet / Ansible
Tabcorp Devops -> API (2y)
AWS – Cloudformation, ECS
CDN – Cloudfront, Akamai
CICD – Buildkite, GoCD
NodeJS, Golang, Postgres
PPT: https://ppt.cc/fsEk3x
![Page 5: END-TO-END HANDS-ON EXPERIENCE ON](https://reader031.fdocuments.in/reader031/viewer/2022011813/61d4a36de2274d7c1658ca19/html5/thumbnails/5.jpg)
CONCLUSION Pros - Deployment/ Maintenance/ Cost can have massive improvement
Production Deployment less then 1 min
Legacy System is rewritable
Optimize more -> Pay less -> Get more
Cons - Infrastructure become mass and messy
hard to understand infra
More time on trouble shoot
Heavily rely on platform provider
Log can be huge
![Page 6: END-TO-END HANDS-ON EXPERIENCE ON](https://reader031.fdocuments.in/reader031/viewer/2022011813/61d4a36de2274d7c1658ca19/html5/thumbnails/6.jpg)
MONOLITHIC SERVICE
![Page 7: END-TO-END HANDS-ON EXPERIENCE ON](https://reader031.fdocuments.in/reader031/viewer/2022011813/61d4a36de2274d7c1658ca19/html5/thumbnails/7.jpg)
MICRO SERVICE
![Page 8: END-TO-END HANDS-ON EXPERIENCE ON](https://reader031.fdocuments.in/reader031/viewer/2022011813/61d4a36de2274d7c1658ca19/html5/thumbnails/8.jpg)
PRODUCTION ARCHITECTURE
CDN
• CACHE CONTENT
• BASIC DDOS PROTECT
WAF - SECURITY
• DDOS,
• IP RATE LIMIT
• WHITE/BLOCK LIST
• SQL INJECTION / XSS
API-GATEWAY
• AUTHENTICATION
• ROUTING
![Page 9: END-TO-END HANDS-ON EXPERIENCE ON](https://reader031.fdocuments.in/reader031/viewer/2022011813/61d4a36de2274d7c1658ca19/html5/thumbnails/9.jpg)
AWS
Service
Log server
Splunk/ES
Firehose Storage
(S3)
(NAS)
Pre-
Process
Cloud
watch
Rsyslog
Fluentd
LOG JOURNEY
![Page 10: END-TO-END HANDS-ON EXPERIENCE ON](https://reader031.fdocuments.in/reader031/viewer/2022011813/61d4a36de2274d7c1658ca19/html5/thumbnails/10.jpg)
LOG JOURNEY
AWS Resource • CloudWatch
• Lambda
• Firehose $
Log collector • Fluentd ( prefer )
• Logstash
• Rsyslog ( not recommend )
Search Service • Splunk $$$ : Fast, Chart, Alarm, Statistic
• Elastic Search $$ : Easy, On-Demand, Chart, additional log retention
• Athena. $ : Slow, no visual
Log Storage
• S3 -> Glacier
• On-Premier
![Page 11: END-TO-END HANDS-ON EXPERIENCE ON](https://reader031.fdocuments.in/reader031/viewer/2022011813/61d4a36de2274d7c1658ca19/html5/thumbnails/11.jpg)
AWS COST OPTIMIZATION - ECS
EC2 Spot instance (50% – 80%)
Situation - • Auto failover, minimum recover time
• DevOps effort for non-downtime
• Bigger instance
• https://github.com/AutoSpotting/AutoSpotting
None ALB (2% - 10%)
• AWS Service Discover or 3rd Party (consul)
AWS Trusted Advisor
• Provision Instance/storage right size
• Design scaled service scale right time
![Page 12: END-TO-END HANDS-ON EXPERIENCE ON](https://reader031.fdocuments.in/reader031/viewer/2022011813/61d4a36de2274d7c1658ca19/html5/thumbnails/12.jpg)
SERVICE DEBUG JOURNEY
interpreted languages
• Edit online/ manual start, test
compiled languages
• Provided artifacts credential
• Build Container from base image
• Edit online -> Start -> Test
• AWS Lambda layer -> Edit Via Console
• Build Container Backup
LAMBDA
ECS
![Page 13: END-TO-END HANDS-ON EXPERIENCE ON](https://reader031.fdocuments.in/reader031/viewer/2022011813/61d4a36de2274d7c1658ca19/html5/thumbnails/13.jpg)
SERVERLESS USE CASE AND PAIN POINT
Use case
• Schedule Worker
• Request happens occasionally
• Not real time perform require
• Platform Event Trigger (s3, cloudwatch…)
Pain Point
• Hard to Trouble Shoot
• Platform Depends (API-Gateway, VPC, network interface, IAM)
• Unpredictable
![Page 14: END-TO-END HANDS-ON EXPERIENCE ON](https://reader031.fdocuments.in/reader031/viewer/2022011813/61d4a36de2274d7c1658ca19/html5/thumbnails/14.jpg)
CONCLUSION Pros - Deployment/ Maintenance/ Cost can have massive improvement
Production Deployment less then 1 min
Legacy System is rewritable
Optimize more -> Pay less -> Salary more
Cons - Infrastructure become mass and messy
Hard to understand infra
More time on trouble shoot
Heavily rely on platform provider
Log can be huge
![Page 15: END-TO-END HANDS-ON EXPERIENCE ON](https://reader031.fdocuments.in/reader031/viewer/2022011813/61d4a36de2274d7c1658ca19/html5/thumbnails/15.jpg)
CONCLUSION
Advise
Allow yourself more time for first try.
Always follow up the latest release on your provider
![Page 16: END-TO-END HANDS-ON EXPERIENCE ON](https://reader031.fdocuments.in/reader031/viewer/2022011813/61d4a36de2274d7c1658ca19/html5/thumbnails/16.jpg)
THANKS