DevOps automation for AWS
-
Upload
sachin-dole -
Category
Technology
-
view
483 -
download
1
Transcript of DevOps automation for AWS
![Page 1: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/1.jpg)
AWS SDKDevOps AutomationSachin Dole
![Page 2: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/2.jpg)
About me
▪ AWS Certified Solutions Architect
▪ Independent AWS Consultant – Genvega Inc.
▪ Trelair.com – now defunct startup
▪ www.cricclubs.com – ½ million+ page views per month
▪ Backup & Archival on S3 – project for law firm
▪ EC2 automation for image processing
▪ Available for AWS consulting – find me: www.genvega.com
![Page 3: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/3.jpg)
About CricClubs (My customer)
30th Jan
2016@2015 CRICCLUBS.COM | www.cricclubs,.com | CONFIDENTIAL 3
Started in 2008
Geographic Usage20+ Countries
600+ Leagues 100,000+ Players
HeadquartersMilwaukee, WI,
USA
Features50+ features on
Website and iOS, Android, Windows
Mobile Apps
Premium Support24/7 via E-mail, Phone andTicketing Systems
Data CentersHigh Availability with servers across 12 Data Centers
4 million pageviews/month
![Page 4: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/4.jpg)
About you?
▪ Have day to day AWS experience?
▪ Have personal projects on AWS?
▪ Want a good introduction to DevOps & AWS?
![Page 5: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/5.jpg)
Console access
▪ https://tvarit.signin.aws.amazon.com/console
▪ ijugdemo
▪ devops123
▪ Read only access to all services
![Page 6: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/6.jpg)
Topics
▪ DevOps for Trelair
▪ Lessons learnt
▪ Tvarit.io & AWS Java SDK
▪ Questions?
![Page 7: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/7.jpg)
Trelair.com
![Page 8: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/8.jpg)
System Architecture
Clients
ELB
Wildfly on EC2
S3 & Postgres
External
Services
![Page 9: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/9.jpg)
Building Trelair
Commit
Build
FTP
Deploy
Test
Backup & Release
1 to 8 person-hrs
DML Disk Compute DB App server Backups
![Page 10: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/10.jpg)
Commit
Build
FTP
Deploy
Test
Release
Enterprises?
1 to 8 person years
DML Disk Compute DB App server Backups
Dev
EA
Test Ops
EA
![Page 11: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/11.jpg)
Trelair Automation
GitCommit• Teamcity
• JBossDMR
Auto-Deploy
SeleniumTest &
Release
DevOps + 0 person hoursCustom Maven Plugin
Used AWS Java SDK
AMI DML/DDL DNS Snapshots ScalingAuto
![Page 12: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/12.jpg)
Automated Flow
A. Commit Code (the only manual step)
B. CI build
C. DB DML patches are applied to replica
D. Deploy code to candidate server
E. UI tests run
F. DML patches applied to production
G. Candidate server is added to cluster
H. Old servers are terminated
![Page 13: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/13.jpg)
AWS API
us - eastAWS Java
SDK
JBoss DMR Wildfly AS
Provision
Deploy
![Page 14: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/14.jpg)
Lessons
▪ Automation is easy – mindset is difficult
▪ Better to automate in small increments – big bang is a waste
▪ Automate everything – partial automation is overhead
▪ Release frequently – rely on automation
▪ Use selenium automation (e.g. saucelabs.com)
▪ Rely on AWS products
![Page 15: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/15.jpg)
Tvarit
▪ http://www.tvarit.io, https://github.com/sdole/tvarit-maven
▪ Trelair automation on steroids, opensource
▪ Automated provisioning, deployment and scaling
▪ Alpha release available in Maven Central
▪ Free support through GitHub issues
▪ Working α version & a deep roadmap
![Page 16: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/16.jpg)
Console access
▪ https://tvarit.signin.aws.amazon.com/console
▪ ijugdemo
▪ devops123
▪ Read only access to all services
![Page 17: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/17.jpg)
Tvarit – demo
▪ CloudFormation templates
▪ Maven goals
▪ VPC, IAM console
▪ EC2: LC, ASG, ELB
▪ S3 bucket
▪ Cloudwatch monitoring, flow logs
▪ Code walkthrough
![Page 18: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/18.jpg)
API structure
▪ Many flavors – Java, C++, JS, Ruby, Python, CLI etc.
▪ APIs are RESTful
▪ Deep documentation
![Page 19: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/19.jpg)
API Structure
AmazonEc2Client ec2 =
new AmazonEc2Client(new BasicAwsCredentials());
SomeEc2Request req = new SomeEc2Request();
req.withSomething(thing1).withSomething(attr1);
SomeEc2Result result = ec2.submitRequest(req);
inspectWithYourLogic(result);
![Page 20: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/20.jpg)
Automation Security - Individuals
IAMus - east
us - west
Access
Keys
groups
users
roles
Authorized
![Page 21: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/21.jpg)
Automation Security - Services
IAMOther
services
Other
services
ec2
lambda
S3
Assumed
RoleAuthorized
![Page 22: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/22.jpg)
AWS Automation Features
▪ Autoscaling
▪ Monitoring
▪ Alarms
▪ Cloudwatch Events
▪ Lambda Functions
▪ S3 Events, SQS and SNS events
▪ SWF
▪ CloudFormation
![Page 23: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/23.jpg)
Tvarit Target Architecture
Clients
WAF
ELB
EC2
ELB
EC2
Lambda
Java SDKAutoscaling
Cloudwatch
Zookeeper
S3, RDS
S3, RDS
Cloudformation
ELB
Router
App 1
Version 1
App 2
Version 1
![Page 24: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/24.jpg)
Other concepts:Blue-Green Deployment
http://martinfowler.com/bliki/BlueGreenDeployment.html
![Page 25: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/25.jpg)
Other concepts:Canary Release
http://martinfowler.com/bliki/CanaryRelease.html
![Page 26: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/26.jpg)
Revisit Tvarit Target Architecture
Clients
WAF
ELB
EC2
ELB
EC2
Lambda
Java SDKAutoscaling
Cloudwatch
Zookeeper
S3, RDS
S3, RDS
Cloudformation
ELB
Router
App 1
Version 1
App 2
Version 1
![Page 27: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/27.jpg)
Tvarit Release concept (future)
ELB
ELB
Router
App Versions
Router: routes customers based on Http Headers to correct app version
![Page 28: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/28.jpg)
Step Back – what is (not) DevOps?
Is…
Infrastructure as code
Not…
Auto-scale, failover, monitoring
Continuous delivery
Dedicated teams, Dev, Test, Ops
Forms, Emails, Release plans
Infrastructure Architects/EA
![Page 29: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/29.jpg)
Soft side of DevOps
▪ Cloud-native vs Lift ‘n shift
▪ CALMS
▪ C – Culture
▪ A – Automation
▪ L – Lean
▪ M – Measurement
▪ S – Sharing
▪ Automation as Kaizen – not a single project
![Page 30: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/30.jpg)
Costs of AWS Automation
▪ In most cases – FREE
▪ Priced as usual: Compute, Storage, Monitoring, Network
▪ Monitoring is a must
▪ CPU, Memory, Network, Disk…
▪ Health checks, failed hardware, traffic spikes
▪ Audit trail – API actions, modifications
▪ Configuration changes
![Page 31: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/31.jpg)
How to improve Tvarit
▪ Open an Issue on GitHub:
“We use a customized version of tomcat – can you make the app server container configurable so we can use our own?”
Fork it on GitHub and DIY
Email me
![Page 32: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/32.jpg)
Other things
▪ Chef, Puppet, Ansible, SaltStack
▪ AWS OpsWorks, Beanstalk
▪ AWS CodeCommit, CodeDeploy
![Page 33: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/33.jpg)
![Page 34: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/34.jpg)
How to find me
@sdoledotcom
https://linkedin.com/in/sdole
http://genvega.com/blog/
https://github.com/sdole/
![Page 35: DevOps automation for AWS](https://reader030.fdocuments.in/reader030/viewer/2022020203/58f0d21b1a28ab677e8b4657/html5/thumbnails/35.jpg)
ΩThank you