AtlasCamp 2016: Art of PaaS - Lessons learned running a platform for hundreds of microservices
43
ROBIN FERNANDES • MICROS TEAM LEAD • ATLASSIAN • @REWBS Art of PaaS: Lessons learned running “Micros”, a platform for hundreds of microservices
-
Upload
robin-fernandes -
Category
Software
-
view
240 -
download
0
Transcript of AtlasCamp 2016: Art of PaaS - Lessons learned running a platform for hundreds of microservices
ROBIN FERNANDES • MICROS TEAM LEAD • ATLASSIAN • @REWBS
Art of PaaS:Lessons learned running “Micros”,
a platform for hundreds of microservices
Microservices,PaaS
&Connect
PaaS Overview
•‿•> <
PaaS Overview
CloudFormation RDS S3 DynamoDB
EC2
Over
services running right now
430
• Big cool statistic
• 2,569
• Add-Ons in Marketplace
Todo: better image
Why do we need a Platform?
Data Centre
JIRA & Confluence Cloud Architecture
R a c k
Cloud instance in OpenVZ container
Atlassian Connect
== Service
Decomposition
Breaking up the Monoliths
* *
Data Centre
R a c k
BlobStore service
PaaS
Breaking up the Monoliths
> <
Data Centre
R a c k
PaaS
_ _ - - /oo\
o o
. .
The platform is set to grow
PaaS
• Big cool statistic
• 2,569
• Add-Ons in Marketplace
Todo: better image
What does the platform do?
+ SD{ }Artifact Descriptor
Deploying a Service
Deploying a Servicename: Micros Workshop
description: A sample app
links:
binary:
type: npm
artifactId: '@atlassian/micros-workshop'
version: 0.0.1
healthcheck:
uri: /healthcheck
source:
url: 'ssh://[email protected]/x.git'
notifications:
email: [email protected]
owners:
organization: Engineering Services
+ SD{ }Artifact Descriptor
https://<service>.atlassian.io
Deploying a Service
Compute: CloudFormation stack with: • Route 53 DNS record • Elastic Load Balancer • EC2 instances in Autoscaling group
Resources: • S3, RDS, DynamoDB • Redis, Memcached, SQS…
Logging & Monitoring
AWSCloudWatch
https://<service>.atlassian.io
. .
cloudymccloudface.atlassian.io
> micros service:deploy my-service -f service-descriptor.yml
Deploying a Service
• Big cool statistic
• 2,569
• Add-Ons in Marketplace
Lessons learned: Balancing consistency & flexibility
Where to favor rules & consistency?
Enforce: Common logging & monitoring infra
Encourage: Structured log data
Structured Log Data
Where to favor rules & consistency?
Enforce: Common logging & monitoring infra
Encourage: Structured log data
Enforce: Standard upgrade flow
https://<service>.atlassian.io
Common Upgrade flow
v1 v2
Where to favor rules & consistency?
Enforce: Common logging & monitoring infra
Encourage: Structured log data
Enforce: Standard upgrade flow
Enforce: Limited set of persistent data stores
Where to favor rules & consistency?
Enforce: Common logging & monitoring infra
Encourage: Structured log data
Enforce: Standard upgrade flow
Enforce: Limited set of persistent data stores
Encourage:Resource isolation
Resource IsolationServiceB.atlassian.io
Datastores
ServiceA.atlassian.io
Datastores
Where to favor rules & consistency?
Enforce: Common logging & monitoring infra
Encourage: Structured log data
Enforce: Standard upgrade flow
Enforce: Limited set of persistent data stores
Enforce: Common service metadata& cost allocation
Encourage:Resource isolation
Where to favor rules & consistency?
Enforce: Common logging & monitoring infra
Encourage: Structured log data
Enforce: Standard upgrade flow
Enforce: Limited set of persistent data stores
Enforce: Restricted ssh access
Enforce: Common service metadata& cost allocation
Encourage:Resource isolation
Where to favor rules & consistency?
Enforce: Common logging & monitoring infra
Encourage: Structured log data
Enforce: Standard upgrade flow
Enforce: Limited set of persistent data stores
Enforce: Statelessness & disposable compute nodes
Enforce: Restricted ssh access
Encourage:Resource isolation
Enforce: Common service metadata& cost allocation
12 Factor(not pets)
Nodes as cattle (not pets)
Where to bend the rules / favor flexibility?
Enable: Service-level technology stack flexibility
Servicesby
Runtime
Python 5%
Node.js 15%
JVM 25%
Docker 55%
Where to bend the rules / favor flexibility?
Enable: Service-level technology stack flexibility
Enable: Integration with 3rd party services
Where to bend the rules / favor flexibility?
Enable: Service-level technology stack flexibility
Adapt to underlying platform limitations
Enable: Integration with 3rd party services
Where to bend the rules / favor flexibility?
Enable: Service-level technology stack flexibility
Adapt: Education& on-boarding
Adapt to underlying platform limitations
Enable: Integration with 3rd party services
Where to bend the rules / favor flexibility?
Enable: Service-level technology stack flexibility
Adapt: Education & on-boarding
Adapt to underlying platform limitations
Enable: Integration with 3rd party services
Open the platform for contributions
Contributions&
Self-hosted PaaS components
BootstrappingService
Deploymentorchestration
service
ChaosMonkey
Log AnalysisService
PaaS
PaaS
PaaS
PaaS
PaaS
Simple Workflow
integration
Service cost estimation
Data PipelinesIntegration
Service-to-service auth
…
Where to bend the rules / favor flexibility?
Enable: Service-level technology stack flexibility
Adapt to underlying platform limitations
Enable: Integration with 3rd party services
Adapt: roadmap to inbound requests & technology trends
Open the platform for contributions
Adapt: Education & on-boarding
Organisation wide: Disaster Recovery
Compliance
Infra improvements: Container Clusters
Serverless
Promises of Service Decomposition
Choice oftech stacks
Scalability Independent deployments
Resilience End-to-endownership
Your platform should enable them – with heart and balance.
PaaS
Healthy PaaS —> Happy Microservices!
Balance consistencywith flexibility
Encourage / enforce“12 factor” & “nodes as cattle”
Keep the barrier toentry low
Tips for a healthy, balanced PaaS & a happy microservice ecosystem.
ROBIN FERNANDES • MICROS TEAM LEAD • ATLASSIAN • @REWBS
Thank you!
Reuters. Chickens perch on the roof of a hennery to escape rising floodwaters after Typhoon Utor hit Maoming, Guangzhou province August 15, 2013. Typhoon Utor hit China's southern Guangdong and Guangxi provinces before easing to a tropical storm.
Image Credits
https://design.atlassian.com/
Courtesy of Scott Monday:- https://au.pinterest.com/pin/146085581635905884/- http://scottmonday.com/
Copyright: velvetocean / 123RF Stock Photo – http://www.123rf.com/profile_velvetocean
Copyright: feedough / 123RF Stock Photo – http://www.123rf.com/profile_feedough'
Copyright: iimages / 123RF Stock Photo – http://www.123rf.com/iimages_feedough
