Microservices on AWS Spot instances

Microservices on AWS Spot Instances

AWS Bay Area Meetup – Jan 13, 2016

Jim Bugwadia; [email protected]

High-performing IT organizations deploy 30X

more frequently with 200X

shorter lead times; they have 60X fewer failures and recover 168X faster.

2015 State of DevOps Report, Puppet Labs

Confidential 3January 14, 2016

Agenda

• AWS Spot Instances

• Microservices and Containers

• Nirmata

• Demo


About me

• Founder at Nirmata

• Software Developer (C++, Java, Javascript, Go)

• Large-scale distributed systems

AWS Spot Instances


AWS Buying Options

• On-demand Instances: hourly pricing

• Reserved Instances:• Up to 75% cheaper than on-demand pricing

• 1 – 3 year commitment

• Large upfront costs; typical breakeven at 50-80% utilization

• Spot Instances:• AWS sells excess capacity to higher bidder

• Hourly commitments at a price you name

• Can be up to 90% cheaper that on-demand pricing

• Dedicated Hosts: fully dedicated physical server


How Spot pricing works

• AWS determines market price based on supply and demand

• Instance is allocated to highest bidder

• Bidder pays market price

• Allocated instance is terminated (with a 2 minute warning) when market price increases above your bid price

• Diversification of instance families, instance types, availability zones increases continuity


c3.xlarge, 4 vCPU, 7.5 GiB; on demand price: $0.239 per Hour

Avg ~$0.03x80% of on-demand

Spot Bid Advisor: https://aws.amazon.com/ec2/spot/bid-advisor/

Spot History: https://us-west-1.console.aws.amazon.com/ec2sp/v1/spot/home?region=us-west-1


Spot Fleet

• Simplified bidding across instance types and regions

• Size based on number of instances, vCPU, or your own units

• Include multiple launch specifications, that vary by instance type, AMI, Availability Zone, or subnet

• Select bid strategy: Lowest price or Diversified

• API and Console!

https://console.aws.amazon.com/ec2sp/v1/spot/launch-wizard?region=us-west-1


AWS Spot Instances

Significant savings potential

Spot Instances are identical as On-Demand (no compromises)

But..

• Initial learning curve

• Not suitable for single point of failure applications

• Not suitable for applications coupled to infrastructure

• More complex to operate (without automation)

Microservices


An initial motivation for

Microservices was scalability

* The Art of Scalability; AKF Scale Cube


X-Axis Scaling

scale by replicating the entire application

Client Load Balancer Application

Application

Application


Y-Axis Scaling

scale by splitting the application into services

Client Load BalancerCustomers

Service

Catalog Service

Orders Service

/catalog

/customers

/orders


Z-Axis Scaling

Client Load BalancerCatalog

Service 3

Catalog Service 1

Catalog Service 3

/catalog [A – I]

/catalog [J – R]

/catalog [S – Z]

scale by splitting the data


Microservices provide Y-X axis scaling

scale by splitting the application and replicating services


Service

Catalog Service

Orders Service

/catalog

/customers

/orders


Microservices provide Y-X axis scaling

scale by splitting the application and replicating services


Service

Catalog Service

Orders Service

/catalog

/customers

/ordersBest scalability

Best availability

Best efficiency


1. Elastic: scales up or down independently of other services

2. Resilient: services provide fault isolation boundaries

3. Composable: uniform APIs for each service

4. Minimal: highly cohesive set of entities

5. Complete: loosely coupled with other services

19

A Microservices application is composed of several cooperating but independent services. Each Service is:

http://nirmata.com/2015/02/microservices-five-architectural-constraints/


Microservices Summary

Programming paradigm for the DevOps era

Builds on best practices of modular design and service-orientation

Enables agility at scale (organization, code, users)

But..

• Requires distributed programming, API foundation


• Requires new tooling

Application Containers


Containers are great for packaging and managing application components!

Immutable portable application images

Standard units of operations

Separation of Dev-Ops concerns

Efficient runtime

Efficient image management


Container Lifecycle

For a comprehensive CLI reference: http://docs.docker.com/reference/commandline/cli/


Containers

Immediate benefits for any application (portability and agility)

Easy to learn and adopt

Gateway to DevOps

Runtime effeciencies

But..

• Requires separation of config from code (12 factor, etc.)

• Requires new tooling


nirmata


DevOps

Public or Private Clouds

Multi-cloud Container Services

nirmata

any cloud – any device

Managed Devices

Data Centers

IT Ops


deploy operate optimize

Turnkey application container lifecycle management

Single control plane across public & private clouds

Highly scalable cloud-native service


Policy Based Orchestration

Real time Analytics

Application Blueprints

DevOps Integrations

Nirmata Control Plane

Host Agent

The Nirmata Solution

Service DB

Service Gateway

Data Plane:• Secure agents• Seamless Service Discovery• Distributed Load-balancing• Programmable routing

Host (or VM)

DNS / LB

Infrastructure

Container

Policy-based Orchestration

Microservices Infrastructure (optional)

Application Lifecycle Management

Resource Management & Scheduling

Host

Where Nirmata fits

You choose


Nirmata: Key Differentiators

1. Pure play application delivery & management

2. Non-intrusive and easy to use SaaS solution

3. Demand-driven Multi-Layer Scheduling

4. Split plane agent-based architecture

5. Integrated Microservices Infrastructure

6. Integrated monitoring & analytics


Demo Steps

1. Configure a AWS Spot Fleet Request

2. Configure a Nirmata Host Group that pools on-demand and spot instances

3. Configure scaling rules, placement rules, and constraints

4. Deploy an application across spot and on-demand instances

5. Terminate spot instances

6. Demonstrate service auto-recovery

7. Demonstrate host auto-scaling


Nirmata continuously monitors and optimizes application availability…

Nirmata Host Groups

Spot On-DemandSpot

1. Host Group Pooling

2. Minimum Placement Constraints

3. Spot Termination Detection

4. Host Auto-Scaling

5. Service Auto-Recovery

6. Dynamic Service Discovery & Load Balancing

Summary


Internet

Auto Scaling Groups

Edge Tier Mid-Tier Data Tier

Spot Fleet EBS Backed Instances

Auto-Scale Host Group

Spot Fleet Host Group

Launch ConfigHost Group


Key Takeways

1. AWS Spot Instances are significantly cheaper and are always available.

2. With the Microservices approach, each individual service is elastic and resilient.

3. Containers provide fast deployment times, improve utilization, and enable fully automated application operations.

4. Nirmata’s multi-cloud scheduling manages hosts and containers, provides policy-based orchestration, and maintains application availability across Spot and On-Demand instance pools!

Thank you!

https://try.nirmata.io

test drive Nirmata for free at:

Microservices on AWS Spot instances

Technology

Transcript of Microservices on AWS Spot instances