Chinmay Naik Lead Software Engineer, Cloud Engineering

Anand Palanisamy Manager, Software Development, Cloud Engineering

(OpenStack Summit – Hong Kong – 2013)

LESSONS LEARNED – BUILDING PAYPAL CLOUD

Confidential and Proprietary2

137,000,000 Users.

$300,000 Payments processed by PayPal each minute.

193 markets / 26 currencies.

PayPal is the World’s Most Widely Used Digital Wallet.

ABOUT PAYPAL

Confidential and Proprietary3

Challenges we are trying to address

Why OpenStack has emerged as a problem solver ?

Getting Openstack ready for production primetime

Success stories

STRUCTURE OF THE PRESENTATION

Confidential and Proprietary4

WHAT ARE WE TRYING TO SOLVE ?

Confidential and Proprietary5

Seamless On-Demand Infrastructure Capacity

Do we really want those hundred tickets to deploy a service ?

Drive developer agility

Provide self-service tool for application life cycle mgmt

Provide a platform to enable faster innovation.

SOME OF OUR CHALLENGES

Confidential and Proprietary6

WHO WILL GET US THERE ?

Confidential and Proprietary7

Solves Infrastructure-as-a-Service

Its open source

No specific vendor lock-ins

Fast growing developer community

Open standards and api driven

Industry best practices, prevent reinventing the wheel

OPENSTACK IS THE WINNER

Confidential and Proprietary8

OPEN SOURCE CANNOT ALWAYS BE USED OFF THE SHELF

Confidential and Proprietary9

OUR TECHNOLOGY STACK

Operations PortalAsgard, Horizon, Ceilometer

PD Deployment Portal

Orchestration Engine Cloud Formation (Heat)

Nova, Cinder, Swift, Keystone, Neutron, Horizon

Cobbler

ISC DHCPSalt Bind RHEL 6.x Hypervisor Zabbix

x86 Compute Local Storage NetworkLoad Balancer

User Interface

Orchestration

Foundational Services

Software Infrastructure

Hardware Infrastructure

Traffic Mgmt Monitoring Metering Stages Workflow Monitoring

LBaaS, DNSaaS FWaaS

PP Specific

Confidential and Proprietary10

Scheduling enhancements for failure and availability domains

Custom PayPal filter scheduler

Tenant based Compute Zone filters with Folsom

Host Aggregate filtering in Grizzly

25% distribution among different fault zone for HA

TUNING NOVA FOR HIGH AVAILABILITY

Confidential and Proprietary11

Instance host naming uniqueness

Auto assigning floating IPs to VMs

Rack aware networking

Leveraging config-drive

Nova conductor - security vs. load on rabbit

NOVA CHANGES

Confidential and Proprietary12

Integrating keystone with LDAP

Auto tenancy feature

Tenant based hostnames & dns zones

Client side token caching

Team admin feature

KEYSTONE CHANGES

Confidential and Proprietary13

Automatic

Project based zones

Floating IPs

DNS-AS-A-SERVICE INTEGRATION

Confidential and Proprietary14

Registration and auto discovery

Rich tenant and operator facing apis

Propagating changes to multiple LBs

Change Management Integration

LOAD BALANCER-AS-A-SERVICE

Confidential and Proprietary15

OTHER SUCCESS STORIES

Confidential and Proprietary16

Ease of use

Adoption

Multi Version

Multi Region

USER EXPERIENCE

Confidential and Proprietary17

USER INTERFACE SCREEN SHOTS

Confidential and Proprietary18

USER INTERFACE SCREENSHOTS

Confidential and Proprietary19

Devstack != Production

Keeping up with trunk

Single keystone service

Performance & Scalability

Error Handling

DEPLOYMENT PAIN POINTS

20 Confidential and Proprietary

cloud@paypal.com

Confidential and Proprietary21

http://www.123rf.com/

http://www.trashionista.com/

http://www.coreytowe.com/

http://www.birst.com/

COURTESIES FOR IMAGES USED

Confidential and Proprietary22

THANK YOU