Building a Microservice Deployment Pipeline with CouchbaseHoward Uman

166,000+worldwide active users

1.4 millionevents managed

1,900+worldwide employees

$1 billion+market capitalization

HQ — Tysons Corner, VA

London, UK

New Delhi, India

Sydney, Australia

Austin, TX

Portland, OR

Atlanta, GA

Santa Barbara, CA

New Brunswick, Canada

Why Couchbase?

• Memcached compatibility

• Clients / Ease of use

• Ease of administration

• Scalability

• Indexing (Views & N1QL)

Where Couchbase fits in @ Cvent

What we have…

• 8 Couchbase clusters

• 215+ applications

• 100 environments

• 45 teams

Microservices

What is a Microservice?

• Modular

• Decentralized

• Scalable

• Well defined interface

Microservices@ Cvent

Ease of Development

Scalability

Testability

Connected

Microservices@ Cvent

Ease of Development

Scalability

Testability

Connected

Microservices@ Cvent

Ease of Development

Scalability

Testability

Connected

Microservices@ Cvent

Ease of Development

Scalability

Testability

Connected

Microservices@ Cvent

Ease of Development

Scalability

Testability

Connected

Microservices@ Cvent

Ease of Development

Scalability

Testability

Connected

Microservices@ Cvent

Ease of Development

Scalability

Testability

Connected

Microservices@ Cvent

Ease of Development

Scalability

Testability

Connected

Microservices@ Cvent

Ease of Development

Scalability

Testability

Connected

Microservices@ Cvent

Ease of Development

Scalability

Testability

Connected

Dropwizard

RetrofitRefit

Microservice Stack @ Cvent

Microservices at Cvent

• Java• Dropwizard• Dropwizard-Couchbase & other libraries• Retrofit / Refit

• Jenkins• Dropkick• View Management• N1QL Index Management• Build & Deploy

Best Practices

Microservice Best Practices

• Managed Schema

• Provide Clients

• Configuration

• Service Discovery

• Utility

• Enforcement

Bucket Best Practices

• 1 Thread Per Bucket

• Shared across applications

• Document key prefix: region, application, and typeex: Staging::Events::Wizard::{id}

• Chose bucket based on nature of data

Indexing Best Practices

• 1 Thread Per Design Document, Share.• Give your indexes Application-Entity-based names.• Indexes should leverage the document prefixes to reduce

data:function(doc, meta) {  var idArray = meta.id.split("::");  var applicationName = idArray[1];  var objectName = idArray[2];  if (applicationName === "YOUR_APPLICATION" && objectName === "DESIRED_OBJECT" ...) {    // Do your processing / emit }}

• Deploy indexes together to minimize scans.

• Deploy to DR cluster at the same time (production)

Configuration

Configuration

Configuration

Configuration

Configuration

Configuration

Deployment

Service Deployment Pipeline

Development

Service Deployment Pipeline

Development

CI

Service Deployment Pipeline

Development

CI Test

Service Deployment Pipeline

Development

CI TestAlpha

Service Deployment Pipeline

Development

CI TestAlpha

Test

Service Deployment Pipeline

Development

CI TestAlpha

TestStaging

Service Deployment Pipeline

Development

CI TestAlpha

TestStaging

Test

Service Deployment Pipeline

Development

CI TestAlpha

TestStaging

TestProduction

Service Deployment Pipeline

Development

CI TestAlpha

TestStaging

TestProduction

Building a ServiceDemo

Managing Services

Jenkins

• Backups

• Copy/Transform data

• Delete Documents

• Manage Views (Update / Delete)

• Manage GSI (Update / Delete)

Jenkins

Managing Services

Demo

Monitoring

Monitoring

• Service Discovery queries services (/ok)

• Datadog dashboard (next) for NOC

• Datadog alerts (Couchbase integration)

• Couchbase alerts

• Custom monitoring (Datadog Agent Checks)

Thank you!Howard Uman @unhumanhuman@cvent.com