Refactor Your Monolithic Rails App to a SOA

Refactoring Your Monolithic Rails App

To A SOA:

Risks and Rewards

Saturday, March 19, 2011

Chris WyckoffAlliance Health Networks

[email protected]@cwyckoff


mailto:[email protected]






The Birth Of A Monolithic Rails App:

A Simple Survey Engine


Pretty Simple App, huh?


“But we need to remarket all those emails.”


“We need more complex surveys.”


“Now we need an admin. And where’s our reporting?”


“Some clients need SOAP deliveries.”


And so on for 4 years.

Until...


“What’s wrong with the site guys?”


1. Slow

2. Buggy

3. New Features Take Forever To Implement

4. Deploys Are A Nightmare

5. Tons Of Dependencies


We Needed To Go From This...


DeliveryClient

BudgetCaps

BudgetSchool

Search

Lead

Referral

LeadProcessController

SearchResult

Campus

LeadStatus

CampusLocation

BudgetCaps

Contact

ContactDemographic

Program

DegreeLevel

StudyArea Company

Invitation

Enrollment

ReportsController

SchoolsController

BudgetsController

CampusLocationsController

ProgramsController

LeadsController

CompanyController

Monolithic RailsApp


To This...


LeadQualification

LeadDelivery

EmailRemarketing

RabbitMQ

LeadConversion

Reporting

BudgetService

AdminSurveyEngine

OperationalData Stores


LeadQualification

LeadDelivery

EmailRemarketing

RabbitMQ

LeadConversion

BudgetService

AdminSurveyEngine


Reporting


LeadQualification

LeadDelivery

EmailRemarketing

RabbitMQ

LeadConversion

Reporting

BudgetService

Admin


SurveyEngine


But How?


Good Code

Bad Code

The Strangler Approach


3 Drivers To Our ‘Strangler’ Approach:

1. Separate Responsibilities

2. Operate Asynchronously

3. Make Incremental Changes


1. Separate Responsibilities


Consolidate Like Functionality


DeliveryClient

BudgetCaps

BudgetSchool

Search

Lead

Referral


SearchResult

Campus

LeadStatus

CampusLocation

BudgetCaps

Contact

ContactDemographic

Program

DegreeLevel

StudyArea Company

Invitation

Enrollment

ReportsController

SchoolsController

BudgetsController


ProgramsController

LeadsController

CompanyController

Qualification Logic

Monolithic RailsApp


DeliveryClient

BudgetCaps

BudgetSchool

Search

Lead

Referral


SearchResult

Campus

LeadStatus

CampusLocation

BudgetCaps

Contact

ContactDemographic

Program

DegreeLevel

StudyArea Company

Invitation

Enrollment

ReportsController

SchoolsController

BudgetsController


ProgramsController

LeadsController

CompanyController

Monolithic RailsApp

Qualification Logic


DeliveryClient

BudgetCaps

BudgetSchool

Search

Lead

Referral


SearchResult

Campus

LeadStatus

CampusLocation

BudgetCaps

Contact

ContactDemographic

Program

DegreeLevel

StudyArea Company

Invitation

Enrollment

ReportsController

SchoolsController

BudgetsController


ProgramsController

LeadsController

CompanyController

Qualification Logic

Monolithic RailsApp


DeliveryClient

BudgetCaps

BudgetSchool

Search

Lead

Referral


SearchResult

Campus

LeadStatus

CampusLocation

BudgetCaps

Contact

ContactDemographic

Program

DegreeLevel

StudyArea Company

Invitation

Enrollment

ReportsController

SchoolsController

BudgetsController


ProgramsController

LeadsController

CompanyController

Delivery LogicMonolithic Rails

App


Legacy Tests As Behavior Scaffolding


describe Contact do describe "#validate" do it "requires 10 digit phone numbers" do ... end

it "rejects blank phone numbers" do ... end

it "rejects areacodes that start with 0 or 1" do ... end

it "requires 5 digit zip" do ... end

it "rejects invalidly formatted emails" do ... end

it "rejects blank emails" do ... end

it "rejects emails with bad words" do ... end ...end









Phone









Zip









Email


describe Phone do describe "#is_satisfied_by?" do

it "returns false if the number is not 10 digits long" do ... end

it "returns false if number matches invalid number list" do ... end

it "returns true if area code is valid" do ... end ... endend

describe Zip do describe "#is_satisfied_by" do

it "returns false if zip is not 5 digits" do ... end

it "returns false if zip does not exist in postal_codes database table" do ... end

... end end


scrubber = Scrubber.configure do |s| s.scrub(:first_name).as(:invalid).if(:blank) s.scrub(:last_name).as(:invalid).if(:blank) s.scrub(:email).as(:invalid).if(:blank) s.scrub(:phone).as(:invalid).if(:blank)

s.scrub(:first_name).as(:invalid).unless(:kid_friendly) s.scrub(:last_name).as(:invalid).unless(:kid_friendly) s.scrub(:email).as(:invalid).unless(:kid_friendly)

s.scrub(:country).as(:filtered).if {|lead| lead.contact.country == 'Other'}.and_warn("Only requests from the United States can be accepted at this time")

s.scrub_with(:email).as(:filtered) s.scrub_with(:education_level).as(:filtered) s.scrub_with(:duplicate).as(:filtered)end

scrubber.process({ :first_name => "Chris", :last_name => "Wyckoff", :email => "[email protected]", :phone => "8011231234", :country => "US" })


Lather, Rinse, Repeat


LeadQualification

Monolithic RailsApp


LeadDelivery

LeadQualification

Monolithic RailsApp


Cohesion


Scrubbing Engine

Zip Phone EmailParameterized

Rule

Scrubbing DSLScrubber

Conditions Operands

Factory

Qualification Service


Delivery Service

Lead FormatterField Mapper

Get/Post Email FTP/SFTP CustomPhone Date Case Truncate

Lead Deliverer

Get/Post Email FTP/SFTP Custom

Response Handler


Adhesion


DeliveryClient

BudgetCaps

BudgetSchool

Search

Lead

Referral


SearchResult

Campus

LeadStatus

CampusLocation

BudgetCaps

Contact

ContactDemographic

Program

DegreeLevel

StudyArea Company

Invitation

Enrollment

ReportsController

SchoolsController

BudgetsController


ProgramsController

LeadsController

CompanyController

Monolithic RailsApp


2. Operate Asynchronously


Client 1Monolithic Rails

App


Client 1

Client 2

Monolithic RailsApp


Client 1

Client 2

Client 3

Monolithic RailsApp


LeadDelivery

Client 1

Client 2

Client 3

Monolithic RailsApp


Client 1

Client 2

Client 3

LeadDelivery

LeadDelivery

LeadDelivery

Monolithic RailsApp


Client 1

Client 2

Client 3

200 ms

LeadDelivery

LeadDelivery

LeadDelivery

Monolithic RailsApp


Client 1

Client 2

Client 3

200 ms

LeadDelivery

LeadDelivery

LeadDelivery

4000 ms

Monolithic RailsApp


Client 1

Client 2

Client 3

200 ms

LeadDelivery

LeadDelivery

LeadDelivery

4000 ms

10000 ms

Monolithic RailsApp


3. Make Incremental Changes


Lead Delivery

Client 1

Client 2

Client 3

Monolithic RailsApp


Router

LeadDelivery

Client 1

Client 1

Client 2

Client 3

Monolithic RailsApp


Monolithic RailsApp

LeadDelivery

Client 1

Client 1

Client 2

Client 3

Router


class DeliveryRouter

def self.route(lead) if(publishable_to_new_delivery?(self)) DeliveryRouter.publish(lead) else lead.submit end end

def self.publish(lead) Bunny.publish(:lead_delivery, DeliveryMapper.map(lead)) end def self.publishable_to_new_delivery?(lead) lead.school.active_for_new_delivery? end end


class DeliveryMapper

def self.map(lead) { :first_name => lead.contact.first_name, :last_name => lead.contact.last_name, :address => lead.contact.address, :city => lead.contact.city, :state => lead.contact.state, :zip => lead.contact.zip, :phone => lead.contact.phone, :email => lead.contact.email, :client_id => lead.client.id, :lead_id => lead.id, ... } end

end


class DeliveryRouter

def self.route(lead) if(publishable_to_new_delivery?(self)) DeliveryRouter.publish(lead) else lead.submit end end

def self.publish(lead) Bunny.publish(:lead_delivery, DeliveryMapper.map(lead)) end def self.publishable_to_new_delivery?(lead) lead.school.active_for_new_delivery? end end


Router

LeadDelivery

Client 1

Client 1

Client 2

Client 3

Monolithic RailsApp


LeadDelivery

Client 1

Client 2

Client 3

Legacy Delivery Code

Monolithic RailsApp


What Did We Gain?


Speed


Maintainability


Testability


Composability


SourceLead

QualificationLead

Delivery


SourceLead

DeliveryLead

QualificationLead

Assignment


SourceLead

QualificationLead

DeliveryCall

Center

Call Center

LeadDelivery Client


LeadQualification

LeadDelivery

LeadDelivery

LeadAssignment

LeadDelivery

Call Center

LeadDelivery Client


Education

LeadDelivery

LeadQualification

Employment

Loan Financing


LeadQualification

LeadDelivery

EmailRemarketing

RabbitMQ

LeadConversion

Reporting

BudgetServiceAdmin


{'event':'delivery','status':'accepted','lead_id':'1234','timestamp':'2011-02-23 08:09:01','lead_details':{

...}

}


LeadQualification

LeadDelivery

EmailRemarketing

RabbitMQ

LeadConversion

Reporting

BudgetServiceAdmin


What Did We Risk?


Maintainability


Testability


Combat over-decomposition by consolidating code w/o necessarily isolating it as a service

1) gems2) local CouchDB stores that sync w/ a master store

Overly Decomposed Services


LeadQualification

LeadDelivery

EmailRemarketing

FieldFormatting

Rules Engine

SurveyEngine


LeadQualification

LeadDelivery

EmailRemarketing

SurveyEngine


LeadDelivery


LeadQualification

LeadDelivery

EmailRemarketing

SurveyEngine


Map Your Architecture


1) http://www.vanderburg.org/Blog/Software/Development/cohesion.rdoc

2) http://www.martinfowler.com/blikiStranglerApplication.html

3) http://www.eaipatterns.com/docs/EDA.pdf


http://www.vanderburg.org/Blog/Software/Development/cohesion.rdoc




http://www.martinfowler.com/blikiStranglerApplication.html

http://www.martinfowler.com/blikiStranglerApplication.html

http://www.eaipatterns.com/docs/EDA.pdf

http://www.eaipatterns.com/docs/EDA.pdf

Thank You.

Questions?


Refactor Your Monolithic Rails App to a SOA

Technology

Transcript of Refactor Your Monolithic Rails App to a SOA