Modern ArchitectureGiragadurai Vallirajan

CTO@Bluemeric @Girag

https://www.linkedin.com/in/Girag

Agenda• What is Cloud?

• Softwares in Cloud?

• Architecting for Cloud

• Application

• Micro Services

• Data Intensive

• Questions

Cloud

“Cloud computing enables companies to consume compute resources as a utility (like electricity) rather than having to build and

maintain computing infrastructures in-house.”

Three of the main benefits of cloud computing include:

• Self-service provisioning: End users can spin up computing resources for almost any type of workload on-demand.

• Elasticity: Companies can scale up as computing needs increase and then scale down again as demands decrease.

• Pay per use: Computing resources are measured at a granular level, allowing users to pay only for the resources and workloads they use.

Clo

ud

Clo

ud L

ayer

s

PAAS

SAAS

IAAS

Build On It

Migrateto It

Consume It

Software As Service

Platform As Service

Infra As Service

Compute

Network

Storage

Server

load balancer

Firewalls

IP

Application Server Web Server

Runtime Database Queue

Monitoring Games

Collaboration

Finance

Communication CRM

Architecting (SAAS)

Factors• Multi-tenancy

• Load Balancing

• Scaling

• Security

• Monitoring and Metering

• Pricing

Tenancy

Shared Nothing Shared Few Shared Everything

☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺

app app app app Application Application

Databasedb db db db db db db db

Load Balancing

Scaling

Scale cube

http://microservices.io/articles/scalecube.html

X-axis scaling consists of running multiple copies of an application behind a load balancer.

/report

/report

/report

Y-axis axis scaling splits the application into multiple, different services. Each service is responsible for one or more closely

related functions.

/login

/purchase

/report

/login

/purchase

/reportMix

ing

It…

Using Z-axis scaling each server runs an identical copy of the code, however handle subset of data

/purchase[A-K]

/purchase[L-Q]

/purchase[R-Z]

Products (A-K)

Products (L-Q)

Products (R-Z)

Monitoring and Metering Cloud application metrics for health tracking and also pricing

• CPU Utilization

• Network (Inbound / Outbound)

• DiskOps, Bytes (Read/Write)

• ServerThroughput (In/Out)

• Service API Usage Count

• Delays

Met

erin

g

Pric

ing

Pricing• Focusing solely on registering accounts with free trials increases

dependency and ultimately, conversions.

• Presenting 5 plans clearly enforces the level of seriousness, capability, and value increase by price plan, without doubting higher price points.

• Focusing on team size directly validates price differences.

• Pairing a free trial with necessary quotes allows potential customers to experience the service first-hand and validate higher-priced plan propositions.

• Transparency and customisation in pricing information allow potential customers to determine an appropriate plan and increase comfort level.

SAAS Application

PAAS

IAAS Metering EndPoints

Collector (by Profile)

Billing Engine

Metrics &

Metering Profile

Secu

rity

Reference : AWS Documentation on Security

SecurityTop 10 security issues to be addressed,

• SQL, operating system or LDAP injection

• Insecure authentication and session management

• Cross-site scripting because of lack of data validation

• Insecure exposure to references like files and directories

• Incorrectly configured (from a security perspective) databases, middleware and operating systems

• Exposing sensitive data like user IDs, passwords and personal identification information

• Checking for access inside the business logic on the server side

• Cross-site request forgery

• Using components with known vulnerabilities

• Unvalidated redirects and forwards

Dep

loym

ent

Assi

gnm

ent

Questionnaire App

We have to design an SAAS based application for helping customer who like to post a survey, poll, census to study their customer expectation, business reachability or quiz competitions.

Functionality,

1. An Account can have many survey and track them

2. Participant can be Anonymous (survey) or known (quiz)

3. Statistical analytics of individual survey

4. Every campaign stay active for duration

5. API Service for custom integration

6. Charges

Factors to be noted,

1. Scale, end user must get seamless feeling even the load high

2. Multi-Tenancy

3. Security

Application (microservices)

Micro Service Architecture is an architectural concept that aims to decouple a solution by decomposing

functionality into discrete services.

µ S

ervi

ces

Reference http://www.pwc.com/us/en/technology-forecast/2014/cloud-computing/features/microservices.jhtml

• the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API.

• These services are built around business capabilities and independently deployable by fully automated deployment machinery.

• There is a bare mininum of centralized management of these services, which may be written in different programming languages and use different data storage technologies.An

atom

y

Design

Monolithic Application Microservices Application

Deployment

Microservices Application

Message Bus

Reference http://www.pwc.com/us/en/technology-forecast/2014/cloud-computing/features/microservices.jhtml

Operating-system-level virtualization is a server-virtualization method where the kernel of an operating system allows for multiple isolated user-

space instances, instead of just one. Such instances, which are sometimes called containers, software containers, virtualization engines (VEs) or jails, may look and feel like a real server from the point of view of

its owners and users.

On Unix-like operating systems, one can see this technology as an advanced implementation of the standard chroot mechanism. In addition to isolation mechanisms, the kernel often provides resource-management

features to limit the impact of one container's activities on other containers.

~wikipedia

Con

tain

ers

Server

Host OS

Guest OS

Bin/Lib

APP

Bin/Lib

APP

Bin/Lib

APP APP APP

VM S

tack

Hypervisor

Guest OS Guest OS

Server / Hypervisor

Host OS

Container Engine(Docker, Rocket)

Bin/Lib

APP

Bin/Lib

APP

Bin/Lib

APP APP APP

Con

tain

er S

tack

Containers

Hypervisor

Bin/Lib

APP

Bin/Lib

APP

Bin/Lib

APP APP APP

Uni

-ker

nel S

tack

OS OS OS

Containers

Con

tain

er L

ayou

t

Layer 1 (Shared)

Layer 2 (Shared)

Layer 3 (A) Layer 3 (B) Layer 3 (C)

APP APP APP

• Lightweight resource utilisation: instead of virtualising an entire operating system, containers isolate at the process level and use the host’s kernel.

• Portability: all of the dependencies for a containerised application are bundled inside of the container, allowing it to run on any container engine host (Docker).

• Predictability: The host does not care about what is running inside of the container and the container does not care about which host it is running on. The interfaces are standardised and the interactions are predictable.

Adva

ntag

e

https://www.mindmeister.com/389671722/open-container-ecosystem-formerly-docker-ecosystem

Refer following link for latest up-to-date Ecosystem

https://msdn.microsoft.com/en-us/virtualization/windowscontainers/about/container_ecosystem

Mic

roso

ft

• Kubernetes• Fleet• Swarm• Mesos• Marathan

Sche

dule

r

• Google Container Engine• Amazon EC2• Azure Container System• Microsoft Service Fabrics

Clo

ud O

fferin

g

Assi

gnm

ent

REST API

Passenger Management Billing

Payment Trip Management

Notification

Driver Management

WEB UI

MySQL Adaptor

Payment Adaptor

SendGrid Adaptor

KooKoo Adaptor

Driver

Passenger

Application (Data Intensive)

Reference : Tiecon Bigdata Landscape

Reference : IBM Infographics

Architecture

Infrastructure

Big-data Framework

Build Cluster

Infrastructure

Big-data Framework

Ingress BatchIngest

EventIngest

Infrastructure

Big-data Framework

Ingress

Batch

Event

Staging StorageFormat

DataPartition

Access Control

Infrastructure

Big-data Framework

Ingress

Batch

Event

Staging

StorageFormat

DataPartition

AccessControl

Data Processing Data Transformation

Analytics

Infrastructure

Big-data Framework

Ingress

Batch

Event

Staging

StorageFormat

DataPartition

AccessControl

Data Processing

Data Transformation

Analytics

Orchestrate Manage

Workflow

Infrastructure

Big-data Framework

Ingress

Batch

Event

Staging

StorageFormat

DataPartition

AccessControl

Data Processing

Data Transformation

Analytics

Orchestrate

Management

Workflow

Infrastructure

Big-data Framework

Ingress

Batch

Event

Staging

StorageFormat

DataPartition

AccessControl

Data Processing

Data Transformation

Analytics

Orchestrate

Management

Workflow

Egress

RDBMS Data

JSON

File

Data Pipeline

Cloud / Virtual Machine / Network / Cluster

Hadoop Framework

Ingress Staging Data Processing Orchestrate Egress

Data Pipeline

FlumeSqoop

HDFSHTTPFS

AVRO

ETL

ACL

YARN

STROM

SPARK

MapReduce

Oozie

CDH Navigator

RDBMS Data

JSON

XML

Lambda Architecture

Speed Layer

Batch Layer Service LayerQuery

Query

Reference : Bigdata Lambda Architecture

Reference : http://www.slideshare.net/mjft01/big-data-landscape-matt-turck-may-2014

Assi

gnm

ent Recommendation System

Analysis of social media activity is one of the most important. Everyone a on social media these days, whether they're "liking" company pages on Facebook or tweeting complaints about products on Twitter. A Big Data solution built to harvest and analyze social media activity, Social media can provide real-time insights into how the market is responding to products and campaigns. With those insights, companies can adjust their pricing, promotion, and campaign placement on the fly for optimal results.

Functionality,

1. Recognise my taste like What Would customer buy in near feature (Personal)

2. Recognise the Market trend in a Particular city (Geo)

3. Feed from FB Pages, Twitter, Review sights

Factors to be noted,

1. Accuracy of result

Q&A