Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications...

51
Tenant-based resource allocation model for cost-effective scaling Software-as-a- Service applications over cloud computing infrastructure Dissertation Defense Ph. D. in Information Technology and Communications Program Javier Mijail Espadas Pech Asesor: Dr. Arturo Molina

description

Computing resources are being transformed into a model consisting of services that are delivered in a similar way to traditional utilities such as water or electricity. One of the computing paradigms that have promised to deliver this utility computing vision is known as Cloud Computing. Cloud computing provides on-demand access to computational resources which together with pay-per-use business models, enables application providers seamlessly scaling their services. With cloud computing definition comes the term of elasticity which is the ability to create a variable number of virtual machine instances depending on the applications demands. Virtualization technology is widely adopted as an enabler of cloud computing because it provides benefits such as security, performance isolation, ease of management and flexibility of running within a user-customized environment. In the other hand, the cloud applications themselves have long been known to as Software as a Service (SaaS). SaaS is a software delivery paradigm where the software is hosted off-premises, developed by service providers and delivered via Internet and the payment mode follows a subscription model. For SaaS providers, having the power to scale up or down an application to only consume and pay for the resources that are required at certain point in time is an attractive capability and if done correctly it will be less expensive than running on traditional hosting. However, cost-effective scalability is not achieved just by deploying large-scale applications over pay-per-use cloud infrastructures, and idle processes and not used resources are wasted but charged to application providers. Over and under provisioning of cloud resources are still unsolved issues. Even if peak loads can be successfully predicted, without an effective elasticity model, costly resources are wasted during nonpeak times (underutilization) or revenues from potential customers are lost after experiencing poor service (saturation). In this sense, SaaS applications give an opportunity to improve this scenario due their multi-tenancy nature, which is the ability to offer one single application instance for several clients/providers (tenants). Each tenant can interact with the application as if it were an unique user and cannot access or view the data of another tenant. Consequently, with the use of cloud computing approaches such as on-demand virtual machine creation, it is possible to efficiently create a mechanism for SaaS applications in order to allocate, consume and charge only the required cloud computing resources by each tenant. This doctoral dissertation establishes formal measurements for under and over provisioning of virtualized resources in cloud infrastructures, specifically for SaaS platforms deployments and it proposes a resource allocation model to deploy SaaS applications over cloud computing platforms by taking into account their multi-tenancy thus creating a cost-effective scalable environment.

Transcript of Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications...

Page 1: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

Tenant-based resource allocation model for cost-effective scaling Software-as-a-

Service applications over cloud computing infrastructure

Dissertation DefensePh. D. in Information Technology and Communications Program

Javier Mijail Espadas Pech

Asesor: Dr. Arturo Molina

Page 2: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 2

Tecnológico de Monterrey – DTC - 2011

Agenda

• Concepts: Cloud computing, SaaS, multi-tenancy

• Background

• Premise, hypothesis and objectives

• Research methodology

• Problem measurement

• Proposed solution

• Results analysis

• Conclusions, contributions and future work

• Thesis’ outcomes

Page 4: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 4

Tecnológico de Monterrey – DTC - 2011

Cloud Computing

• Computing resources are being transformed into a model consisting of services that are commoditized and delivered in a similar way to traditional utilities such as water, electricity or gas.

• One of the computing paradigms that have promised to deliver this utility computing vision is known as Cloud Computing.

• Cloud Computing provides on-demand access to computational resources (commonly virtual machines) which together with pay-per-use business models, enables application providers seamlessly scaling their services.

Page 5: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 5

Tecnológico de Monterrey – DTC - 2011

Cloud computing layers

Applications

(Software as a Service)

Abstract Services

(Platform as a Service)

Flexible infrastructure

(Infrastructure as a Service)

Service Service ServiceInfrastructure

Interface

Development

DeploymentData Storage

Virtualization

OS

Hardware

Cloud Actors

Service Users Service Providers

Cloud Layers

Salesforce.com

Microsoft Live OfficeCisco Webex

Google Apps

Cloud Services

Amazon S3

Microsoft AzureGoogle SDK

Amazon Elastic Cloud

MassoHosting Cloud

Virtualization

OS

Hardware

Virtualization

OS

Hardware

Page 6: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 6

Tecnológico de Monterrey – DTC - 2011

Software-as-a-Service

• SaaS is a software delivery paradigm where the software is hosted off-premise, developed by service providers and delivered via Internet and the payment mode follows a subscription model.

• IDC estimates customers spending on SaaS solutions to increase to $14.8 billion by 2011.

• In order to achieve higher profit margin by leveraging the economics of scale, an important requirement for SaaS applications is the support of multiple tenants.

Page 7: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 7

Tecnológico de Monterrey – DTC - 2011

SaaS multi-tenancy• A tenant is a customer that uses or provides a SaaS

application.

• SaaS provider offers the same instance of a SaaS application to multiple tenants.

• Multi-tenancy means that each tenant can interact with the application as if it were the unique owner of the application.

• In particular a tenant cannot access or view the data of another tenant.

Page 8: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 8

Tecnológico de Monterrey – DTC - 2011

Background: Large-scale• Applications deployed on the Internet (i.e. SaaS) are immediately

accessible to a vast population of potential users.

• They tend to experience fluctuating and unpredictable load, especially due to events such as breaking news (e.g., 9/11) and sudden popularity spikes (e.g., the "Slashdot Effect").

• Facebook dynamically generates more than two billion pages per day resulting in over 23,000 page views a second .

• Image-processing Animoto web application experimented viral growth that caused to go from ten to more than 3500 servers in only three days.

Wor

ker

serv

ers

on A

maz

on E

C2

Page 9: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 9

Tecnológico de Monterrey – DTC - 2011

Background: Resources

• For Internet software providers, having the power to scale up or down an application to only consume and pay for the resources that are required at certain point in time is an attractive capability and if done correctly it will be less expensive than running on traditional hosting.

• However, cost-effective scalability is not achieved just by deploying large-scale applications over pay-per-use cloud infrastructures, and idle processes and not used resources are wasted but charged to application providers.

Page 10: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 10

Tecnológico de Monterrey – DTC - 2011

Background: Problem

• Real elasticity in pay-per-use cloud models is not achieved yet.

• Over provisioning effect happens due the resource underutilization: even if peak loads are successfully anticipated, resources are wasted (light gray areas).

• Under provisioning occurs when potential revenue from customers (dark gray areas) is lost due poor performance (saturation).

Pay-per-use resourcesReal demandExpected demand

Under provisioning (saturation)

Over provisioning (underutilization)

Page 11: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 11

Tecnológico de Monterrey – DTC - 2011

Premise

• SaaS applications give an opportunity to tackle the former problem due their multi-tenancy architecture.

• Due this multi-tenant nature of SaaS applications it is possible to create on-demand resources and distribute workload for each tenant within pay-per-use cloud infrastructures.

• SaaS platforms can take advantage of tenant-aware parallelism and work distribution by using cloud computing platforms.

Page 12: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 12

Tecnológico de Monterrey – DTC - 2011

Hypothesis

• Is possible to reduce over and underutilization of virtualized resources in order to obtain cost-effective scalability for Software-as-a-Service applications within a cloud computing pay-per-use infrastructure by implementing tenant-based resource allocation mechanisms such as isolation, virtual machine creation and load balancing.

Page 13: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 13

Tecnológico de Monterrey – DTC - 2011

Objectives

• To design a resource allocation model for scaling SaaS platforms and their applications over cloud computing platforms in order to create cost-effective scalability. – Analyze SaaS and cloud architectures.

– Define workload mechanisms.

– Develop monitoring mechanisms.

– Define mechanisms for tenant-level isolation.

– Create algorithms for virtual machine allocation.

– Design a load balancer.

– Gather, consolidate and analyze results.

Page 14: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 14

Tecnológico de Monterrey – DTC - 2011

Research Methodology

• Action-Research (AR) to implement actions and learning cycles from previous results.

• Two AR cycles, starting diagnosing the initial problem and finishing with the measuring of such problem within a test bed infrastructure for the first cycle.

• Second cycle starts by designing the solution (resource allocation model) and it outcomes an evaluation of this solution against former results through a statistical analysis.

Page 15: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 15

Tecnológico de Monterrey – DTC - 2011

Research MethodologyDiagnosing

Action Planning

Action Taking

Evaluation

Action Planning

Action Taking

Evaluation Learning

Background (Chapter 1)Problem Statement (Chapter 1)Literature Review & State-of-art (Chapter 3)

Over and Underutilization Measurements (Chapter 7)

Cloud Resources: Concept & Measurement (Chapter 4)

Proposed Model (Chapter 8) SaaS Platform: Design and

Implementation (Chapter 5)Test Bed Architecture (Chapter 6)

Tenant-based Isolation (Chapter 8)Tenant-based VM Allocation (Chapter 9)Tenant-based Load Balancing (Chapter 10)

Result Analysis(Chapter 11)

Conclusions and Future Work(Chapter 12)

Phase 2Phase 1

Page 16: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 16

Tecnológico de Monterrey – DTC - 2011

Literature ReviewResearch area Section Review

Cloud Computing 3.2 Cloud computing Concepts about cloud computing, its layers and definitions.

Software-as-a-Service 3.3 Software-as-a-Service Software-as-a-Service (SaaS), its impact, characteristics and

implementations.

SaaS platforms for test bed 3.4 SaaS Multi-tenancy How multi-tenancy is implemented currently and the existent

tenant-based patterns.

Cloud platforms for test bed 3.5 Eucalyptus Cloud Platform Eucalyptus platform is analyzed and detailed.

Virtualization for cloud computing 3.6 Virtualization

3.7 Virtualization Models for Cloud

Computing

Will be detailed how virtualization models fits within a cloud

environment.

Monitoring and measuring in cloud

computing

3.8.1 Cloud resource monitoring and

measuring

Measuring the problem is an important step in the research

methodology, therefore is analyzed related work to monitoring

cloud environments.

Isolation models for cloud computing 3.8.2 Toolkit for SaaS granularity and

isolation

SaaS isolation establishes the base mechanism to separate

execution from tenants. This section will present a related

work on this problem.

Virtual machine allocation for cloud

computing

3.8.3 Virtual Machine Assignment Relevant research and work is the allocation of virtual

machines as these are basic units for pay-per-use resource.

Balancing and scaling 3.8.4 Profile-based just-in-time scalability

3.8.5 Auto Scaling from Amazon Web

Services

Finally, it is presented in Section 3.7 related work for just-in-

time scalability and profile-based workload balancing.

Page 17: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 17

Tecnológico de Monterrey – DTC - 2011

Problem measurement

• How cloud computing resources are defined and measured?

• What is under and over resource provisioning?

• How they can be gathered and measured in virtualized environments such as cloud computing?

• Diagnosing: is important to measure resources within virtual machines (CPU, Java heap memory).

• Action planning: to deploy traditional scaling within cloud computing

Page 18: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 18

Tecnológico de Monterrey – DTC - 2011

Traditional scaling: Load balancing

• The higher level of granularity for cloud computing resources is the virtual machine.

• Each virtual machine or cloud instance has its own capabilities (CPU, memory, storage, etc.).

• When running parallel servers such as Apache Tomcat, a Web Server is required on top to distribute the workload over the underlying server instances.

Page 19: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 19

Tecnológico de Monterrey – DTC - 2011

Traditional scaling in cloud computing

• For the application server instances (Tomcat) is recommended to use large VM instances.

VMTomcat

Database

Cloud Platform (IaaS)

VMTomcat

VMTomcat

VMTomcat

VM = Virtual Machine

SaaS Platform

Page 20: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 20

Tecnológico de Monterrey – DTC - 2011

Distributed metering

• JConsole accesses several core monitoring and management functionalities provided by the Java platform.

• Java Management Extension (JMX) Remote framework connects to remote Tomcat processes and gather all related resource consumption information about them.

• SIGAR (stands for System Information Gatherer And Reporter) API provides a portable interface for gathering system information.

Page 21: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 21

Tecnológico de Monterrey – DTC - 2011

Distributed monitoring

• Resource Consumption State (RCS) is composed by the metering of CPU and heap memory in different time periods, not only for individual application servers, instead the measuring represents the status of the whole deployed cluster.

VMTomcat

VMTomcat

VMTomcat

VMTomcat

VMTomcat

RCSMonitor

Page 22: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 22

Tecnológico de Monterrey – DTC - 2011

Overutilization (saturation)

• For overutilization definition is used the term of point of exhaustion.

• Point of exhaustion for cloud computing can be defined by the maximum useful work (throughput) that can be executed on a single virtual machine.

• When resource utilization skyrockets, throughput generally declines.

• throughput = (number of requests) / (total time)

M1 M2 M3

Resource utilization

Throughput

T1

R1

R2

T2

T3

R3

Page 23: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 23

Tecnológico de Monterrey – DTC - 2011

Underutilization (resource wasting)

• The quantity of underutilized resources for a set of VMs is the number of virtual machines that can be released by allocating their resources within another VM.

• Each VM is evaluated against the rest at certain point time.

• In this dissertation, this number is obtained through a knapsack (bin packing) approach.

• The knapsack function returns the maximum number of VMs that can be released because it maximizes resource availability and gives low weight to VMs with low usage.

40 % used

Total VM1 memory

Threshold or maximum allowed (70%)

10 % used

Total VM2 memory

20 % used

Total VM3 memory

65 % used

Total VM4 memory

Page 24: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 24

Tecnológico de Monterrey – DTC - 2011

SaaS platform• Diagnosing: is important to measure resources within virtual machines (CPU, Java heap

memory).

• Action planning: to develop and deploy a SaaS platform with traditional scaling within cloud computing

• (1) multitenant - supporting large number of tenants/subscribers.

• (2) single version - single version per application and this one is available and shared among all the application users.

• (3) logical data separation - with just one data domain for each tenant but all the data is stored in a single database.

• SaaS platform enables a simple development, deployment and management of software applications delivered as services.

SaaS PlatformSubscription Service

Metering Service

Communications Service(Mail, SMS)

Integrations Service(Google Apps)

SaaSApplication

SaaSApplication

SaaS SDK

SaaS TagLibsJava Libraries

XML Configuration Files

Multi-tentant SaaS DB

SaaS Core W

eb A

pp

SaaSApplication

Dep

loymen

t M

anager

Infrastructure Management

SOA

Auth, Log, Account

Page 25: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 25

Tecnológico de Monterrey – DTC - 2011

Test bed architecture

itesm-cloud

Da

tab

ase

s

Apache HTTP

VM

SaaS platform(Tomcat instance)

VM VM

Workload Generation

JMeter machines

Requests

Resource Consumption State (RCS) Monitor

SaaS platform

(Tomcat instance)

VM

SaaS platform(Tomcat instance)

VM

itesm-cloud platform

Databases

Apache HTTP

VM

SaaS platform(Tomcat instance)

VM

VM

…SaaS platform

(Tomcat instance)

VMSaaS platform

(Tomcat instance)

VMSaaS platform

(Tomcat instance)

VM

VM = Virtual Machine instance

Page 26: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 26

Tecnológico de Monterrey – DTC - 2011

Generating workload

• Apache JMeter tool was selected to create heavy workload to the Tomcat cluster installed in a cloud environment and where the SaaS platform is deployed.

JMeter

Site / Server

JMeter

JMeter

JMeter

Network

Host 1

Host 2

1

2

Page 27: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 27

Tecnológico de Monterrey – DTC - 2011

Generating workload• Incremental. For each time period, workload starts from

the peak of the previous time period and increases until to reach the maximum peak of established users at the end of current period (solid line).

• Peak load. For each time period, workload starts from zero users and increments until to reach the maximum peak of users at the middle of the period. Then, the workload starts to decrease until zero at the end of the period (dotted line).

0

200

400

600

800

1000

1200

1400

1600

1800

1 2 3 4 5 6 7 8 9 10 11 12

Concurrent Users per Month

Concurrent Users

Each Tomcat server (VM) can handle 100 users as top

Page 28: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 28

Tecnológico de Monterrey – DTC - 2011

Problem measurement: Results

• % UU (Underutilization) = ( Combined UU / Measurements per hour) / Server-hours

• % OU (Overutilization) = Combined OU / (Measurements per month * Number of virtual machines)

Page 29: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 29

Tecnológico de Monterrey – DTC - 2011

Resource allocation model• Tenant-based isolation: creates tenant-level granularity and

separates execution contexts for different tenants

• Tenant-based VM allocation: implements mechanisms to calculate the number of VM instances needed, given a set of tenants.

• Tenant-based load balancing: implements a distribution mechanism to process and dispatch workload requests concerning to each tenant.

Cloud Communication Layer

VM Factory VM Status Handler

SaaS Core Web App

Tenant-based Load Balancer

Tenant Context Manager

Requests

Tenant-Based VM Calculator

Datab

ases

SaaS platform

(Tomcat instance)

VMVM

SaaS platform

(Tomcat instance)

VM

SaaS platform

(Tomcat instance)

VM

itesm-cloud

Eucalyptus API

Tenant Context Interceptor

Page 30: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 30

Tecnológico de Monterrey – DTC - 2011

Tenant-based isolation

• Each user belongs to one or more subscriber or tenant. Once the user has been authenticated through an email and password, SaaS platform links the user to its subscriber ID.

• The tenant-based information is stored in the context session and the rest of the applications can access it through the aspect-oriented mechanisms.

Saa

SC

ore

Web

Ap

p

Look for

subscriber

Internet

Get subscriber

(tenant) ID

Set session

cookieUser

User

User

User

Page 31: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 31

Tecnológico de Monterrey – DTC - 2011

Tenant-based isolation

• Excepting the login request, each subsequent request done to the platform has an associated subscriber / tenant.

• This isolation mechanism has to be dependent of each request done and it represents the portion of the resources used by each subscriber.

Ten

ant

Co

nte

xt

CPU, Memory, Storage CPU, Memory, Storage

Persistence Layer# loaded objects

User.subscriberId = 101

Subscriber 101# logged users

# apps used

Subscriber 102# logged users

# apps used

App Service 1

App Service 2

App Service 3

User.subscriberId = 101 User.subscriberId = 101

User.subscriberId = 102

User.subscriberId = 102

# database connections # database connections

Persistence Layer# loaded objects

Page 32: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 32

Tecnológico de Monterrey – DTC - 2011

Tenant-based isolation

// TenantContextInterceptor.javapublic class TenantContextInterceptor extends AbstractInterceptor implements

StrutsStatics {...public String intercept(ActionInvocation invocation) throws Exception {

ActionContext context = invocation.getInvocationContext ();HttpServletRequest request = (HttpServletRequest)

context.get(HTTP_REQUEST);...UserVO user = SidelAuth.getAuthenticatedUser(request);

String username = user.getEmail();long subscriberId = user.getSubscriberId();

TenantContextManager tenantContextManager = TenantContextManager.getInstance();

tenantContextManager.updateTenantContext(subscriberId,username);

...return invocation.invoke ();

}

class Class Model

TenantContextInterceptor

+ intercept(ActionInvocation) : String

TenantContext

- activeUsers: int

- tenantId: long

- users: List<String>

TenantContextManager

- contexts: Map<Long, TenantContext>

+ getInstance() : TenantContextManager

+ updateTenantContext(long, String) : void

updates

uses

• TenantContextInterceptor class implements an AbstractInterceptorinterface in order to catch the request invocation performed by the user.

• When the request is intercepted, it is retrieved the TenantContextManager instance which implements a Singleton design pattern to have only one single instance of such class.

• TenantContextManager holds a map where the TenantContextobjects are stored.

Page 33: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 33

Tecnológico de Monterrey – DTC - 2011

Tenant-based VM allocation

• Virtual Machine Assignment Problem (VMAP): find an assignment which distributes the load on the VMs such that all have access to adequate resources and the number of servers used is minimized.

• Thesis adaptation calculates the minimum number of virtual machine instances with specific and homogeneous capacity that need to be run when certain workload is received in the cluster.

Page 34: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 34

Tecnológico de Monterrey – DTC - 2011

Tenant-based VM allocation

• Tenant Context weight = Active users * ( heap size / maxThreads )

• VM capacity = Heap size - ( ( heap size / maxThreads) * platform threads)

Page 35: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 35

Tecnológico de Monterrey – DTC - 2011

Tenant-based VM allocationint VMs = 0;

public int calculateVMs(int capacity,int []tenantWeights){

VMs = preProcessWeights(capacity, tenantWeights); //STEP_1 Initial VMs for overweight tenants

tenantWeights = processWeightArray( capacity,tenantWeights ); //STEP_2 Eliminate overweight values

calculateVmsAllocation(capacity, tenantWeights); //STEP_3 Calculate VMs

return VMs;

}//end of function

private void calculateVmsAllocation(int capacity,int []tenantWeights){

if (tenantWeights.length==0) return;

else{

//EVERY ITERATION INCREMENTS VM INSTANCES NUMBER

VMs++;

//SOLVING KNAPSACK

Knapsack KS = new Knapsack(capacity, tenantWeights, tenantWeights); KS.search(0,0,0);

int [] take = KS.getBestSolution();

//FILTERING NOT-TAKEN ELEMENTS

List<Integer> newWeightList = new ArrayList<Integer>();

for (int c=0;c<take.length;c++){

if (take[c]==0){ newWeightList.add(tenantWeights[c]); } //not taken

}

//REDUCING ARRAY

tenantWeights = convertToIntArray(newWeightList);

//RECURSIVE TO NEXT ITERATION

calculateVmsAllocation(capacity,tenantWeights);

}

} //end of function

Page 36: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 36

Tecnológico de Monterrey – DTC - 2011

Cloud Communication Layer• The Cloud Communicator Layer contains the components for monitoring

and accessing the itesm-cloud private infrastructure through SOAP interfaces.

• It uses jClouds library which is a Java open source framework that implements portable abstractions to get connected to public clouds.

//initialize credentialsString accesskeyid = "...";String secretkey = "...";

//connect to the Eucalyptus itesm-cloudProperties overrides = new Properties();overrides.setProperty("eucalyptus.endpoint","http://itesm-cloud:8773/services/Eucalyptus");ComputeServiceContext context = new ComputeServiceContextFactory().createContext("eucalyptus", accesskeyid,secretkey, ImmutableSet.<Module> of(new Log4JLoggingModule(), new JschSshClientModule()), overrides);

//create a VM template with SaaS platform image and small VM typeTemplate template =context.getComputeService().templateBuilder().hardwareId("m1.small").imageId("Eucalyptus/emi-9ACB1363").build();

//set security paramstemplate.getOptions().as(EC2TemplateOptions.class).securityGroups("default");template.getOptions().as(EC2TemplateOptions.class).noKeyPair();

//creating N instancesSet<? extends NodeMetadata> nodes = context.getComputeService().runNodesWithTag("saasplatform", N, template);...//or shutdown an instance by IDcontext.getComputeService().destroyNode("Eucalyptus/i-3FB706CB");

Page 37: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 37

Tecnológico de Monterrey – DTC - 2011

Tenant-based load balancing

• Load balancing is an effective solution scheme which purpose is to make the clustered servers equalizing sharing the workload through mutual cooperation and work distribution.

• Tenant-based load balancing is proposed to take advantage of isolated parallelism in Web applications, meaning that requests from users are totally independent each other.

Page 38: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 38

Tecnológico de Monterrey – DTC - 2011

Tenant-based load balancing

• Tenant-based load balancer is part of the SaaS platform and it uses the Tenant Context Managercomponent to retrieve information about tenant statuses.

• The tenant-based load balancer tries to allocate requests of the same tenant to an unique virtual machine, depending of the VM capacity.

SaaS Core Web App

Tenant-based Load Balancer

Server Preparer

Cookie Manager

Response ParserRe

qu

est

Pro

cess

or

Ten

ant

Re

qu

est

Sc

he

du

ler

Tenant Context Manager

Workload

Requests

Da

tab

ase

s

SaaS platform

(Tomcat instance)

VM VM

SaaS platform(Tomcat instance)

VM

SaaS platform

(Tomcat instance)

VM

Cloud Communication Layer

itesm-cloud

Page 39: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 39

Tecnológico de Monterrey – DTC - 2011

Tenant-based load balancing

• Request Processor receives requests from HTTP or HTTPS ports and interprets the request method (GET, POST, etc).

• Server Preparer instantiates a new HTTP client, set the request headers and establishes timeouts.

• Cookie Manager copies cookie information into the request.

• Response Parser returns the server answer meta data to the client.

• Tenant Request Scheduler communicates with Tenant Context Manager component in order to determine to which VM dispatch the request.

Page 40: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 40

Tecnológico de Monterrey – DTC - 2011

Tenant-based load balancingprivate void readTenantInformation(long tenantID, Request schedRequest) {

String VM_IP_ADDRESS = DEFAULT_IP_ADDRESS;TenantRequestScheduler tenantRequestScheduler = TenantRequestScheduler.getInstance(); TenantContextManager tenantContextManager = TenantContextManager.getInstance();int activeUsers = 0; //initialize users

Map<Long, TenantContext> tenantContextsMap = tenantContextManager.getTenantContextMap();TenantContext tenantContext = tenantContextsMap.get(tenantID);VM_IP_ADDRESS = tenantRequestScheduler.calculateVMIPAddress(tenantContext);activeUsers = tenantContext.getActiveUsers();

TenantHostInfo tenantHostInfo = new TenantHostInfo();tenantHostInfo.setTenantId(tenantContext.getTenantId());tenantHostInfo.setActiveUsers(activeUsers+1);hostMap.put(VM_IP_ADDRESS, tenantHostInfo);

// UPDATING THE LOAD INDEXloadIndex+= tenantHostInfo.getActiveUsers();

prepareServerRequest(schedRequest, VM_IP_ADDRESS)

}

Page 41: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 41

Tecnológico de Monterrey – DTC - 2011

Solution measurements

Cloud Communication Layer

VM Factory VM Status Handler

SaaS Core Web App

Tenant-based Load Balancer

Tenant Context Manager

Requests

Tenant-Based VM Calculator

Data

bases

SaaS platform

(Tomcat instance)

VMVM

SaaS platform

(Tomcat instance)

VM

SaaS platform

(Tomcat instance)

VM

itesm-cloud

Eucalyptus API

Tenant Context Interceptor

After setting up the tenant-based components and deploying them over the test bed, all the simulations and tests were run again. Over and underutilization measurements where performed against workload tests (incremental and peak-based).

itesm-cloud

Da

tab

ase

s

Apache HTTP

VM

SaaS platform(Tomcat instance)

VM VM

Workload Generation

JMeter machines

Requests

Resource Consumption State (RCS) Monitor

SaaS platform

(Tomcat instance)

VM

SaaS platform(Tomcat instance)

VM

vs.

Page 42: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 42

Tecnológico de Monterrey – DTC - 2011

Result Analysis

vs.

Control group Experimental group

Page 43: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 43

Tecnológico de Monterrey – DTC - 2011

Result Analysis

• t-student test allows to determine if two averages are significantly different.

• (N1+N2-2) = (12 + 12 -2 ) = 22 degrees of freedom = 99.5 accuracy

• t-student distribution table produces a value of t =

2.8188 as base parameter.

• If t > t , we can say with a 99.5% of certainty that one average (in Table 11.2) is statistically less than another column (in Table 11.1).

1 2

2 2

1 2

1 2

X Xt

S S

N N

Page 44: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 44

Tecnológico de Monterrey – DTC - 2011

Result Analysis

Control groupExperimental

Group

t-student

value

>2.8188 t

parameter

Incremental UU 18.42% 8.21% 3.2437 Yes

Incremental OU 13.26% 13.25% 1.0282 No

Peak-based UU 29.12% 8.77% 4.7208 Yes

Peak-based OU 12.22% 9.9% 0.7485 No

• Averages for underutilization before tenant-based components have been statistically improved.

• On the other hand, both t-student values corresponding to overutilization (OU) are not higher than t, even if the averages were reduced.

Page 45: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 45

Tecnológico de Monterrey – DTC - 2011

Conclusions• It was designed and deployed a SaaS platform architecture and

applications that run within a cloud computing environment.

• It was defined the workload mechanisms and tools to generate request for the test beds.

• It was established a formal measure for under and over provisioning of virtualized resources (CPU and Heap memory) in cloud infrastructures.

• It was developed monitors for resource utilization (CPU and memory) within virtual machines.

• It was defined, designed and developed a mechanism for tenant-level isolation in order separate execution and gather information about tenant' resource usage.

• It was created the mechanisms and algorithms for virtual machine allocation based on tenant information and virtual machine capacity.

• It was designed and implemented a load balancer in order to distribute requests based on virtual machine statuses and tenant workload distribution.

Page 46: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 46

Tecnológico de Monterrey – DTC - 2011

Conclusions

• This doctoral thesis demonstrates that resource allocation models have important impact in how the virtualized resources are created and instantiated within cloud environments.

• The stated hypothesis has been validated, it was reduced the over and underutilization of virtualized resources in order to obtain cost-effective scalability for Software-as-a-Service applications within a cloud computing pay-per-use infrastructure by implementing tenant-based resource allocation mechanisms such as isolation, virtual machine creation and load balancing.

Page 47: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 47

Tecnológico de Monterrey – DTC - 2011

Main contributions

• Establishment of formal measurements for under and over provisioning of virtualized resources in cloud infrastructures.

• Presentation of an architecture based on open source components for developing, deploying and managing SaaS applications.

• This work provides a detailed deployment design about how to setup a cloud infrastructure over commodity facilities and the use of open technologies.

• This thesis contributes to the resource allocation topic specifically in cloud computing infrastructures, through new mechanisms for distribute workload in virtualized environments and dynamic load balancing proposal based on global statuses of tenant information.

Page 48: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 48

Tecnológico de Monterrey – DTC - 2011

Future Work

• Different software platforms

• Measuring other resources

• Database clustering

• Multiple database in SaaS applications

• Different workload generation

• SaaS architecture and cloud platform

• Analytical testing

• Performance improvement

Page 49: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 49

Tecnológico de Monterrey – DTC - 2011

Thesis outcomes

• 2 conference papers.

• 2 journal papers.

• Research internship at Microsoft Research with Cloud Computing Futures group. Redmond, WA. (2009).

• Business incubation cell about Software-as-a-Service platforms and applications. (2010).

• National conferences and collaborations.

Page 50: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 50

Tecnológico de Monterrey – DTC - 2011

Publications• Espadas, J.; Concha, D.; Molina, A. “Application Development over

Software-as-a-Service Platforms” (2008). The Third International Conference on Software Engineering Advances ICSEA 2008 - Sliema, Malta.

• Espadas, J.; Concha, D.; Romero, D.; Molina, A. (2010). "Open Architecture for Developing Multitenant Software-as-a-Service Applications“. The First International Conference on Cloud Computing, GRIDs, and Virtualization. Published by XPS. November 21-26, 2010 - Lisbon, Portugal.

• Concha, D.; Espadas, J.; Romero, D. and Molina, A. (2010). "The e-HUB Evolution: From a Custom Software Architecture to a Software-as-a-Service Implementation" in Journal Computers in Industry, Volume 61, Issue 2, Elsevier, ISSN: 0166-3615, Impact Factor 2.014.

• Javier Espadas, Arturo Molina, Guillermo Jiménez, Martín Molina, David Concha, Raúl Ramírez. “Tenant-based resource allocation model for scaling Software-as-a-Service applications over cloud computing infrastructures”. Journal of Future Generation Computer Systems, Special Issue on Clouds and Service Oriented Architectures. Impact Factor 2.229 (In review process with minor changes).

Page 51: Tenant-based resource allocation model for cost-effective scaling Software-as-a-Service applications over cloud computing infrastructure

6/9/2011 51

Tecnológico de Monterrey – DTC - 2011

The end

• Questions