Ms.Shinde Rupali R Prof.Mr.Maral Vikas B. K.J.COEMR,Pune May …€¦ · Scalability in Cloud...

Scalability in Cloud computing by using VmWareVfabric SQLFire

Ms.Shinde Rupali R Prof.Mr.Maral Vikas [email protected]

Department of Computer EngineeringK.J.COEMR,Pune

May 8, 2013

Abstract

Cloud computing is a computing style inwhich scalable and flexible IT functional-ities are delivered as a service to exter-nal customers using Internet technologies.Cloud computing is not a revolutionaryidea; Instead, it is an evolutionary conceptthat integrates various existing technologiesto offer a useful new IT provisioning tool.As more and more organizations embracecloud computing to save money, increaseproductivity, and to gain the ability to scaletheir infrastructures up or down at a mo-ments notice, concerns remain around thelevel of performance, availability, and over-all security of their databases once theymove to a hosting vendor. Many of thefears organizations have with moving theirdatabases from a dedicated environment tothe cloud start with the overall comfortlevel of managing their data in a virtual en-vironment.However, advancements in bothMicrosoft SQL Server and cloud infrastruc-tures have changed all that. More specifi-

cally, combining SQL Server with a provenhosting service provider offers highly scal-able, flexible and cost-effective solutions.Scalability - Cloud computing enables uni-versities to quickly scale up their IT op-erations as provisioning of new computingresources and software applications can bedelivered at a desired pace. Furthermore,constraints on pre-purchasing of resourcesto meet peak requirement in traditional ITno longer exist.In this paper we are goingto use Providing Scalable SQL data in thecloud With VMware vFabric SQL Fire.

Keywords

Cloud computing, Scalabe SQL data,VMware vFabric SQL Fire,OLAP/OLTP.

1 Introduction

With the increasing popularity of Web 2.0applications, massive amounts of dierenttypes of data are being generated at an

1

International Journal of Scientific & Engineering Research, Volume 4, Issue 7, July-2013 SSN 2229-5518

1207

IJSER © 2013 http://www.ijser.org

IJSER

unprecedented scale[8]. Given this rate ofcontinuous growth, coupled with advance-ment in broadband connectivity, virtualiza-tion, and other technologies, the cloud com-puting model, with its capability to dynam-ically provide for computation and storage,has emerged as an ideal choice for data-intensive IT infrastructures. The need toprovide for capacity both in terms of stor-age and computation, and to support onlinetransactional processing and online analyti-cal processing in the cloud, has given rise tomajor challenges in architecting elastic andecient data servers. The web service ap-plications provided by Internet companiessuch as emailing, online shopping and socialnetworking, are all based on online transac-tions that are essentially similar to those intraditional OLTP (online transaction pro-cessing) systems. However, in such webapplications, system scalability, service re-sponse time and service availability demandhigher priority than transactional data con-sistency, which is the foremost requirementof traditional OLTP systems. To bettersupport search and data sharing, large-scalead-hoc analytical processing of data col-lected from those web services is becomingincreasingly valuable to improving the qual-ity and eciency of existing services, and sup-porting new functional features. Due to themassive size of web data, traditional OLAP(online analytical processing) solutions (i.e.,parallel database systems) fail to scale dy-namically with needs. Therefore, both com-mercial companies and open-source commu-nities have proposed new large-scale dataprocessing systems, such as Hadoop , Hive, Pig and Dryad . Historically, OLTP andOLAP workloads are handled separatelyby two systems with dierent architectures

RDBMS for OLTP and data warehous-ing system for OLAP. Periodically, datain RDBMS are extracted, transformed andloaded (aka. ETL) into the data warehouse.The system-level separation was motivatedby the facts that OLAP is computationallyexpensive and its execution on a separatesystem will not compete for resources withthe response-critical OLTP operations, andsnapshot-based results are generally su-cient for decision making. Although thissystem-level separation provides exibilityand eciency, it also results in several inher-ent limitations, for example, lack of datafreshness in OLAP, redundancy of datastorage. Several data mining languages pur-port to implement a cloud computing so-lution for certain SQL construc-tions, eachattempting to reduce the effort needed tolaunch a MapReduce job. Googles propri-etary software, both its distributed file sys-tem1 and its Bigtable relational overlay,2have open source equivalents in the Apache-supported Hadoop3 project. Both use theMapReduce algorithm, and both find writ-ing code for the parallel map and reducephases to be a tedious and error-prone task.Consequently, both evolved scripting lan-guages to capture query intent and produceMapReduce code automatically.

2 Related Work

VMware vFabric SQLFire is a memory-optimized distributed SQL database deliv-ering dynamic scalability and high perfor-mance for data-intensive modern applica-tions[1]. SQLFires memory-optimized ar-chitecture minimizes time spent waiting fordisk access, the main performance bottle-

2


1208


IJSER

neck in traditional databases. SQLFireachieves dramatic scaling by pooling mem-ory, CPU and network bandwidth across acluster of machines and can manage dataacross geographies. Built on the provenvFabric GemFire distributed data manage-ment platform, SQLFire is ideal for pri-mary data stores that require high transac-tion rates, continuous availability, and sup-port by database programming staff with-out specialized coding assistance. Deployedas the front-end datamanagement layer foran Oracle or other RDBMS, SQLFire de-livers substantially better performance andscalability for modest investments, and re-tains full compatibility with current DBAskills and tools.

3 What is Cloud com-

puting ?

Cloud computing is a significant advance-ment in the delivery of information tech-nology and services.[5] By providing on de-mand access to a shared pool of comput-ing resources in a self-service, dynamicallyscaled and metered manner, cloud comput-ing offers compelling advantages in speed,agility and efficiency. Today, cloud comput-ing is at an early stage in its lifecycle, butit is also the evolution and convergence ofseveral trends that have been driving enter-prise data centers and service providers overthe last several years. Cloud computingbuilds off a foundation of technologies suchas grid computing, which includes cluster-ing, server virtualization and dynamic pro-visioning, as well as SOA shared servicesand large-scale management automation.

For the better part of a decade, Oracle hasbeen the leader in these areas with thou-sands of customer successes and high levelof investment. Today, Oracle offers theindustrys most complete, open and inte-grated products and services to enable pub-lic, private and hybrid clouds. Oracle aimsto make cloud computing fully enterprise-grade and supports both public and privatecloud computing to give customers choice.Oracle offers technology that enables or-ganizations to build private clouds, lever-age public clouds and provide cloud ser-vices to others. Oracle also offers a broadset of horizontal and industry applicationsthat run in a shared services private cloudmodel as well as a public Software-as-a-Service (SaaS) cloud model. This white pa-per provides an overview of Oracles cloudcomputing strategy and how Oracle helpscustomers and partners plan their evolutionand adoption of a cloud computing model.

3.1 Cloud computing servicemodel

Cloud computing encompasses many as-pects of computing (from hardware to soft-ware) that a single solution is not able toprovide all aspects. Generally speaking,cloud computing applications incorporatethe combination of the following functionalservice models[5]:

3.1.1 Infrastructure as a Service(IaaS)

IaaS solutions provide users with physicalor virtual resources that satisfy the require-ments of the user applications in terms ofCPU, memory, operating system and stor-

3


1209


IJSER

age. Such Quality of Service (QoS) pa-rameters are established through a ServiceLevel Agreement (SLA) between the cus-tomer and the service provider. The enduser has full controls over the virtualisedcomputer instance, and can customise theinstance accordingly. Unlike purchasing thephysical servers, IaaS is usually charged ona utility basis depending on the consump-tion of the resources. A big name in IaaSspace is Amazon.com, which launched Elas-tic Compute Cloud (EC2) in 2006 to offer apay-as-you-go hosting service for customerscomputer applications. In 2008, Fujitsualso opened its London North Data Cen-tre to outsource data storage and comput-ing services with security options coveringUK and International regulations, and com-pliance auditing according to the ISO27001standard.

3.1.2 Platform as a Service (PaaS)

PaaS delivers cloud-based application de-velopment tools in addition to services fortesting, deploying, collaborating, hosting,and maintaining applications. It hides allthe complexity of managing the underlyinghardware, provides all the facilities requiredto support the complete lifecycle of buildingand deploying web applications and servicesentirely from the Internet. With PaaS,users can develop new applications or ser-vices in the cloud that do not depend on aspecific platform to run, and can make themwidely available to users through the Inter-net. The accessibility of PaaS offerings en-ables any programmer to create enterprise-scale systems that integrate with other webservices and databases. Two well-knownPaaS solutions, Google AppEngine and Mi-

Figure 1: Infrastructure As A service(IaaS)

crosoft Azure, provide users with a develop-ment platform for creating distributed ap-plications that can automatically scale ondemand. Other PaaS service providers are3Tera, RightScale, DataSynapse, Manjra-soft, Univa UD, Elastra and Enomaly.

3.1.3 Software as a Service (SaaS)

SaaS is a model of software deploymentwhere a provider delivers its software asa service to be used by customers on de-mand. Under the traditional SaaS model,an application resides at an offsite data cen-tre where the service provider maintainsdata, servers and related hardware. End-users access the application remotely viaan Internet browser. The SaaS model ispredicated on a one-to-many or multi-client

4


1210


IJSER

Figure 2: Platform As A service(PaaS)

delivery model whereby an application isshared across clients, providing opportuni-ties to customise the system to the needsof each customer. Examples of SaaS areSalesforce.com and Clarizen.com, which re-spectively provide online Customer Rela-tionship Management (CRM) and projectmanagement services. Google Apps pro-vides desktop applications which are hostedin the cloud and replaces traditional desk-top based Microsoft Office software. North-gatearinso offers on demand Human Re-sources (HR) solutions based on SAP Hu-man Capital Management (HCM) module.

3.2 Key Benefits Achievedthrough Cloud Comput-ing

• Flexibility - Cloud computing allowsuniversities to expand or contract com-

Figure 3: Softare As A service(SaaS)

puting power as required and al-lows bursts of computing power tobe utilised on an on-demand basis.This flexibility helps ensure resource-intensive processes will not slow downother business processes and comput-ing services are always operating at op-timal cost.

• Scalability - Cloud computing enablesuniversities to quickly scale up theirIT operations as provisioning of newcomputing resources and software ap-plications can be delivered at a desiredpace. Furthermore, constraints on pre-purchasing of resources to meet peakrequirement in traditional IT no longerexist.

• Economics - Traditional IT has mul-

5


1211


IJSER

tiple fixed and variable cost elements.In order to fulfil business requirementsand sustain day-to-day business oper-ations, universities must invest a largefixed amount for initial IT infrastruc-ture establishment and continue tospend variably for software and hard-ware maintenance. By outsourcing ITfunctions to the cloud, universities canleverage the features of a lean IT struc-ture to reduce the overall IT expen-ditures involved in software licensing,infrastructure development, on-goingsupport and upgrades.

• Inherited Resiliency - Cloud computingremoves single points of failure sincethe Internet is a highly resilient com-puting environment. Some competitiveservice providers also add extra func-tionalities to enhance resiliency. Forexample, the Availability Zones andElastic IP Address features of Ama-zon.com EC2 allow multi-location ofapplication software and dynamic IPaddress re-mapping mechanism in anevent of service interruption.

• Highly Automated Cloud comput-ing services are maintained by ded-icated IT professionals of cloud ser-vice providers. As a result, universi-ties IT staff no longer need to worryabout complex details behind the de-livered computing services, such ashardware maintenance, constant soft-ware update, etc.

4 What is vFabric SQL-

Fire?

• vFabric SQLFire enables dynamichorizontal scaling by creating ashared pool of memory across mul-tiple standalone physical devices orx86 rack/blade hardware, and thenexpanding or shrinking the pool asdemand changes.[3]

• Replicated or partitioned tables can bemanaged in memory alone, or in mem-ory and on disk. Application logic canbe routed to the data location, for in-creased performance from parallel exe-cution.

• Used as either the primary datastore ora front-end data-management layer forone or more existing databases, vFab-ric SQLFire assures continuous avail-ability for data within and across dat-acenters. Any table can be configuredto be replicated or partitioned into oneor more redundant copies.

• Built on a foundation of VMware vFab-ric GemFire technology, vFabric SQL-Fire benefits from years of testing andcommercial production reliability of itsdistribution subsystem. vFabric SQL-Fire includes a fast SQL query en-gine that compiles a query plan intobyte codes, and a sophisticated cost-based optimizer. Anyone with rela-tional database experience will find thevFabric SQLFire configuration and de-ployment model simple and intuitive touse and adapt. Unlike many populardata grids.

6


1212


IJSER

• vFabric SQLFire offers native persis-tence and recovery capabilities, andcan be used as a distributed datastore.Its use of established standards suchas SQL, JDBC and ADO.NET makesvFabric SQLFire easy to adopt bytraditional database developers writ-ing custom applications. Configurationand deployment is straightforward,and the product works effectively witha large ecosystem of compatible prod-ucts and frameworks, including object-relational mapping tools (Hibernate,NHibernate, etc.), schema-editing anddatabase management tools, SpringJDBC, and others.

• Applications using the standard SQLsyntax supported by vFabric SQLFirecan easily migrate to and from otherrelational databases, for flexibility andfuture-proofing as well as unparalleledperformance.

5 vFabric SQLFire Ar-

chitecture

5.1 Overview

vFabric SQLFire is an in-memory dis-tributed data management platform thatcan be spread across many virtual ma-chines, JVMs, and vFabric SQLFire serversto manage application data. Using dy-namic replication and partitioning, vFab-ric SQLFire offers the following featuresin the platform: data durability, trigger-based event notification, parallel execution,high throughput, low latency, high scal-ability, continuous availability, and WAN

distribution. The following figure1 showsvFabric SQLFire as the middle tier datalayer that orchestrates data delivery to en-terprise data consuming applications. Asdemand from consuming applications in-creases, the middle tier data layer expandsto appropriately meet seasonal workloads.Further, vFabric SQLFire is a full datamanagement system capable of managingtransactions and data consistency so thatenterprise data consuming application canrely on vFabric SQLFire as the system ofrecord. [Fig.4.] For additional persistence,

Figure 4: Enterprise Data Managementwith vFabric SQLFire

data can be written behind to a backupstore like a relational database, or otherdisk stores for archival purposes. vFabricSQLFire provides full persistence durabilityusing its own native shared-nothing persis-tence mechanism. Figure 5 illustrates howvFabric SQLFire can write and read syn-chronously and/or asynchronously to exter-nal datastores. vFabric SQLFire is based onthe VMware vFabric GemFire distributeddata management product and the ApacheDerby project the Apache Derby project isused for its RDBMS components, JDBCdriver, query engine, and network server.

7


1213


IJSER

Figure 5: vFabric SQLFire Architecturewith Capability to Write to TraditionalStorage Systems

The partitioning technology of GemFireis used to implement horizontal partition-ing features of vFabric SQLFire. vFabricSQLFire specifically enhances the ApacheDerby components,such as the query en-gine, the SQL interface, data persistence,and data eviction, and adds additional com-ponents like SQLcommands, stored proce-dures, system tables,functions, persistencedisk stores, listeners, and locators, to oper-ate a highly distributed and fault tolerantdata management cluster.[1]

6 VMware vFabric SQL-

Fire Highlights

• Higher throughput and lower latencythan traditional databases SQLFiremanages data and indexes in mem-ory, and is optimized for distributeddata access rather than for disk I/O.The SQLFire architecture introduces

no singlepoints of contention, eitherwith disk seeks, CPU cycles, or net-work access.[2]

• Scales easily to meet the highest de-mand - SQLFire is scalable becauseit partitions data dynamically acrossmany member nodes, distributing theload uniformly across servers, andmaking it easy to scale in or out at anytime

• Provides built-in fault tolerance andhorizontal scalability - In SQLFire,each member node can persist data indisk files independently of other mem-bers. Failures in disks or cache fail-ures in one node do not affect other in-stances being able to safely operate ontheir disk files. This ”shared nothing”persistence architecture allows applica-tions to be configured such that ta-bles can be persisted on different nodesacross the cluster, reducing the impactof disk latencies.

• High availability around the globe andDisaster Recovery options SQLFirelets you synchronously replicate dataat memory speed within one datacen-ter which makes scalability and faulttolerance simple. SQLFire can alsoasynchronously replicate data acrossthe WAN to remote datacenters, whichmakes it easy to run the same applica-tion in different locations and meetingdisaster recovery and/or regulatory re-quirements.

• Ideal for high transaction rates SQL-Fire memory-speed write performanceis ideal for large-scale databases with

8


1214


IJSER

high transaction volumes and demand-ing Service Level Agreements (SLAs).It is architected to manage complextransaction-oriented web portals wheredisk and network overheads choke op-timum service delivery to customers.

• Caching options for existing dataSQLFire can be used as a primarydatabase or as a middle-tier cachefor an existing database, offeringread-through, write-through and write-behind caching.

• Accommodates application demandwith near-linear scale SQLFire canbe used as a primary database oras a middle-tier cache for an existingdatabase, offering read-through, write-through and write-behind caching.

• Standard SQL Syntax and toolsDatabase administrators can use andadapt SQLFire using their current skillsets and tools.

7 vFabric SQLFire De-

sign and Sizing

Designing and sizing is a three step pro-cess shown in Fig.6 Step 1, determine entitygroups, step 2, determine memory size ofthe data fabric, and step 3, establish build-ing block virtual machine and JVM size,and how many vFabric SQLFire membersare needed. After these steps have beenconcluded, a load test should be used to ver-ify the calculation and sizing assumptions.

Figure 6: vFabric SQLFire Three Steps De-sign and Sizing Process

8 Partitioning

For custom enterprise applications backedby RDBMS schema tables that are highlyaccessible during the course of a regularbusiness day, a partitioning strategy im-proves performance to more easily meetSLA requirements[1]. These types of fre-quently accessed tables are often referred toas hot tables due to their high rate of datainsert, update, delete, and read operations.In addition to the high rate of data change,it is common that these hot tables also be-come unmanageable within a single nodedue to their size. In such cases, a vFab-ric SQLFire horizontal partitioning strategycan be used to split the data volume intosmaller, more manageable data partitions.With vFabric SQLFire horizontal partition-ing, an entire row is stored in the same hashindexed bucket. Buckets are the contain-ers for data that determine its storage site,redundant sites, and the unit of migrationfor rebalancing. You can hash-partition atable based on its primary key or on aninternally-generated unique row ID if thetable has no primary key. Other partition-ing strategies can be specified in the PAR-

9


1215


IJSER

TITION BY clause in a CREATE TABLEstatement. Strategies supported by vFab-ric SQLFire include hash-partitioning oncolumns other than the primary key, range-partitioning, and list-partitioning. In Fig. 7an example Flights table is partitioned byFLIGHTID (primary key for Flights table)into three buckets, where the first bucketfor rows 1 to 3 resides on vFabric SQLFireserver 1, the second bucket for rows 4 to6 resides on vFabric SQLFire server 2, andthe third bucket for rows 7 to 9 resides onvFabric SQLFire server 3. vFabric SQLFiredirects that all access to flights data by theFLIGHTID primary key for rows 1 to 3 areexecuted on vFabric SQLFire Server 1, andsimilarly for the other rows. vFabric SQL-Fire automatically manages all partitioningthrough this bucketing system so long asthe designer has provided the correct PAR-TITION BY COLUMN(FLIGHTID) clausein the table definition.

Figure 7: Partitioning Flights Table UsingvFabric SQLFire Bucket System

9 CONCLUSION AND

FUTURE WORK

ThrivOn’s VMware vFabric Consulting Ser-vices can help accelerate your journey toPlatform as a Service based on VMwarevFabric solutions. Our expert vFabric con-sultants will help you leverage best prac-tices for planning migrations, expand yourknowledge base around vFabric solutions,develop a PaaS Plan and vFabric PaaSRoadmap, and help transform your cloudapplication development environment.

• ThrivOns SQL Fire Planning Service: Provides an efficient transition planto Platform as a Service for applica-tion development. With guidance fromThrivOns vFabric consultants, you’llgain a through in-depth understandingof successful PaaS deployment factorsand related process, steps, implemen-tation and production scenarios.

• ThrivOns SQL Fire Accelerator: Pro-vides core knowledge, expert guidance,and a non-production installation, con-figuration of Data Director in your en-vironment; including a high-level archi-tectural review.

References

[1] Emad Benjamin,Lyndon Adams, Vin-cent Ford, Jonathan Fullam, MarkJohnson, Catherine Johnson, GideonLow, Jeff Markham, Sudhir Menon,Jags Ramnarayan, Mike Stolz, LiseStorc, Guillermo Tantachuco, SumedhWale, and James Williams. ” VMware

10


1216


IJSER

vFabric SQLFire Best Practices Guide”2012.

[2] ” VMware vFabric sqlFireFast, ScalableSQL Data in The Cloud” 2011.

[3] ” VMware vFabric SQLFireMemory-Optimized Distributed SQLDatabase”2011.

[4] ”Providing Scalable SQL data in thecloud With VMware vFabric SQL Fire”2011.

[5] ” Cloud Computing A newsletter for ITProfessionals Issue 3 ”.

[6] James l. Johnson ” SQL in the Clouds”2009.

[7] ” Cloud Computing Basics ”.

[8] Yu Cao , Chun Chen , Fei Guo , DaweiJiang , Yuting Lin ,Beng Chin Ooi, Hoang Tam Vo , Sai Wu , Quan-qing Xu”A Cloud Data Storage Sys-tem for Supporting Both OLTP andOLAP”2010.

11


1217


IJSER

Ms.Shinde Rupali R Prof.Mr.Maral Vikas B. K.J.COEMR,Pune May …€¦ · Scalability in Cloud...

Documents

Transcript of Ms.Shinde Rupali R Prof.Mr.Maral Vikas B. K.J.COEMR,Pune May …€¦ · Scalability in Cloud...