Post on 15-Dec-2015
BalticGrid-II ProjectBalticGrid-II Project
BG 2nd AHM, GOD, 12.05.2009, Riga 1
BalticCloud: Cloud Technologies for Industry and Academia
BalticCloud: Cloud Technologies for Industry and Academia
IlIljja Livenson, NICPBa Livenson, NICPB, Estonia, EstoniaTechnical coordinator Technical coordinator of BalticCloudof BalticCloud
ilja@kbfi.eeilja@kbfi.ee
2
OutlineOutline
Quick introduction to Cloud computingQuick introduction to Cloud computing
BalticCloudBalticCloud What services will we offer Technological solution
““New” application classesNew” application classes
Collaboration with SMEsCollaboration with SMEs
BG 2nd AHM, GOD, 12.05.2009, Riga
Cloud computing Cloud computing
““Cloud Computing refers to both the Cloud Computing refers to both the applications applications delivered as services over the Internet delivered as services over the Internet and the and the hardware and systems software in the hardware and systems software in the datacenters that provide those servicesdatacenters that provide those services… …
The datacenter hardware and software is what The datacenter hardware and software is what we will call a Cloudwe will call a Cloud……
Cloud computing has the following characteristicsCloud computing has the following characteristics
1. The illusion of infinite computing resources..
2. The elimination of an up-front commitment by Cloud users..
3. The ability to pay for use … as needed…” UC Berkeley RAD Labs
3BG 2nd AHM, GOD, 12.05.2009, Riga
Spectrum of CloudsSpectrum of Clouds
Instruction Set VM (Amazon EC2)Instruction Set VM (Amazon EC2)
Bytecode VM (Microsoft Azure)Bytecode VM (Microsoft Azure)
Framework VM (Google App Engine)Framework VM (Google App Engine)
EC2 Azure AppEngine
Lower-level,Less management
Higher-level,More management
Automatic scalability and failover4BG 2nd AHM, GOD, 12.05.2009, Riga
Cloud model and BCCloud model and BC
5
… as a Service
BalticCloud
Users
BG 2nd AHM, GOD, 12.05.2009, Riga
Building brick: Datacenter as IaaSBuilding brick: Datacenter as IaaS
Virtualize computational resourcesVirtualize computational resources There are drawbacks!
– Virtualization overhead (CPU, IO), “noisy neighbours”, state preservation, etc
On-demand resourcesOn-demand resources Creating virtual machine with a specified set of
resources is possible Applications consume as much resources as they
actually need, overprovisioning is still there, but to a lesser extent
““Cloud of clouds” approachCloud of clouds” approach Open question
6BG 2nd AHM, GOD, 12.05.2009, Riga
Virtualization toolkitVirtualization toolkit
There are several projects out there offering cloud There are several projects out there offering cloud solutions (IaaS)solutions (IaaS) OpenSource: Eucalyptus, Nimbus, AbiCloud, … Commercial: VMWare, Citrix, …
Our choice: EucalyptusOur choice: Eucalyptus Great team! Integration with RightScale Latest version: 1.5.1
– EC2, S3, EBS
7BG 2nd AHM, GOD, 12.05.2009, Riga
Eucalyptus servicesEucalyptus services
EC2 virtual machine provisioningEC2 virtual machine provisioning
IImplementation of Amazon S3 interfacesmplementation of Amazon S3 interfaces - Walrus - Walrus Very simple bucket based filesystem with ACLs
And Amazon's Elastic Block StoreAnd Amazon's Elastic Block Store analogue analogue Persistent volumes
Doesn’t include scalability/reliability mechanismsDoesn’t include scalability/reliability mechanisms But...
8BG 2nd AHM, GOD, 12.05.2009, Riga
Security (access)Security (access)
Eucalyptus uses X.509 security infrastructureEucalyptus uses X.509 security infrastructure “Light” version No VOMS extensions, no OCSP, no CRLs, no SAML
assertions, etc
Can we use BalticGrid/EGEE infrastructure?Can we use BalticGrid/EGEE infrastructure? Well, yes, but… One of priorities for us
9BG 2nd AHM, GOD, 12.05.2009, Riga
Security (process)Security (process)
Job/Application isolation levelJob/Application isolation level Low (same users within VO)
– what if if size(VO) > 1000? Semi-low (different user groups) Medium (e.g. jail/chroot, shared kernel) High (separate VM for each job)
Lower level = lower security level, less Lower level = lower security level, less customization options, (more efficient)customization options, (more efficient)Network securityNetwork security Not every switch supports VLANs, or does that well Need that if we want to give local root to the users
– Or just encrypt everything critical
10BG 2nd AHM, GOD, 12.05.2009, Riga
Use case: InteroperabilityUse case: Interoperability
Complicated problemComplicated problem
OGF has published a number of standardsOGF has published a number of standards E.g. OGSA-BES for execution or GLUE for information
system
Middleware of interest for our region:Middleware of interest for our region: gLite, ARC and UNICORE Implementation of standards support is slow
Mid-term solutionMid-term solution Provide core components for every system as virtualized
images On-demand WN creation
11BG 2nd AHM, GOD, 12.05.2009, Riga
“New” application classes“New” application classes
Grid was built for batch processingGrid was built for batch processing There are some workarounds
– Pull-mode execution– “VO-box” component of gLite
But no solution!
OLAP/OLTPOLAP/OLTP Databases Application servers, Web servers BI Load balancing
Hadoop frameworkHadoop frameworkAppScale and other PaaS solutionsAppScale and other PaaS solutions
12BG 2nd AHM, GOD, 12.05.2009, Riga
Missing functionalityMissing functionality
MissingMissing Monitoring Accounting Reasonable OS image management Billing
Some issues couldSome issues could be solved by integration with be solved by integration with RightScaleRightScale Vendor lock-in Not open-source
We are thinking of reinventing some wheelsWe are thinking of reinventing some wheels
13BG 2nd AHM, GOD, 12.05.2009, Riga
Collaboration with SMEs (1)Collaboration with SMEs (1)
SME = Small and Medium EnterprisesSME = Small and Medium EnterprisesSo far so badSo far so bad It has been very painful with gLite based solution It hasn’t been to easy with “pbs cluster access”
ReasonsReasons Shared WN (typical case for multicore) is not acceptable Interfaces are too complicated and components are too
unstable Not enough control: “What do you mean by SL3/4? What do
you mean by CLI? What do you mean by queuing? We already have J2EE application, just give us the cluster to deploy it on!”
Accounting Interactive applications
14BG 2nd AHM, GOD, 12.05.2009, Riga
Collaboration with SMEs (2)Collaboration with SMEs (2)
We hope to make it better with cloud approachWe hope to make it better with cloud approach Industry driven, not HEP driven Reasonable economical models Existing success stories
Courses for SMEs on best practices using cloudsCourses for SMEs on best practices using clouds In all countries involved Building a network of adopters
Free access to resources for academic Free access to resources for academic startups/spinoffsstartups/spinoffsHybrid cloudsHybrid clouds Selling unused cycles to SMEs
15BG 2nd AHM, GOD, 12.05.2009, Riga
SummarySummary
Cloud computing is not a silver bulletCloud computing is not a silver bullet Too much hype
But…But… It does improve on many aspects
– Security, resource usage, interactive applications Economically motivated
– Startups and prototyping, one-off tasks, research at scale– Scaling solutions
BalticCloud will strive to provide cloud services to BalticCloud will strive to provide cloud services to academia and industryacademia and industry
16BG 2nd AHM, GOD, 12.05.2009, Riga
CreditsCredits
Many thanks to people involvedMany thanks to people involved Aake Edlund - BalticGrid Project Director (Sweden) Mario Kadastik (Estonia) Janis Kulins (Latvia) Dalius Mazeika (Lithuania) Eduardas Kutka (Lithuania) Yuri Ziamtsou (Belarus) (your name here)
http://cloud.balticgrid.eu
17BG 2nd AHM, GOD, 12.05.2009, Riga
Happy endHappy end
Questions?Questions?
Many thanks to the Many thanks to the Grid Open Day Grid Open Day organizers!organizers!
18BG 2nd AHM, GOD, 12.05.2009, Riga