QCloud 1

QCloud

Queensland Cloud Data Storage and Services27Mar2012

27Mar2012

QCloud 2

NATIONAL RDSI

• 6 primary, 4 additional nodes• 150-200 PB by 2014• 100Gb/sec node interconnect

• Storage for large national collections• Shared access by research communities• Research data management• On-shore data cloud• Low-cost research production quality of service27Mar2012

QCloud 3

NATIONAL NeCTAR RESEARCH CLOUD

• 6 primary nodes (maybe 7)• 24,000 cores by 2013

• On-demand compute capacity for research• Hosted shared services for research communities• Virtual laboratories• Research tools, workflows

Uni of Melbourne QCIF/UQ ANU

Monash Uni27Mar2012

QCloud 4

QCLOUD COMPONENTS

RDSIDATASTORE

NeCTARRESEARCH

CLOUD

QERN CLOUD

GENOMICS CLOUD

5x

>15x

27Mar2012

QCloud 5

QUEENSLAND RDSI NODES

• ~ 30 Petabytes by 2014• Primary node hosted at UQ (embargoed)• Additional Node at JCU (awaiting decision)• National focus: ecosystems, genomics

75% Merit Allocated Collections 25% “CollectionDevelopment”

Can add RDSI+ commercial research storage

27Mar2012

QCloud 6

COLLECTIONS

Market Data CollectionsActive research projects

Collaborations across multiple locationsFast disk, flash memory – immediate access

Vault Data CollectionsRarely accessed data archival storage

Broad availabilityTape, slow disk

27Mar2012

QCloud 7

RESEARCH COMMUNITY

RDSI NODE SERVICESOWNERQualification

CaptureIngest

Bulk Update Curation

Metadata (with ANDS)Resilience (multiple copies)

ManagementEntitlements to Access

MonitoringSupport, Help Desk, Training

USEREntitlements to Access

DiscoveryRead

UpdateDuplicate

MonitoringSupport, Help Desk, Training

27Mar2012

QCloud 8

TARGET RESEARCH USERS/SERVICESPOWER USERSPB-scale data collectionsLow-level access servicesSpecialised treatment

LARGE DATA USERSTB-scale data collectionsDirect data access, FTPAutomated allocations

LONG-TAIL USERSGB-scale data collectionsCloud accessDropbox style on PCLittle allocation surveillance

DATA

CO

LLEC

TIO

N S

IZE

NUMBER OF USERS27Mar2012

QCloud 9

QCIF/UQ NeCTAR NODE

• 4,000 cores, up to 4,000 virtual machines• Research service quaility• 70% for NeCTAR Allocated Services• 30% for Queensland Allocation

27Mar2012

QCloud 10

SERVICES

HOSTED SERVICESPermanent web sites

Research communitiesVirtual laboratories

Research toolsCollaboration

COMPUTATIONOn-demand virtual

machinesOff-load “small” jobs from

HPC/clusters

27Mar2012

QCloud 11

COMPUTE USERS/SERVICES

LONG-TAIL USERSHOSTED SERVICES1-4 coresSmall HPC jobsN

UM

BER

OF

CORE

S RE

QU

IRED

NUMBER OF USERS

Tier 1,2,3 HPCMore than 16 coresSpecialised treatment

MEDIUM COMPUTE USERS4-16 coresLarge memory VMs (1TB+)Automated allocationsSmaller HPC jobs

27Mar2012

QCloud 12

EARLY NODE (QERN)• pQERN– Operational October 2011– Pre-RDSI/NeCTAR– Genomics++

• QERN– Delivered February– Operational ~April– 330 cores – 0.5 PB disk– RDSI/NeCTAR services– Early adopter researchers– Operational experience27Mar2012

QCloud 13

PROJECTED NODE SCALE-UP

27Mar2012

Cores4000

3000

2000

1000

0

2011 2012 2013 2014

NeCTAR Research Cloud

RDSI Data Storage

Today, QERN

QCloud 14

PARTICIPANT ROLES

27Mar2012

QCIF Governance, Management, Research Support

RDSI Lead Agent

NeCTARLead Agent

UQ RCCManagement,

SysAdmin

JCUManagement,

Operations

UQ ITSAcquisition, Operations

PartnersResearch Support

Collection & Service Owners & Users

QCloud 1527Mar2012

QCloud 16

QERN Progress

27Mar2012

Familiarisation• Hardware installation• Software systems

Service Definition and

Delivery

• COO and Solution Architect start• Service catalogue• Software stack• Registration processes• Trial usage

Production• Early adopters• Basic services• Continuing service introduction

Primary Node(Jul2012)

Mid Feb

QCloud 17

QCLOUD PARTNERS

• QCIF• UQ• JCU• CQU• Griffith• QUT• USQ• USC• Bond

• CSIRO• NICTA• TERN• DERM• DTMR• DEEDI• Ergon Energy

27Mar2012

QCloud 18

FUNDING FOR QCLOUD

27Mar2012

Cash $15M

RDSI Primary NoDE $1.5M (contracting)

NeCTAR Research Cloud $2M (contracting)

Smart Futures Co-investment Fund $3.6M (signed)

RDSI Additional NoDE $266K (awaiting decision)

RDSI ReDS, DaSh, NRN funds $6.5M+ (projected)

DERM, DTMR co-investments $600K (offered)

QCIF QERN $200K

In-kind $5M

UQ Data centre facilities, operations and power consumption plus researcher support $2.3M

JCU Data centre facilities, operations and power consumption plus researcher support $700K

Other members, CSIRO, NeCTAR, NICTA, TERN, DERM, DTMR, Ergon Energy, QCIF researcher support $2M

QCloud 19

Agreements Progress

27Mar2012

Smart Futures

• Signed with DEEDI, Jan2012; funds flow in July• Contingent on RDSI, NeCTAR

RDSI• Final adjustments in progress• Signature expected within April

NeCTARResearch

Cloud

• Feedback provided to NeCTAR• Signature expected within April

Operations (UQ,

JCU)

• Drafting agreement text• RDSI, NeCTAR agreements as back-to-back schedules

Data, service owners

• To be completed• Preliminary arrangements for early users

QCloud 20

RISK MANAGEMENT – QCIF

• Power upgrade at UQ DC2 (JCU also?)– Decision deferred to May

• Transfer to new UQ data centre– PACE building offer replacing DC2 – network

• Staff and skills availability• Software and skills maturity• Authentication, entitlements

27Mar2012

QCloud 21

RISK MANAGEMENT – NATIONAL

• Levels of interaction: Lead Agents, other nodes• National support and help desk structure• RDSI ReDS and DaSh Programs

27Mar2012

QCloud 22

RISK MANAGEMENT – LONG TERM

• RDSI value proposition to researchers• Growth past 2014• Business model for sustainability• Commercial service quality

27Mar2012

QCloud 23

QUESTIONS?

27Mar2012