Next-generation IT infrastructure: status and opportunities

Dave BerryGrid Computing Now!

National e-Science CentreAHM 2007 11th September 2007

Overview

What industry cares about

- and what it doesn’t care about

Underestimating the industrial state of the art

- and overestimating industry’s needs

- some examples from Grids Mean Business

How GCN! can help

- active areas of interest

My focus on IT infrastructure

- there are, of course, other areas of e-science that have plenty to offer UK industry

Innovation that Matters © 2005 IBM Corporation

What does a business really care about?

“Cost of ownership”

“Time to market”

“Service levels”

John Easton, CoreGRID

Of little interest:

• What the technology is called

• Technology for its own sake

• Technological purity

Trends in IT infrastructure

GridComputing

Softwareas a

Service

Service Oriented

Architecture VirtualisedResources

UtilityComputing

Grid: Applying the resources of many computers in a distributed network, in parallel, to a single problem

SaaS: Providing software capability as a consumable commodity, at commodity prices

SOA: Delivering IT functionality as reusable, interoperable, location independent services

Virtualisation: Providing an IT platform independent of the hardware underlying it.

Utility: giving users all the resources they need at the time they are needed, at a cost that is related to the business value delivered

Innovation that Matters © 2005 IBM Corporation

Academia under-estimates industry

Multi-thousand node “grids” are relatively commonplace Geographic “grids” are becoming more common Ultra-‘bleeding edge’ solutions are already being implemented

– but you won’t necessarily hear about them… except maybe rumours Don’t underestimate what is achievable today

–This is the benchmark that a new solution has to beat.

John Easton, CoreGRID

Innovation that Matters © 2005 IBM Corporation

Academia over-estimates industry

Multi-organisation systems are extremely rare

–Those that do exist are usually master-slave relationships rather than peer-to-peer relationships

Web Services will not happen in the foreseeable future for most companies

Most grids are ‘inside the firewall’– Security is NOT an issue for these

John Easton, CoreGRID

Using 15,000 CPUs 66% of the Time

Micron Grid Overview14077 Processors, 8.053 TFlops, 63rd Top500

Rank

529 TeraBytes Disk

102 user accounts in all pools

1,281,636 job hours, 1780 Processor-months

Primarily Windows, plus Linux, some Solaris

Condor system managed in-house

Centralized governance, distributed

management

Brooklin Gore, OGF20

OGFMayl, 2007 ® 2007 Citigroup Confidential

Move to shared resource environment in a controlled manner

End State: all resources are shared with grid scheduling and policies ensuring SLA’s are met

Roadmap and Design

Andrew Dolan, OGF20

9

Virtual Resource Market - Details

VirtualResource

Market

VirtualResource

Market

Netw

ork

F

abrics

Sto

rage

F

abrics

BidsBids OffersOffers

Co

mp

ute

Fab

rics

Message Hub

n-tier application

Compute Farm

Canonical Application Architectures

Physical ResourcesVirtualized Resources

$/Unit Performance$/Virtual Unit Performance

Time Slice Offers

Time Slice Bids

Minimize $/Unit Performance Maintain SLAs

$ for SLAs (Budget) Match

$ for SLA to $/Virtual Unit Performance

Compute Fabric C1

Canonical Architecture A

Canonical Architecture B

Canonical Architecture C

Bid for Storage Fabric

Bid for Network Fabric

Bid for Compute Fabric

$/Fabric

Compute Fabric C2

Network Fabric N1

Network Fabric N2

Storage Fabric S2

Storage Fabric S1

Bid for Storage Fabric

Bid for Network Fabric

Bid for Compute Fabric

Bid for Network Fabric

Bid for Compute Fabric

Offers of C2

Offers of N1

Offers of N2

Offers of S1

Offers of S2

Offers of C1

SLA

Chris Swan, OGF20

What can GCN! offer e-scientists?

Potential collaborators

Via Advisory Council, GCN! membership, personal contacts, other KTNs, etc.

Kick-off Meetings

Bringing potential collaborators together to form study groups or discuss funding proposals

Dissemination

Webinars, events, web site

Knowledge Transfer Networks exist to share information and bring people together

Road traffic modelling

Software Licensing

Old World New World

Static

Silo

Physical

Manual

Application

Dynamic

Shared

Virtual

Automated

Service

Need: 2xProcessing Capacity per annum; Target: 60% Energy Reduction* by 2050

Modelling power management & cooling,

Utilisation optimisation via virtualisation

GCN! Roadmapping Event

The Next Information The Next Information InfrastructureInfrastructure • Workshop, May 30th, Intellect• 25 attendees representing high-tech; users; KTNs; Government• Excellent feedback from vast majority• Generating insight on potential innovations; promoters and next steps for KTN• Report available soon

Some suggested innovations

Security threat detection and response

New Infrastructure

Wrapping applications for Grid deployment

Energy-efficient IT

Smart travel

Traffic modelling in real time – signage control

Dynamic journey planning

Dial-a-ride public transport

Proactive health care

24/7 monitoring

At-home advisory services (web 3.0)

Personal ‘MOT’

Integrative biology – driving personalised medicine

More possible gaps in near-market R&D

Application design for next generation infrastructuresApplications may have multiple instances, run anywhereMigration techniquesDynamic integration

Managing large-scale infrastructuresDynamic, possibly conflicting, policiesData provisioning

Green ITModelling of power management, cooling, etcUtilisation optimisation

Markets, brokering, security, etc.Data licensing & securityMicro-paymentsSLA & QoS management

Upcoming Events

GCN! Technical Webinars:

Green IT Webinar

25 October 2007

Software Licensing Webinar

November 2007

What people say about our webinars:

“The event was very useful - content and speakers very good - format fine” – Oracle

“ I would recommend this webinar Distributed Systems in e-Health to a colleague.”- IBM

“Excellent” - Platform