Plumbing the Computing Platforms of Big Data

Dilma Da Silva

Professor & Department Head

Computer Science & Engineering

Texas A&M University

TAMU Big Data Workshop 2

Where should I put host my big data app?

• Systems == hardware + software enabled to run applications (+admins)

• Opportunity for two-way collaboration:– You know which platform you want to use, but your

application is not performing well in its target platform

• Get resources where needed when needed

– You are not sure about the right platform for the application

– Your app can server as representative workload for evaluation system software research ideas2/13/15

TAMU Big Data Workshop 3

Do I need large scale systems ?

• Different communities, different requirements:– High End Computing / SuperComputing– Scale-out enterprise systems

• But similar trends ?!

2/13/15

TAMU Big Data Workshop 4

My app is running, now what?

• ‘plug’ your app in the system, and it just works– Support for evolution

• Efficiency: work done without wasting resources• Elasticity: grow and shrink to adapt to demand• Scale: can handle the ‘big’ part of your problem• Agility• Heterogeneity• Resiliency: failures happens without loss of data or work• Problem determination: system helps you find your

‘bugs’, • Security, privacy• On the cheap

2/13/15

TAMU Big Data Workshop 5

Computing Industry Offerings(and where CSE people can help)

• High End Computing (HPC, SC, commodity clusters)– Often application needs to be optimized for

the platform• Large gap between domain and platform experts

• Cloud computing platforms– So far optimized for enterprise systems

2/13/15