1

Data-IntensiveResearch: Actions

to make better use of Data for

Research

Malcolm Atkinson (& David De Roure)

mpa@nesc.ac.uk

2 February 2010

Reportto

EPSRC Research Facilities SAT, London

Mission goal: learn how researchers use data

Acknowledgements: the UK e-Science Directors CIR authors, our teams, the EPSRC (Sarah Fulford), the RCUK, USA office (Ruth Lee) and all of our hosts in the USA,

their good ideas; opinions, observations and recommendations are our own.

Mission

• How will research be conducted in the future?• Discussion and insight from the experts at the

cutting edge– What are the changes in

method and practice?– What works and what

doesn’t work?• Share our e-Science experiences and do the

groundwork for future collaborations• BBSRC and ESRC missions too

Senseable Cities

Andrey Rzhetsky: hand-crafted datascopes

Institute for Data-Intensive Engineering & Science

2D X-Ray Detector SystemsEuropean X-Ray Laser XFEL @ DESY

•≥ 106 pixels per frame for one detector•O(400-500) frames per train (goal, likely will start with less)•10 trains per second (machine allows up to 30 Hz…)•With 2 Byte/pixel average rate ≥10 Gbyte/sec for one 2D detector!•Time between frames as short as 200ns buffering needed•5 Detectors

600 μs

99.4 ms

100 ms 100 ms

200 ns

LPD

Text

Year Rate Capability[Gbyte/sec]

Storage Space[Petabyte]

2009 1 3

2012 5 26

2016 40 200

Datascopes Summary

• Better methods for extracting information from data• better algorithms for

discovery, selection, fusion, distillation, aggregation, presentation

algorithms transformed to run incrementally

• Better strategies for using the algorithms• Better data/metadata and semantics• Better platforms supporting the strategies• Data centres hosting data and computation

• Coping with more complexity, more users & more questions

• Knowledge, questions & datascopes co-evolve

Rally cr

oss-d

isciplin

ary

effortRally cr

oss-d

isciplin

ary

effort

Ramps: Summary

• An easy path to use a data analysis method• An opportunity• Not an obligation• Engage as far as you want

• Use a service for routine tasks• Types of ramp

• in browser - now powerful - can reach the GPU• in familiar tools• support from centres and crowd-sourced

• Strongly linked with education• Removes distracting technical clutter• Rescues educators & students• Ramp & education co-evolve

Boost investm

ent

hereBoost investm

ent

here

Actions

1. Workshops on DIR

2. DIR education3. Sand-pit to

inject initial momentum

4. Test best practice

5. Immediate research challenges

6. DIR facilities pool7. Boost reference

data services8. Foundational

research9. Green DIR10.Coordination

http://tinyurl.com/ye8x4bw

Phase 1

1. Workshops on DIR

2. DIR education3. Sand-pit to

inject initial momentum

4. Test best practice

5. Immediate research challenges

In Edinburgh 15-19 March 2010

http://tinyurl.com/ye8x4bw

http://www.nesc.ac.uk/esi/events/1047/

Phase 2

6. DIR facilities pool7. Boost reference

data services8. Foundational

research9. Green DIR10.Coordination

S/W, H/W & support: what services do researchers need? How much? How soon?

More softw

are; Less

hardwareMore so

ftware; L

ess

hardware

More bandwidth; Fewer

FLOPSMore bandwidth; F

ewer

FLOPS

Summary

• Much research is data intensive

• More of it will be

• Exploiting the opportunity is urgent for the UK (you/your org.)

• This requires changes• In facility provision

• In research investment

• In research behaviour (incentives)

• In education

• These changes are part of the digital revolution• Understand, engage and ride the wave (co-evolution)

• Investing in data-intensive research• Will accelerate research

• Deliver more applicable research

• Provide a better return on investment

24

ADMIRE – Framework 7 ICT 215024

?

Picture compositionbyLuke Humphrybased on prior art by Frans Hals

www.omii.ac.uk

www.admire-project.eu

www.ogsadai.org.uk

www.nesc.ac.uk