1

iPlant Data Store (iDS)Supporting the Lifecycle of Data

Nirav Merchant

1

2011

2008

A Big Problem...

A Big Problem...

Transfer Storage Analysis Visualization Metadata Mark-up Search and Discover Share/Collaborate Publish

High-throughput Data Acquisition

In 11 Days• Generates 4TB of raw data• 600,000,000,000 bases of DNA sequence (200 human genomes)

5

and it a global phenomenon

• $70K for ~30 camera sets• ~200 movies of plants undergoing a dynamic growth process• “Only” 4GB a day

High-throughput Phenotyping(Watching Grass Grow)

Big Data in EcologyGlobal Multidimensional Data

GGGTGCCCAAAAGCCCGGTTTGTTAGCCCCTTTCCGATTCCCTCACCCAATCTCATTAAAAGCAAGCCCAGCAGGCCCTGCCTTAACCTGTCCCCAGACAGCCAGCCCTCACCAGGCCGCTGGCATTACTCAATGCTCCGCCACGAAGCAAGCTCTCCCGAATACGACAGATGCGGAAGTGGCTCAAGAAGTCGGAGGAATCAAGTAGTTGGAAAACTGTATGCACGAGGGGACATGAGTCTTCTGGGAATTGGCAACATTGGCAGAAAATAAAGGGGAATACAAGGGGGGTAGGAATCCACTTTGTTAGGTGTAGCTATACTCACGTAAGTAGTCGGCCTAACCTTCGGTTCCCGTAACCAAGTTGTTCTTTCTCACTCCTGATTGACTTTTGATTACTGAATCCATACTTTTTTACTTTTTTTGAATTTGAAGTGTGGGGAAAAGGGCGCCCTCTACTTCTACTTCTAACTACAGGCGAAAAGCTGGCATTGCAAGCAAATAGAGAGCCCCCGCCCGTTTGAGTCGTTGCGAGCCGGAAAGCGTACCAGCGGTTTGAGTCGCGAAAGGGCCGCTTGCTTAATTATATTATAATATAATATATATAATCTTCTATCTCTATCTATCAACAATAAAATCAGAAGAAAGTAAAAAAAATATAAAAAAGAAAATCATTTTTTGTATCCAATTTTGCATTCCTGGGAAGAGGAAGAAGCAGATAGAGCAAAGGCCTCCTCTTTCCGTCCGCTCTTCCCGAAGTGAGCGAATTGCATGTAGAGATCCGTAGGGGCTTATAGTTTAATTGGTTGAAACGTACCGCTCATAACGGTGATATTGTAGGTTCGAGCCCTACTAAGCCTACCACCCCCTTCTCTTCACCCGATACAAGGCAGTCGAAGTCCCCGCCACCCTGCAGATCTCAATCTAGCGACGGCACCTAGAACCACACTGCTGCCGCTGCCCGAAGGGCACGCCTCCTACGCTCTTGCAGCATGCCCCCTTCGGGGCAGATGTTACTATACTAAAAAAGAAGGCCCTCGCTAAGCGCTGGTTCTATCCCGGCCAAGCAACCAAGGTGGGAATAGTGAACGAAAGAGAAGGACATTGTTCAGAGTGAAACTGAACCCCCTTGATCAATCCTGTAAGAACGAAGACTTCACCAATCGACCAATGGGCCTTTCCTTGTAGGCGGCGAAGGGCAGGTGAACACTCTTCCCTTGGAACCTGCGCATAATGAAATATAACATTTTTTACTTCCATGGTCATACTATATTTATCTTTATATTGCGAATGAGTCTGGACCATCTCCTATTGTAGTATCAAAATGAATATGACTTTACTTTGAAGTTTAGCCCTCTTTCAACAGTATGACAACCTTCCCAATCACTCGGTTCAATCCTTACCTGAGGATGACAAGGCTTGGCTGATAGGCCGAGGACGAAGCATGCACCTTCGCTTAACCTTCGATGTTGTCATCACGCTGCCTTTCGCATGTCGGGCCTATACACGCTCGTCAAGTTACACCTTAACTGCTTTCTCAACGCGCGGCTCTTATAGATAACCCTTCCTTATCAATCAAATAAGCATTTGTGAGTTGAGATTCCTTCCCTATGTTTCAAAGCTAGCTTCTCTAGCTATACTGTGTGACCCACCTCCTCCCTTCGCTCTCTTGCTAGAGCTGGTTCTAAGCCTACCTTTTCCTTCCCCACCCTCTGTCTGACCAACCTCTGACTTCGATAATGACCTATAAACAATTATTCCAAGTGAAACGATAAGCATGTTGGCTTAGACCTTATCTTACCAGACAAGACATTGATGTCTGCCCGAACATCGAAATGAATCTTTCATATGCGGATTTTCACTTCCCCTCATTAAGCTAGCTAGCGCAAGTGTCAGAAAGGATGAATGCATTCCGAGATCGAATTAGCCCCTTACATCTTAGAACATTTTATACAAGGAATGTGTGTGACCTCTCTGATAGTAAGAGCGCACTAACGGAAAGAGAAATTGTTATAAGGAAAGAGAACACACTAGGGTAGAGTGGAAAGGGAACAGGAAAAAACTTTAGTCGACTAACTCTAGTATTCATGCTAGAACAGAGCTCCCGAAATAAAATATATCAATTATAGCGCTTCATCACTTGAAATAGGATCTTGCCTACGGCCCTAGTACTTTACTTTAGTCGATCGACATCTCATTAGCAAACAAACATAGAAGAGTCAGCTTCCTCAGTCTTGGTTATCGAGTTATCTTACCTGACAGGGTCGGCTAGGTGAGTTTGATTCCATTCCCGTGGCAAAAGGAAAAGAGCTTGATATCCGGGCTTCTATCGGTGAAGAAATGTTATGCCCACGGTTCCGTACTAAAGAATGAGCCAACAGCTATCTCCTTAGCTTCTTAAGGCACTCTTTTTCTGTTTAGTTATTGGTAATCCATCCGAGTGATCTATCTTATCGATAAAGAAATTCTCTCCCCTTACCGATCTTGTTATGCCTCCCGCGGTACATACAAAGGAACCTTCTATCCCATCGGTTAATCAAAGAAATTAGGTGCTCCTACGCCTGAAGTTATCGGTGAAGGCTTCCCCTCCATTTGATCTGTAGGATATCGAGTTTTCTTACCGCCTCTATCGGCTATGGGATATGCAATTCTCTTCTCTGACTTAACACAGAGCAAAGTAGACTGATTTCGCGCTAGTGCTAGTACACGAGTAGACCGCTTTCACCTAGCTATTGCTCACTAACAGAACCTTCTCGTACTGGAGAAAAGAACTTGAGCTCTGCTTCGAGGAACTAGCAGTCGAAGGGTGACGATTTCTGATCACTGGATTCAAGAGCTTTTAGGGTGTTCGGAACAGTTATTAGTAGAAGATAAGACTTTCTCGGCTTGTTTACTAAGTCTCTGATTCGAATAAGCGACTCGGAACTCTGTTCGCGGTTAGCTGAGAATGTTCTTGCTTCTTGCCAGTTAGATTAGCTTGAAAGGGAATGAGTGAGTCGAAAGTATGACAACGGGCATAGATAGAGGAGTTCCTGATCCCGGTACTAGGGCGAATGGCATGGGTGCCCAAAAGCCCGGTTTGTTAGCCCCTTTCCGATTCCCTCACCCAATCTCATTAAAAGCAAGCCCAGCAGGCCCTGCCTTAACCTGTCCCCAGACAGCCAGCCCTCACCAGGCCGCTGGCATTACTCAATGCTCCGCCACGAAGCAAGCTCTCCCGAATACGACAGATGCGGAAGTGGCTCAAGAAGTCGGAGGAATCAAGTAGTTGGAAAACTGTATGCACGAGGGGACATGAGTCTTCTGGGAATTGGCAACATTGGCAGAAAATAAAGGGGAATACAAGGGGGGTAGGAATCCACTTTGTTAGGTGTAGCTATACTCACGTAAGTAGTCGGCCTAACCTTCGGTTCCCGTAACCAAGTTGTTCTTTCTCACTCCTGATTGACTTTTGATTACTGAATCCATACTTTTTTACTTTTTTTGAATTTGAAGTGTGGGGAAAAGGGCGCCCTCTACTTCTACTTCTAACTACAGGCGAAAAGCTGGCATTGCAAGCAAATAGAGAGCCCCCGCCCGTTTGAGTCGTTGCGAGCCGGAAAGCGTACCAGCGGTTTGAGTCGCGAAAGGGCCGCTTGCTTAATTATATTATAATATAATATATATAATCTTCTATCTCTATCTATCAACAATAAAATCAGAAGAAAGTAAAAAAAATATAAAAAAGAAAATCATTTTTTGTATCCAATTTTGCATTCCTGGGAAGAGGAAGAAGCAGATAGAGCAAAGGCCTCCTCTTTCCGTCCGCTCTTCCCGAAGTGAGCGAATTGCATGTAGAGATCCGTAGGGGCTTATAGTTTAATTGGTTGAAACGTACCGCTCATAACGGTGATATTGTAGGTTCGAGCCCTACTAAGCCTACCACCCCCTTCTCTTCACCCGATACAAGGCAGTCGAAGTCCCCGCCACCCTGCAGATCTCAATCTAGCGACGGCACCTAGAACCACACTGCTGCCGCTGCCCGAAGGGCACGCCTCCTACGCTCTTGCAGCATGCCCCCTTCGGGGCAGATGTTACTATACTAAAAAAGAAGGCCCTCGCTAAGCGCTGGTTCTATCCCGGCCAAGCAACCAAGGTGGGAATAGTGAACGAAAGAGAAGGACATTGTTCAGAGTGAAACTGAACCCCCTTGATCAATCCTGTAAGAACGAAGACTTCACCAATCGACCAATGGGCCTTTCCTTGTAGGCGGCGAAGGGCAGGTGAACACTCTTCCCTTGGAACCTGCGCATAATGAAATATAACATTTTTTACTTCCATGGTCATACTATATTTATCTTTATATTGCGAATGAGTCTGGACCATCTCCTATTGTAGTATCAAAATGAATATGACTTTACTTTGAAGTTTAGCCCTCTTTCAACAGTATGACAACCTTCCCAATCACTCGGTTCAATCCTTACCTGAGGATGACAAGGCTTGGCTGATAGGCCGAGGACGAAGCATGCACCTTCGCTTAACCTTCGATGTTGTCATCACGCTGCCTTTCGCATGTCGGGCCTATACACGCTCGTCAAGTTACACCTTAACTGCTTTCTCAACGCGCGGCTCTTATAGATAACCCTTCCTTATCAATCAAATAAGCATTTGTGAGTTGAGATTCCTTCCCTATGTTTCAAAGCTAGCTTCTCTAGCTATACTGTGTGACCCACCTCCTCCCTTCGCTCTCTTGCTAGAGCTGGTTCTAAGCCTACCTTTTCCTTCCCCACCCTCTGTCTGACCAACCTCTGACTTCGATAATGACCTATAAACAATTATTCCAAGTGAAACGATAAGCATGTTGGCTTAGACCTTATCTTACCAGACAAGACATTGATGTCTGCCCGAACATCGAAATGAATCTTTCATATGCGGATTTTCACTTCCCCTCATTAAGCTAGCTAGCGCAAGTGTCAGAAAGGATGAATGCATTCCGAGATCGAATTAGCCCCTTACATCTTAGAACATTTTATACAAGGAATGTGTGTGACCTCTCTGATAGTAAGAGCGCACTAACGGAAAGAGAAATTGTTATAAGGAAAGAGAACACACTAGGGTAGAGTGGAAAGGGAACAGGAAAAAACTTTAGTCGACTAACTCTAGTATTCATGCTAGAACAGAGCTCCCGAAATAAAATATATCAATTATAGCGCTTCATCACTTGAAATAGGATCTTGCCTACGGCCCTAGTACTTTACTTTAGTCGATCGACATCTCATTAGCAAACAAACATAGAAGAGTCAGCTTCCTCAGTCTTGGTTATCGAGTTATCTTACCTGACAGGGTCGGCTAGGTGAGTTTGATTCCATTCCCGTGGCAAAAGGAAAAGAGCTTGATATCCGGGCTTCTATCGGTGAAGAAATGTTATGCCCACGGTTCCGTACTAAAGAATGAGCCAACAGCTATCTCCTTAGCTTCTTAAGGCACTCTTTTTCTGTTTAGTTATTGGTAATCCATCCGAGTGATCTATCTTATCGATAAAGAAATTCTCTCCCCTTACCGATCTTGTTATGCCTCCCGCGGTACATACAAAGGAACCTTCTATCCCATCGGTTAATCAAAGAAATTAGGTGCTCCTACGCCTGAAGTTATCGGTGAAGGCTTCCCCTCCATTTGATCTGTAGGATATCGAGTTTTCTTACCGCCTCTATCGGCTATGGGATATGCAATTCTCTTCTCTGACTTAACACAGAGCAAAGTAGACTGATTTCGCGCTAGTGCTAGTACACGAGTAGACCGCTTTCACCTAGCTATTGCTCACTAACAGAACCTTCTCGTACTGGAGAAAAGAACTTGAGCTCTGCTTCGAGGAACTAGCAGTCGAAGGGTGACGATTTCTGATCACTGGATTCAAGAGCTTTTAGGGTGTTCGGAACAGTTATTAGTAGAAGATAAGACTTTCTCGGCTTGTTTACTAAGTCTCTGATTCGAATAAGCGACTCGGAACTCTGTTCGCGGTTAGCTGAGAATGTTCTTGCTTCTTGCCAGTTAGATTAGCTTGAAAGGGAATGAGTGAGTCGAAAGTATGACAACGGGCATAGATAGAGGAGTTCCTGATCCCGGTACTAGGGCGAATGGCATAACTGCTTCTTTCTCTTTTTACGGGTAGAATCCGCTATAGTTGAGGAAGCCCAGAGATGAGGATAAAATCTCTTGTTTAAGAAGCAACTCATGTTTCAGGGGGTGCCCAAAAGCCCGGTTTGTTAGCCCCTTTCCGATTCCCTCACCCAATCTCATTAAAAGCAAGCCCAGCAGGCCCTGCCTTAACCTGTCCCCAGACAGCCAGCCCTCACCAGGCCGCTGGCATTACTCAATGCTCCGCCACGAAGCAAGCTCTCCCGAATACGACAGATGCGGAAGTGGCTCAAGAAGTCGGAGGAATCAAGTAGTTGGAAAACTGTATGCACGAGGGGACATGAGTCTTCTGGGAATTGGCAACATTGGCAGAAAATAAAGGGGAATACAAGGGGGGTAGGAATCCACTTTGTTAGGTGTAGCTATACTCACGTAAGTAGTCGGCCTAACCTTCGGTTCCCGTAACCAAGTTGTTCTTTCTCACTCCTGATTGACTTTTGATTACTGAATCCATACTTTTTTACTTTTTTTGAATTTGAAGTGTGGGGAAAAGGGCGCCCTCTACTTCTACTTCTAACTACAGGCGAAAAGCTGGCATTGCAAGCAAATAGAGAGCCCCCGCCCGTTTGAGTCGTTGCGAGCCGGAAAGCGTACCAGCGGTTTGAGTCGCGAAAGGGCCGCTTGCTTAATTATATTATAATATAATATATATAATCTTCTATCTCTATCTATCAACAATAAAATCAGAAGAAAGTAAAAAAAATATAAAAAAGAAAATCATTTTTTGTATCCAATTTTGCATTCCTGGGAAGAGGAAGAAGCAGATAGAGCAAAGGCCTCCTCTTTCCGTCCGCTCTTCCCGAAGTGAGCGAATTGCATGTAGAGATCCGTAGGGGCTTATAGTTTAATTGGTTGAAACGTACCGCTCATAACGGTGATATTGTAGGTTCGAGCCCTACTAAGCCTACCACCCCCTTCTCTTCACCCGATACAAGGCAGTCGAAGTCCCCGCCACCCTGCAGATCTCAATCTAGCGACGGCACCTAGAACCACACTGCTGCCGCTGCCCGAAGGGCACGCCTCCTACGCTCTTGCAGCATGCCCCCTTCGGGGCAGATGTTACTATACTAAAAAAGAAGGCCCTCGCTAAGCGCTGGTTCTATCCCGGCCAAGCAACCAAGGTGGGAATAGTGAACGAAAGAGAAGGACATTGTTCAGAGTGAAACTGAACCCCCTTGATCAATCCTGTAAGAACGAAGACTTCACCAATCGACCAATGGGCCTTTCCTTGTAGGCGGCGAAGGGCAGGTGAACACTCTTCCCTTGGAACCTGCGCATAATGAAATATAACATTTTTTACTTCCATGGTCATACTATATTTATCTTTATATTGCGAATGAGTCTGGACCATCTCCTATTGTAGTATCAAAATGAATATGACTTTACTTTGAAGTTTAGCCCTCTTTCAACAGTATGACAACCTTCCCAATCACTCGGTTCAATCCTTACCTGAGGATGACAAGGCTTGGCTGATAGGCCGAGGACGAAGCATGCACCTTCGCTTAACCTTCGATGTTGTCATCACGCTGCCTTTCGCATGTCGGGCCTATACACGCTCGTCAAGTTACACCTTAACTGCTTTCTCAACGCGCGGCTCTTATAGATAACCCTTCCTTATCAATCAAATAAGCATTTGTGAGTTGAGATTCCTTCCCTATGTTTCAAAGCTAGCTTCTCTAGCTATACTGTGTGACCCACCTCCTCCCTTCGCTCTCTTGCTAGAGCTGGTTCTAAGCCTACCTTTTCCTTCCCCACCCTCTGTCTGACCAACCTCTGACTTCGATAATGACCTATAAACAATTATTCCAAGTGAAACGATAAGCATGTTGGCTTAGACCTTATCTTACCAGACAAGACATTGATGTCTGCCCGAACATCGAAATGAATCTTTCATATGCGGATTTTCACTTCCCCTCATTAAGCTAGCTAGCGCAAGTGTCAGAAAGGATGAATGCATTCCGAGATCGAATTAGCCCCTTACATCTTAGAACATTTTATACAAGGAATGTGTGTGACCTCTCTGATAGTAAGAGCGCACTAACGGAAAGAGAAATTGTTATAAGGAAAGAGAACACACTAGGGTAGAGTGGAAAGGGAACAGGAAAAAACTTTAGTCGACTAACTCTAGTATTCATGCTAGAACAGAGCTCCCGAAATAAAATATATCAATTATAGCGCTTCATCACTTGAAATAGGATCTTGCCTACGGCCCTAGTACTTTACTTTAGTCGATCGACATCTCATTAGCAAACAAACATAGAAGAGTCAGCTTCCTCAGTCTTGGTTATCGAGTTATCTTACCTGACAGGGTCGGCTAGGTGAGTTTGATTCCATTCCCGTGGCAAAAGGAAAAGAGCTTGATATCCGGGCTTCTATCGGTGAAGAAATGTTATGCCCACGGTTCCGTACTAAAGAATGAGCCAACAGCTATCTCCTTAGCTTCTTAAGGCACTCTTTTTCTGTTTAGTTATTGGTAATCCATCCGAGTGATCTATCTTATCGATAAAGAAATTCTCTCCCCTTACCGATCTTGTTATGCCTCCCGCGGTACATACAAAGGAACCTTCTATCCCATCGGTTAATCAAAGAAATTAGGTGCTCCTACGCCTGAAGTTATCGGTGAAGGCTTCCCCTCCATTTGATCTGTAGGATATCGAGTTTTCTTACCGCCTCTATCGGCTATGGGATATGCAATTCTCTTCTCTGACTTAACACAGAGCAAAGTAGACTGATTTCGCGCTAGTGCTAGTACACGAGTAGACCGCTTTCACCTAGCTATTGCTCACTAACAGAACCTTCTCGTACTGGAGAAAAGAACTTGAGCTCTGCTTCGAGGAACTAGCAGTCGAAGGGTGACGATTTCTGATCACTGGATTCAAGAGCTTTTAGGGTGTTCGGAACAGTTATTAGTAGAAGATAAGACTTTCTCGGCTTGTTTACTAAGTCTCTGATTCGAATAAGCGACTCGGAACTCTGTTCGCGGTTAGCTGAGAATGTTCTTGCTTCTTGCCAGTTAGATTAGCTTGAAAGGGAATGAGTGAGTCGAAAGTATGACAACGGGCATAGATAGAGGAGTTCCTGATCCCGGTACTAGGGCGAATGGCATAACTGCTTCTTTCTCTTTTTACGGGTAGAATCCGCTATAGTTGAGGAAGCCCAGAGATGAGGATAAAATCTCTTGTTTAAGAAGCAACTCATGTTTCAGGGGGTGCCCAAAAGCCCGGTTTGTTAGCCCCTTTCCGATTCCCTCACCCAATCTCATTAAAAGCAAGCCCAGCAGGCCCTGCCTTAACCTGTCCCCAGACAGCCAGCCCTCACCAGGCCGCTGGCATTACTCAATGCTCCGCCACGAAGCAAGCTCTCCCGAATACGACAGATGCGGAAGTGGCTCAAGAAGTCGGAGGAATCAAGTAGTTGGAAAACTGTATGCACGAGGGGACATGAGTCTTCTGGGAATTGGCAACATTGGCAGAAAATAAAGGGGAATACAAGGGGGGTAGGAATCCACTTTGTTAGGTGTAGCTATACTCACGTAAGTAGTCGGCCTAACCTTCGGTTCCCGTAACCAAGTTGTTCTTTCTCACTCCTGATTGACTTTTGATTACTGAATCCATACTTTTTTACTTTTTTTGAATTTGAAGTGTGGGGAAAAGGGCGCCCTCTACTTCTACTTCTAACTACAGGCGAAAAGCTGGCATTGCAAGCAAATAGAGAGCCCCCGCCCGTTTGAGTCGTTGCGAGCCGGAAAGCGTACCAGCGGTTTGAGTCGCGAAAGGGCCGCTTGCTTAATTATATTATAATATAATATATATAATCTTCTATCTCTATCTATCAACAATAAAATCAGAAGAAAGTAAAAAAAATATAAAAAAGAAAATCATTTTTTGTATCCAATTTTGCATTCCTGGGAAGAGGAAGAAGCAGATAGAGCAAAGGCCTCCTCTTTCCGTCCGCTCTTCCCGAAGTGAGCGAATTGCATGTAGAGATCCGTAGGGGCTTATAGTTTAATTGGTTGAAACGTACCGCTCATAACGGTGATATTGTAGGTTCGAGCCCTACTAAGCCTACCACCCCCTTCTCTTCACCCGATACAAGGCAGTCGAAGTCCCCGCCACCCTGCAGATCTCAATCTAGCGACGGCACCTAGAACCACACTGCTGCCGCTGCCCGAAGGGCACGCCTCCTACGCTCTTGCAGCATGCCCCCTTCGGGGCAGATGTTACTATACTAAAAAAGAAGGCCCTCGCTAAGCGCTGGTTCTATCCCGGCCAAGCAACCAAGGTGGGAATAGTGAACGAAAGAGAAGGACATTGTTCAGAGTGAAACTGAACCCCCTTGATCAATCCTGTAAGAACGAAGACTTCACCAATCGACCAATGGGCCTTTCCTTGTAGGCGGCGAAGGGCAGGTGAACACTCTTCCCTTGGAACCTGCGCATAATGAAATATAACATTTTTTACTTCCATGGTCATACTATATTTATCTTTATATTGCGAATGAGTCTGGACCATCTCCTATTGTAGTATCAAAATGAAT

Transforming genomes of information into knowledge

Biologist

IS5gnd wbbK wbbJ

wbbL

~1kb

Lifecycle of Data

Transfer Storage Analysis Visualization Metadata Mark-up Search and Discover Share/Collaborate Publish

EndUsers

ComputationalUsers

iPlant Layered Services and Access

iPlant Data StoreScalableReliable

RedundantHigh-Performance

iPlant Data StoreFree Your Data

Different Users, Different Access Needs:

One Data Store

iPlant Data Store (iDS)WebDAV DE

i-commands iDrop

API

iPlant Data StoreWeb-Integrated High Performance Big Data Transfers

iPlant Data Store: MetadataData About Data

Preview of Next Version of DE

iPlant Data Store: MetadataData About Data

More Data; Smarter Data

iPlant Data Store

Texas

Replication

Arizona

Grid Computing

Cloud Computing HPCCommunity

Super Computing

iDrop

WebDAV

FoundationAPI

DE

i-commands

iPlant Data StoreScalableReliable

RedundantHigh-Performance

Connected

The journey of data to iDSand challenges along the way !

Hard Drive Network card

Building network

Campus network Internet

InternetUA/

TACC network

iDS Network card

Hard Drive

http://en.wikipedia.org/wiki/List_of_device_bandwidthsCheck: USB, HDD, Network capabilities

iPlant Data Store PerformanceUC Berkeley to iDS

• Dec 5th, 2011: • 100GB: 29m15s

36,000 Students 2000 Faculty

39,000 Students 2900 Faculty/Staff

100GB: 29m15s

iPlant Data Store PerformanceUC Berkeley to iDS

Source Destination Copy Method Time (seconds)

CD Desktop PC cp 320

Berkeley Server Desktop PC scp 150

External Drive Desktop PC cp 36

USB 2.0 Flash Desktop PC cp 30

iDS Desktop PC iget 18

Desktop PC Desktop PC cp 15

https://pods.iplantcollaborative.org/wiki/display/start/How+fast+is+the+iPlant+Data+Store

1 GB / 17.5 seconds

Desktop PC (UA): Mac OS X with 7.2K Internal Hard DriveExternal Drive: USB 2.0: 5.4k Hard DriveFlash Drive: USB 2.0 Patriot XT

iPlant Data StoreConnecting people with data and computation:

Lifecycle of Data

Transfer Storage Analysis Visualization Metadata Mark-up Search and Discover Share/Collaborate Publish

Cyberinfrastructure for Life SciencesScalableCapable

Extensible

Where to Get Information

• Data Store Quick Start: • http://www.iplantcollaborative.org/Zki

• Data Store Manual:• http://www.iplantcollaborative.org/Zko

• iPlant Forums:• http://forums.iplantcollaborative.org

Exercise Your Data Storehttps://pods.iplantcollaborative.org/wiki/x/e4hy

• Go to http://data.iplantcollaborative.org• Login and click on “home”

icon• Upload any file you like

(small)• Explore menu to the right• Now download idrop.jar (find

it from the wiki)• Drag and drop folders