Talk at Microsoft Cloud Futures 2010
-
Upload
deepak-singh -
Category
Technology
-
view
3.556 -
download
0
description
Transcript of Talk at Microsoft Cloud Futures 2010
![Page 1: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/1.jpg)
Scien&fic Compu&ng with Amazon Web ServicesDeepak Singh
Cloud Futures 2010
![Page 2: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/2.jpg)
![Page 3: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/3.jpg)
![Page 4: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/4.jpg)
Via Reavel under a CC-BY-NC-ND license
![Page 5: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/5.jpg)
![Page 6: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/6.jpg)
![Page 7: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/7.jpg)
life science industry
![Page 8: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/8.jpg)
Credit: Bosco Ho
![Page 9: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/9.jpg)
![Page 10: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/10.jpg)
By ~Prescott under a CC-BY-NC license
![Page 11: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/11.jpg)
![Page 12: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/12.jpg)
data
![Page 13: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/13.jpg)
Image: Wikipedia
![Page 14: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/14.jpg)
![Page 15: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/15.jpg)
Image via image editor under a CC-‐BY License
![Page 16: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/16.jpg)
Image: Matt Wood
![Page 17: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/17.jpg)
Image: NOAA
![Page 18: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/18.jpg)
![Page 19: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/19.jpg)
![Page 20: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/20.jpg)
![Page 21: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/21.jpg)
![Page 22: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/22.jpg)
gigabytes
![Page 23: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/23.jpg)
terabytes
![Page 24: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/24.jpg)
petabytes
![Page 25: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/25.jpg)
petabytes
![Page 26: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/26.jpg)
petabytes
exabytes?
![Page 27: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/27.jpg)
really fast
![Page 28: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/28.jpg)
Image: http://www.broadinstitute.org/~apleite/photos.html
![Page 29: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/29.jpg)
![Page 30: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/30.jpg)
![Page 31: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/31.jpg)
![Page 32: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/32.jpg)
data management
![Page 33: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/33.jpg)
data processing
![Page 34: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/34.jpg)
data sharing
![Page 35: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/35.jpg)
Image: Chris Dagdigian
![Page 36: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/36.jpg)
compute & storage limited
![Page 37: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/37.jpg)
![Page 38: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/38.jpg)
amazon web services
![Page 39: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/39.jpg)
the cloud
![Page 40: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/40.jpg)
has_many :definitions
![Page 41: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/41.jpg)
infrastructure as a service
![Page 42: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/42.jpg)
![Page 43: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/43.jpg)
ComputeAmazon Elastic Compute
Cloud (EC2)- Elastic Load Balancing- Auto Scaling
StorageAmazon Simple
Storage Service (S3)- AWS Import/Export
DatabaseAmazon RDS and
SimpleDB
![Page 44: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/44.jpg)
ComputeAmazon Elastic Compute
Cloud (EC2)- Elastic Load Balancing- Auto Scaling
StorageAmazon Simple
Storage Service (S3)- AWS Import/Export
Content DeliveryAmazon CloudFront
MessagingAmazon Simple
Queue Service (SQS) Amazon Simple
Notification Service (SNS)
PaymentsAmazon Flexible Payments Service
(FPS)
On-Demand Workforce
Amazon Mechanical Turk
Parallel ProcessingAmazon Elastic
MapReduce
DatabaseAmazon RDS and
SimpleDB
![Page 45: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/45.jpg)
ComputeAmazon Elastic Compute
Cloud (EC2)- Elastic Load Balancing- Auto Scaling
StorageAmazon Simple
Storage Service (S3)- AWS Import/Export
Content DeliveryAmazon CloudFront
MessagingAmazon Simple
Queue Service (SQS)Amazon Simple
Notification Service (SNS)
PaymentsAmazon Flexible Payments Service
(FPS)
On-Demand Workforce
Amazon Mechanical Turk
Parallel ProcessingAmazon Elastic
MapReduce
MonitoringAmazon CloudWatch
ManagementAWS Management Console
ToolsAWS Toolkit for Eclipse
AWS SDK for .NET
Isolated NetworksAmazon Virtual Private
Cloud
DatabaseAmazon RDS and
SimpleDB
![Page 46: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/46.jpg)
ComputeAmazon Elastic Compute
Cloud (EC2)- Elastic Load Balancing- Auto Scaling
StorageAmazon Simple
Storage Service (S3)- AWS Import/Export
Your Custom Applications and Services
Content DeliveryAmazon CloudFront
MessagingAmazon Simple
Queue Service (SQS)Amazon Simple
Notification Service (SNS)
PaymentsAmazon Flexible Payments Service
(FPS)
On-Demand Workforce
Amazon Mechanical Turk
Parallel ProcessingAmazon Elastic
MapReduce
MonitoringAmazon CloudWatch
ManagementAWS Management Console
ToolsAWS Toolkit for Eclipse
AWS SDK for .NET
Isolated NetworksAmazon Virtual Private
Cloud
DatabaseAmazon RDS and
SimpleDB
![Page 47: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/47.jpg)
![Page 48: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/48.jpg)
![Page 49: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/49.jpg)
![Page 50: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/50.jpg)
![Page 51: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/51.jpg)
elasticity
![Page 52: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/52.jpg)
3000 CPU’s for one firm’s risk management application
!"#$%&'()'*+,'-./01.2%/'
344'+567/'(.'
8%%9%.:/'
;<"&/:1='
>?,3?,44@'
A&B:1='
>?,>?,44@'
C".:1='
>?,D?,44@'
E(.:1='
>?,F?,44@'
;"%/:1='
>?,G?,44@'
C10"&:1='
>?,H?,44@'
I%:.%/:1='
>?,,?,44@'
3444JJ'
344'JJ'
![Page 53: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/53.jpg)
scalability
![Page 54: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/54.jpg)
> 1PB of data in S3
![Page 55: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/55.jpg)
![Page 56: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/56.jpg)
highly available
![Page 57: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/57.jpg)
Image: Chris Dagdigian
![Page 58: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/58.jpg)
![Page 59: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/59.jpg)
“Everything fails, all the time”-- Werner Vogels
![Page 60: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/60.jpg)
![Page 61: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/61.jpg)
“Things will crash. Deal with it”-- Jeff Dean
![Page 62: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/62.jpg)
2-4% of serverswill die annually
Source: Jeff Dean, LADIS 2009
![Page 63: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/63.jpg)
1-5% of disk drives will die every year
Source: Jeff Dean, LADIS 2009
![Page 64: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/64.jpg)
human errors
![Page 65: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/65.jpg)
human errors~20% admin issues have unintended consequences
Source: James Hamilton
![Page 66: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/66.jpg)
scalable & available
![Page 67: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/67.jpg)
assume sw/hw failure
![Page 68: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/68.jpg)
design apps to be resilient
![Page 69: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/69.jpg)
automation & alarming
![Page 70: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/70.jpg)
US East Region
Availability Zone A
Availability Zone B
Availability Zone C
Availability Zone D
!"#$%&'()*+
T
TT
![Page 71: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/71.jpg)
![Page 72: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/72.jpg)
elastic load balancing
CloudWatch
auto scaling
elastic block store
elastic IP
SQSSNS
![Page 73: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/73.jpg)
cost effective
![Page 74: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/74.jpg)
cost effective
pay as you go
![Page 75: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/75.jpg)
on-demand instancesreserved instances
spot instances
![Page 76: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/76.jpg)
![Page 77: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/77.jpg)
Your Network
AmazonWeb Services
CloudSecure VPN Connection
over the Internet
Customer’s isolated AWS
resources
VPN Gateway
ExternalCustomers
Subnets
10.32.1.0/24
10.32.2.0/24
10.32.3.0/24
AMAZON VPC ARCHITECTURE
![Page 78: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/78.jpg)
AWS + science = win
![Page 79: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/79.jpg)
![Page 80: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/80.jpg)
3.7 million classifications in just over three days~15 million in less than a month>2.6 million clicks in 100 hours
![Page 81: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/81.jpg)
![Page 82: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/82.jpg)
lots and lots and lots and lots and lots and lots of data andlots and lots of lots of data
![Page 83: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/83.jpg)
scalability & availability
![Page 84: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/84.jpg)
we are data geeks not data center geeks
![Page 85: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/85.jpg)
data management
![Page 86: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/86.jpg)
Shaq Image: Keith Allison under a CC-BY-SA license
![Page 87: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/87.jpg)
Shaq Image: Keith Allison under a CC-BY-SA license
![Page 88: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/88.jpg)
Shaq Image: Keith Allison under a CC-BY-SA license
![Page 89: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/89.jpg)
Shaq Image: Keith Allison under a CC-BY-SA license
![Page 90: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/90.jpg)
Shaq Image: Keith Allison under a CC-BY-SA license
![Page 91: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/91.jpg)
![Page 92: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/92.jpg)
![Page 93: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/93.jpg)
Biomarker Warehousepre-clinical, clinical, 3rd party data and publications
!"#$%"&&'
!#%&$(%&&&'
!)*(%"&&'
+,'-./01'
23,3415'61789:1'
;<./5'=>?6@'
6178170' 6A.7341' B817-135'
Estimated cost: 10 TB warehouse over 3 years
![Page 94: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/94.jpg)
data processing
![Page 95: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/95.jpg)
![Page 97: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/97.jpg)
http://web.mit.edu/stardev/cluster/
![Page 98: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/98.jpg)
http://cyclecomputing.comhttp://wiki.github.com/documentcloud/cloud-crowd
sudo gem install cloud-crowd
![Page 100: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/100.jpg)
http://leonardo.phys.washington.edu/feff/
XAFS
![Page 101: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/101.jpg)
http://bioteam.net
![Page 102: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/102.jpg)
ASSEMBLING GENOMES
140 million 454 reads
Image: Ma) Wood
![Page 103: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/103.jpg)
Map 100 million, 100 base paired end readsQuad core with 5 GB of RAM would take 16 days
30 high-memory instances; 32 hours; $195
BLAT @ U. PENN
![Page 104: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/104.jpg)
HEAVY-ION COLLISIONS
Problem: Quark matter physics conference imminent but no compute resources handy
Solution: NIMBUS context broker allowed researchers to provision 300 nodes and get the simulations done
![Page 105: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/105.jpg)
BELLE MONTE CARLO
Credit: Tom Fifield
![Page 106: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/106.jpg)
disk read/writesslow & expensive
![Page 107: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/107.jpg)
data processingfast & cheap
![Page 108: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/108.jpg)
distribute the dataparallel reads
![Page 109: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/109.jpg)
![Page 110: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/110.jpg)
data processing for the cloud
![Page 111: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/111.jpg)
distributed file system(HDFS)
![Page 112: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/112.jpg)
map/reduce
![Page 113: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/113.jpg)
Via Cloudera under a Creative Commons License
![Page 114: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/114.jpg)
Via Cloudera under a Creative Commons License
![Page 116: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/116.jpg)
apache pig
http://hadoop.apache.org/pig/
![Page 117: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/117.jpg)
apache hive
http://hadoop.apache.org/hive/
![Page 118: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/118.jpg)
work by @peteskomoroch
![Page 119: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/119.jpg)
![Page 120: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/120.jpg)
High Throughput Sequence AnalysisMike Schatz, University of Maryland
![Page 121: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/121.jpg)
CloudBurst
Catalog k-mers Collect seeds End-to-end alignment
http://cloudburst-bio.sourceforge.net; Bioinformatics 2009 25: 1363-1369
![Page 122: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/122.jpg)
Crossbow: Rapid whole genome SNP analysis
Ben Langmead
http://bowtie-bio.sourceforge.net/crossbow/index.shtml
![Page 123: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/123.jpg)
Preprocessed reads
Map: Bowtie
Sort: Bin and partition
Reduce: SoapSNP
Crossbow: Rapid whole genome SNP analysis
Langmead B, Schatz MC, Lin, J, Pop M, Salzberg SL. Genome Biol 10(11): R134.
![Page 124: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/124.jpg)
Crossbow condenses over 1,000 hours of resequencing computa:on into a few hours without requiring the user to own or operate a computer cluster
![Page 125: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/125.jpg)
Assembly of Large Genomes with Cloud Computing.Schatz MC, Sommer D, Kelley D, Pop M, et al. In Preparation.http://contrail-bio.sourceforge.net
Scalable Genome Assembly
![Page 126: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/126.jpg)
Input S3 bucket
Output S3 bucket
Amazon S3
Hadoop
Amazon EC2 Instances
Input dataset
outputresults
Deploy Application
Web Console, Command line tools
End
Notify
Get ResultsInput Data
Amazon Elastic MapReduce
Hadoop Hadoop
Hadoop
Hadoop
Hadoop
Elastic MapReduce
Elastic MapReduce
![Page 127: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/127.jpg)
![Page 128: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/128.jpg)
data storage & distributionpublic & private
![Page 129: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/129.jpg)
http://aws.amazon.com/publicdatasets/
![Page 130: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/130.jpg)
![Page 131: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/131.jpg)
sharing and collaboration
![Page 132: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/132.jpg)
software distribution
![Page 133: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/133.jpg)
![Page 136: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/136.jpg)
application platforms
![Page 139: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/139.jpg)
Image: O’Reilly Radar
![Page 140: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/140.jpg)
business models
![Page 141: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/141.jpg)
![Page 142: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/142.jpg)
![Page 143: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/143.jpg)
to conclude
![Page 144: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/144.jpg)
![Page 145: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/145.jpg)
built for scale
![Page 146: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/146.jpg)
built for availability
![Page 147: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/147.jpg)
shared dataspacesglobal namespaces
![Page 148: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/148.jpg)
task-based resources
![Page 149: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/149.jpg)
new software architectures
![Page 150: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/150.jpg)
new computing platforms
![Page 151: Talk at Microsoft Cloud Futures 2010](https://reader038.fdocuments.in/reader038/viewer/2022103114/555012aeb4c905af648b49d4/html5/thumbnails/151.jpg)
available today