Amazon resource for bioinformatics
-
Upload
brad-chapman -
Category
Documents
-
view
682 -
download
6
description
Transcript of Amazon resource for bioinformatics
![Page 1: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/1.jpg)
Amazon resources for bioinformatics
Brad Chapman
Bioinformatics Interest Group, 18 Oct 2012
![Page 2: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/2.jpg)
Goals
Automate:Reduce stepsRemove activation energyIncrease abstraction
Improve:SharingReproducibilityTeaching
![Page 3: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/3.jpg)
Installation
![Page 4: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/4.jpg)
Easier installation
![Page 5: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/5.jpg)
No installation
![Page 6: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/6.jpg)
Challenge
Biology computing platform
Widely accessible
Customizable
Community driven
![Page 8: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/8.jpg)
Not only Amazon
http://gigaom.com/cloud/what-google-compute-engine-means-for-cloud-computing/
![Page 9: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/9.jpg)
CloudBioLinux
Amazon image with bioinformatics software andlibraries
Automated build framework
Community e�ort to maintain and extend
http://cloudbiolinux.org
![Page 10: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/10.jpg)
CloudMan
SGE cluster plus automation
Web interface and monitoring
Persistence and sharing
Powers the Galaxy Cloud o�ering
http://usecloudman.org/
![Page 11: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/11.jpg)
BioCloudCentral
Automate setup of Amazon instance
Launch CloudBioLinux and CloudMan
Provide easy ssh access, no key pairs
http://biocloudcentral.org
![Page 13: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/13.jpg)
Acknowledgments
CloudBioLinux: Ntino Krampis, Tim Booth,Dawn Field, Pjotr Prins, John Chilton andCloudBioLinux community.
CloudMan: Enis Afgan, James Taylor
BioCloudCentral: Enis Afgan, John Chilton,Dannon Baker
![Page 14: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/14.jpg)
Documentation
http://cda.currentprotocols.com/WileyCDA/CPUnit/
refId-bi1109.html
![Page 15: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/15.jpg)
What we'll do
1 Sign up for Amazon
2 Start a CloudBioLinux/CloudMan instance
3 Add nodes to create a compute cluster
4 Run variant calling pipeline
Everything done through the web
![Page 16: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/16.jpg)
Getting started
Sign up for Amazon Web Serviceshttp://aws.amzaon.com
Get security credentials: Access Key and Secret Keyhttp://portal.aws.amazon.com/gp/aws/
securityCredentials
![Page 18: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/18.jpg)
Ready two minutes later
![Page 19: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/19.jpg)
Login to CloudMan
![Page 20: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/20.jpg)
Shared CloudMan images
Package a complete analysis environmentDataCustomizations
Sharable with other users
Share string with NGS analysis platform:
cm-b53c6f1223f966914df347687f6fc818/shared/2012-07-23--19-23/
![Page 21: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/21.jpg)
Start CloudMan
![Page 22: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/22.jpg)
CloudMan console
![Page 23: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/23.jpg)
CloudMan admin page
![Page 24: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/24.jpg)
CloudMan: managing a cluster
![Page 25: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/25.jpg)
Associated Galaxy instance
![Page 26: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/26.jpg)
Analysis data on shared instance
![Page 27: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/27.jpg)
Graphical variant-calling pipeline
![Page 28: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/28.jpg)
Analysis data linked to pipeline
![Page 29: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/29.jpg)
Con�gure pipeline
![Page 30: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/30.jpg)
Run pipeline
![Page 31: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/31.jpg)
Shut everything down
![Page 32: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/32.jpg)
What happened
1 Sign up for Amazon
2 Start a CloudBioLinux/CloudMan instance
3 Add nodes to create a compute cluster
4 Run variant calling pipeline
Everything done through the web
![Page 33: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/33.jpg)
ssh to the machine
$ ssh [email protected]
[email protected]'s password:
Welcome to Ubuntu 12.04 LTS
(GNU/Linux 3.2.0-23-virtual x86_64)
ubuntu@ip-10-72-197-11:~$
![Page 34: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/34.jpg)
NX graphical client: login
http://www.nomachine.com/download.php
![Page 35: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/35.jpg)
NX graphical client: desktop
![Page 36: Amazon resource for bioinformatics](https://reader030.fdocuments.in/reader030/viewer/2022020101/555012bcb4c90555618b4b0e/html5/thumbnails/36.jpg)
Summary
Use cloud resources to build:
Machines with standard software
Cluster management
Analysis pipelines
Reproducible, sharable instances
Web-based interfaces