AWS support for academic research - Kent State University · Harvard University uses AWS to develop...

30
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Irena Zlatanovic, Sr. Program Manager, Cloud Credits for Research Program March 22, 2018 AWS support for academic research

Transcript of AWS support for academic research - Kent State University · Harvard University uses AWS to develop...

Page 1: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Irena Zlatanovic, Sr. Program Manager, Cloud Credits for Research Program

March 22, 2018

AWS support for academic research

Page 2: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Supporting the mission of higher education

Discovery Education

Open Data

Scientific Computing

AWS Academy

Amazon Research Awards

AWS Machine Learning Research Awards

Earth on AWS

Page 3: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

AWS programs supporting research

Cloud Credits for Research

Research and Technical Computing

Public Datasets

• AWS hosts large and globally significant datasets for community use.

• Developing solutions to enable researchers to use AWS to host their own datasets.

• AWS ”grant” program to support individual research projects to faculty, staff, and students.

• Dedicated team of scientific experts focusing on scientific computing and research workloads.

• Developing solutions to enable researchers to benefit from AWS.

Page 4: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

AWS Research Cloud Program

Science first, not servers. Simple start up procedures.

Budget management tools.

Large catalog of researchSolutions.

Solutions to procurement challenges.

Best practices for data security.

http://aws.amazon.com/rcp

Page 5: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Solving Procurement Challenges

Invoice-backed billingmeans no need for credit cards in order to sign up to AWS and use services.Simple procedure to be on-boarded quickly.

Global Data Egress Waiver removes data egress charges (typically ~3-5%) from monthly bills so researchers don’t have to estimate the cost of data movement between the cloud and their on premise services.

Single Sign-up

http://aws.amazon.com/rcp

Page 6: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

AWS Cloud Credits for Research

AWS has been providing credits toward AWS services to researchers since 2009. An average $ per project is between $10k-$20k.

We support researchers who seek to:• Build reusable tools to facilitate their future

research and the research of their community• Perform proof of concept or benchmark tests

evaluating the efficacy of moving research workloads or open data sets to the cloud

• Train a broader community on the usage of cloud for research workloads via workshops or tutorials

Page 7: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Application Process

Online application: http://aws.amazon.com/research-credits/

Proposal should contain the following information:1. Brief description of problem to be solved.2. Proposed AWS solution (including specific AWS tools, timeline, key milestones).3. Plan for sharing outcomes (tools, data, and/or resources) created during project.4. Any potential future use of AWS beyond grant duration by individual research group or broader community.5. Names of any AWS employees you have been in contact with (this is not a prerequisite for the application).6. Any AWS Public Data Sets to be used in your research.7. Keywords to facilitate proposal review.We also ask for a budget estimate using simple monthly calculator:

https://calculator.s3.amazonaws.com/index.html

Page 8: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Caffe: Deep Learning Framework

Page 9: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Researchers from all domains seek AWS support

0 20 40 60 80

Humanities

Social Sciences

Earth Science

Astronomy / Physics / Chemistry

Other

Bioinformatics / Life Sciences

Computer Science

Percentage of Submitted Proposals

Page 10: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

We support researchers at all career stages

0 10 20 30 40 50 60 70

Postdoc

Staff

Student

Faculty

Percentage of Funded Comp Sci Proposals

Page 11: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Frequently Asked Questions

For how long are the credits valid?Research credits expire after 1 year from the date of redemption.

What services are NOT covered by credits?Reserved Instance purchases, Mechanical Turk, Marketplace purchases, Business or Enterprise Support.

Can I apply for an extension or for additional credits?While credits are generally provided as a one-time support for a single project, we can evaluate additional support on individual bases.

Page 12: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Application Process

Submission Date Notification Date

January 1 - March 31 June

April 1 - June 30 September

July 1 - September 30 December

October 1 - December 31 March

Quarterly application deadlines.

Panel of reviewers across Amazon.

Historical funding line ~40% .

Notification of decisions occurs 2-3 months following deadline.

http://aws.amazon.com/research-credits/

Page 13: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Collaborative Grant Programs: AHA/AWSAmerican Heart Association: Precision Cardiovascular Medicine

Goal is to learn how to predict and prevent rather than just manage cardiovascular disease.

AHA provides cash, AWS provides credits.View current opportunity at AHA website: https://tinyurl.com/AHA-Precision-Medicine-Grants These include grants for fellowships and projects on data mining, methods validation, and innovative development around cardiovascular medicine.

Page 14: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Public datasets on AWS

AWS hosts a selection of datasets that anyone can access for free

Earth ScienceLandsat

NEXRAD

Life Sciences1000 Genomes Project

The Cancer Genome AtlasRice Genome

Machine LearningCommon Crawl CorpusMultimedia Commons

Computer Vision

http://aws.amazon.com/public-data-sets/

Page 15: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.https://aws.amazon.com/earth/

Page 16: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Amazon Research Awards (ARA) programhttps://ara.amazon-ml.com/

Call for Research Proposals • Amazon Research Awards (ARA) program for machine translation, computer

vision, general AI, machine learning, and other.• Combination of cash (up to $80K) and AWS service credits• Aims to support projects leading toward a Ph.D. degree or conducted as a

part of post-doctoral work• Full-time faculty members of institutions in North America and Europe

granting PhD degrees in fields related to Machine Learning are eligible to apply.

• Applications submitted online at https://ara.amazon-ml.com/proposals/#apply. Next deadline: September 2018

Page 17: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

AWS Machine Learning Research Awardshttps://aws.amazon.com/aws-ml-research-awards/

• The AWS Machine Learning Research Awards (MLRA) program funds eligible universities, faculty, PhD students and post-docs under the supervision of faculty, that are conducting novel research in machine learning (ML).

• For more information or to receive an application please contact [email protected]

Page 18: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Thank you!Contact:

[email protected] [email protected]

Page 19: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

How have customers used AWS Cloud Credits for Research Awards?

Page 20: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

US National Institutes for Health develops an AWS-hosted web service for microbiome analysis

• NIH collaborators used AWS credits to develop Nephele, a platform to facilitate identification of all the microorganisms in a biological sample

• Nephele enables scientists across the world to upload DNA microbiome sequence data and obtain a list of corresponding microorganisms

• Nephele uses AWS Lambda to spin up EC2 instances and custom genome analysis tools along with user-submitted sequence data

“AWS was the most mature of the cloud platforms evaluated and it had the most number of research users. Amazon also offered funding to some of the Nephelecollaborators through its research grant program, which helped get some development efforts off the ground…

- GenomeWeb article, February 2016

Page 21: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

International Centre for Radio Astronomy Researchuses AWS to analyze crowd-sourced data

• The Australian International Centre for Radio Astronomy Research (ICRAR) has developed a community computing initiative called theSkyNet

• theSkyNet allows ICRAR to use spare CPU cycles volunteered by the public to simulate a supercomputer

• ICRAR uses Amazon Route53 to route users to theSkyNetwebsite and uses Amazon EBS to store upwards of 400 GB worth of imaging data monthly as it is processed by the community

• By using AWS to manage crowd-sourced CPU cycles, ICRAR has the compute capacity to analyze between 400 and 500 galaxies simultaneously

The scalability of AWS has been enormously helpful. I can add more capacity as I need it with minimal fuss. Using AWS allows us to process upwards of 150 GB of sky images and store more than 400 GB of imaging data every month.

-Kevin Vinsen, ICRAR

Page 22: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

Stanford University’s Archaeology Center uses AWS to manage and share archaeological data

The AWS cloud gives our researchers faster access to cleaner data so they can collaborate more effectively on

new research questions and arrive at more cohesive interpretations of data.

-Lindsay Der, Stanford Archeology Center,Stanford University

• Stanford Archaeology center needed to improve the speed and accuracy of data entries for the Çatalhöyük Research Project in Turkey

• Used AWS credits to develop a system to run Esri’sArcGIS Server on Amazon EC2

• Updates to geo-spatial data are always current and synchronized, eliminating the need for 20+ hours weekly of manual data reconciliation

• Researchers in different locations around the world can quickly, easily access data for collaboration

*NSF funded PI

Page 23: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

CSIRO and the Black Dog Institute use AWS to analyze social media data

By using AWS, we were able to get the application up and

running in just a couple of months and now it’s allowing us

to analyze millions of tweets in real time.

-Cécile Paris, CSIRO

”“

• We Feel is a project that explores the use of social

media to monitor large-scale changes in mood

• We Feel was developed by Australian institutions

Commonwealth Scientific and Industrial Research

Organization (CSIRO) and the Black Dog Institute, with

the use of AWS credits

• We Feel uses Amazon Kinesis to analyze the emotional

content of approximately 27 million tweets/day

• Visitors to the website are able to drill into the results by

gender, location, and emotional quality and to link shifts

in mood to current events and social context

From website wefeel.csiro.au:Distribution of

tweets related to

“sadness” on a

single day in

March, 2016

Page 24: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

Harvard University uses AWS to develop workflow

for personalized medicine studies

The team of researchers tapped the Amazon cloud for a scalable framework to test the limits of genomics

workflows. Supported by a grant from Amazon Web Services, they were able to show how a custom workflow they developed was able to reduce the cost of analysts of

whole-genome data by 10x.

• The Laboratory for Personalized Medicine at Harvard University focuses on assessing the value of new genetic tests for use in preventative healthcare

• They were looking for an efficient way to handle large amounts of DNA sequence data without troubleshooting computing technology

• The lab used AWS credits to develop COSMOS, a workflow management system for genomic sequence data

• COSMOS takes advantage of AWS spot-instance pricing to dramatically reduce the cost of analysis

• COSMOS is available to researchers across the world who want to take advantage of highly scalable, low-cost computing solutions

-The Next Platform article, July 2015

Page 25: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

Penn State uses AWS to make biotech research available to the scientific community

Penn State teaches students to be leaders with a global perspective, conducts research that improves lives, and

contributes millions to the economy and sharing expertise.

We can create an algorithm that can take 32, 64, 128 different

cores, and make it accessible to anyone in the world.

• The Biological Engineering Department wanted to give biotech researchers an easy way to share data and run computationally intensive simulations on DNA

• Moved from an office server to AWS to make Penn State’s design methods and optimization algorithms available to researchers all over the world

• Scaled to meet the demands of more than 6,000 users, who have designed more than 50,000 synthetic DNA sequences

• Reduced time to market so that research findings are more immediately available to the scientific community

• Eliminated the need to buy, power, and maintain expensive hardware

Howard SalisPh.D., Assistant Professor, Penn State

Page 26: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

• San Francisco State University researchers working on a molecular research project, FEATURE, in collaboration with the Stanford Helix Group

• FEATURE uses machine learning to predict functional sites in proteins and other three-dimensional (3D) molecular structures

• Shared university resources led to long delays or impacted the scope of the research

• AWS removes size and scope limitations for research experiments while reducing costs

AWS helps accelerate protein research for the FEATURE machine learning project

With AWS high performance computing, experiments that used to take weeks now run

overnight

-Dragutin Petkovic, San Francisco State University ”

*NSF funded PI

Page 27: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

Code.org scales to support 20 million students for Hour of Code campaign on AWS

Running on AWS kept our application running when traffic spiked from zero to 20 million

student programmers in one hour.” -Geoffrey Elliott, Code.org

“ • Code.org wanted to launch its website to coincide CSEdWeek and promote its Hour of Code campaign

• The company turned to AWS to build an environment with high availability and redundancy

• AWS provided the elasticity to keep the website running when traffic spiked to 20 million student programmers in one hour

• AWS also provided the scalability to spin down instances even as traffic grew and save on costs

Page 28: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Caffe: Deep Learning Frameworkdeveloped at UC Berkeley

Page 29: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

FEATURE Project Architecture

Page 30: AWS support for academic research - Kent State University · Harvard University uses AWS to develop workflow for personalized medicine studies The team of researchers tapped the Amazon

San Francisco State University Featurize Concept