AWS support for academic research - Kent State University · Harvard University uses AWS to develop...
Transcript of AWS support for academic research - Kent State University · Harvard University uses AWS to develop...
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Irena Zlatanovic, Sr. Program Manager, Cloud Credits for Research Program
March 22, 2018
AWS support for academic research
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Supporting the mission of higher education
Discovery Education
Open Data
Scientific Computing
AWS Academy
Amazon Research Awards
AWS Machine Learning Research Awards
Earth on AWS
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS programs supporting research
Cloud Credits for Research
Research and Technical Computing
Public Datasets
• AWS hosts large and globally significant datasets for community use.
• Developing solutions to enable researchers to use AWS to host their own datasets.
• AWS ”grant” program to support individual research projects to faculty, staff, and students.
• Dedicated team of scientific experts focusing on scientific computing and research workloads.
• Developing solutions to enable researchers to benefit from AWS.
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Research Cloud Program
Science first, not servers. Simple start up procedures.
Budget management tools.
Large catalog of researchSolutions.
Solutions to procurement challenges.
Best practices for data security.
http://aws.amazon.com/rcp
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Solving Procurement Challenges
Invoice-backed billingmeans no need for credit cards in order to sign up to AWS and use services.Simple procedure to be on-boarded quickly.
Global Data Egress Waiver removes data egress charges (typically ~3-5%) from monthly bills so researchers don’t have to estimate the cost of data movement between the cloud and their on premise services.
Single Sign-up
http://aws.amazon.com/rcp
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Cloud Credits for Research
AWS has been providing credits toward AWS services to researchers since 2009. An average $ per project is between $10k-$20k.
We support researchers who seek to:• Build reusable tools to facilitate their future
research and the research of their community• Perform proof of concept or benchmark tests
evaluating the efficacy of moving research workloads or open data sets to the cloud
• Train a broader community on the usage of cloud for research workloads via workshops or tutorials
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Application Process
Online application: http://aws.amazon.com/research-credits/
Proposal should contain the following information:1. Brief description of problem to be solved.2. Proposed AWS solution (including specific AWS tools, timeline, key milestones).3. Plan for sharing outcomes (tools, data, and/or resources) created during project.4. Any potential future use of AWS beyond grant duration by individual research group or broader community.5. Names of any AWS employees you have been in contact with (this is not a prerequisite for the application).6. Any AWS Public Data Sets to be used in your research.7. Keywords to facilitate proposal review.We also ask for a budget estimate using simple monthly calculator:
https://calculator.s3.amazonaws.com/index.html
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Caffe: Deep Learning Framework
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Researchers from all domains seek AWS support
0 20 40 60 80
Humanities
Social Sciences
Earth Science
Astronomy / Physics / Chemistry
Other
Bioinformatics / Life Sciences
Computer Science
Percentage of Submitted Proposals
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
We support researchers at all career stages
0 10 20 30 40 50 60 70
Postdoc
Staff
Student
Faculty
Percentage of Funded Comp Sci Proposals
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Frequently Asked Questions
For how long are the credits valid?Research credits expire after 1 year from the date of redemption.
What services are NOT covered by credits?Reserved Instance purchases, Mechanical Turk, Marketplace purchases, Business or Enterprise Support.
Can I apply for an extension or for additional credits?While credits are generally provided as a one-time support for a single project, we can evaluate additional support on individual bases.
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Application Process
Submission Date Notification Date
January 1 - March 31 June
April 1 - June 30 September
July 1 - September 30 December
October 1 - December 31 March
Quarterly application deadlines.
Panel of reviewers across Amazon.
Historical funding line ~40% .
Notification of decisions occurs 2-3 months following deadline.
http://aws.amazon.com/research-credits/
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Collaborative Grant Programs: AHA/AWSAmerican Heart Association: Precision Cardiovascular Medicine
Goal is to learn how to predict and prevent rather than just manage cardiovascular disease.
AHA provides cash, AWS provides credits.View current opportunity at AHA website: https://tinyurl.com/AHA-Precision-Medicine-Grants These include grants for fellowships and projects on data mining, methods validation, and innovative development around cardiovascular medicine.
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Public datasets on AWS
AWS hosts a selection of datasets that anyone can access for free
Earth ScienceLandsat
NEXRAD
Life Sciences1000 Genomes Project
The Cancer Genome AtlasRice Genome
Machine LearningCommon Crawl CorpusMultimedia Commons
Computer Vision
http://aws.amazon.com/public-data-sets/
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.https://aws.amazon.com/earth/
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Research Awards (ARA) programhttps://ara.amazon-ml.com/
Call for Research Proposals • Amazon Research Awards (ARA) program for machine translation, computer
vision, general AI, machine learning, and other.• Combination of cash (up to $80K) and AWS service credits• Aims to support projects leading toward a Ph.D. degree or conducted as a
part of post-doctoral work• Full-time faculty members of institutions in North America and Europe
granting PhD degrees in fields related to Machine Learning are eligible to apply.
• Applications submitted online at https://ara.amazon-ml.com/proposals/#apply. Next deadline: September 2018
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Machine Learning Research Awardshttps://aws.amazon.com/aws-ml-research-awards/
• The AWS Machine Learning Research Awards (MLRA) program funds eligible universities, faculty, PhD students and post-docs under the supervision of faculty, that are conducting novel research in machine learning (ML).
• For more information or to receive an application please contact [email protected]
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you!Contact:
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How have customers used AWS Cloud Credits for Research Awards?
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
US National Institutes for Health develops an AWS-hosted web service for microbiome analysis
• NIH collaborators used AWS credits to develop Nephele, a platform to facilitate identification of all the microorganisms in a biological sample
• Nephele enables scientists across the world to upload DNA microbiome sequence data and obtain a list of corresponding microorganisms
• Nephele uses AWS Lambda to spin up EC2 instances and custom genome analysis tools along with user-submitted sequence data
”
“AWS was the most mature of the cloud platforms evaluated and it had the most number of research users. Amazon also offered funding to some of the Nephelecollaborators through its research grant program, which helped get some development efforts off the ground…
- GenomeWeb article, February 2016
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
International Centre for Radio Astronomy Researchuses AWS to analyze crowd-sourced data
• The Australian International Centre for Radio Astronomy Research (ICRAR) has developed a community computing initiative called theSkyNet
• theSkyNet allows ICRAR to use spare CPU cycles volunteered by the public to simulate a supercomputer
• ICRAR uses Amazon Route53 to route users to theSkyNetwebsite and uses Amazon EBS to store upwards of 400 GB worth of imaging data monthly as it is processed by the community
• By using AWS to manage crowd-sourced CPU cycles, ICRAR has the compute capacity to analyze between 400 and 500 galaxies simultaneously
The scalability of AWS has been enormously helpful. I can add more capacity as I need it with minimal fuss. Using AWS allows us to process upwards of 150 GB of sky images and store more than 400 GB of imaging data every month.
-Kevin Vinsen, ICRAR
“
”
Stanford University’s Archaeology Center uses AWS to manage and share archaeological data
The AWS cloud gives our researchers faster access to cleaner data so they can collaborate more effectively on
new research questions and arrive at more cohesive interpretations of data.
-Lindsay Der, Stanford Archeology Center,Stanford University
”
“
• Stanford Archaeology center needed to improve the speed and accuracy of data entries for the Çatalhöyük Research Project in Turkey
• Used AWS credits to develop a system to run Esri’sArcGIS Server on Amazon EC2
• Updates to geo-spatial data are always current and synchronized, eliminating the need for 20+ hours weekly of manual data reconciliation
• Researchers in different locations around the world can quickly, easily access data for collaboration
*NSF funded PI
CSIRO and the Black Dog Institute use AWS to analyze social media data
By using AWS, we were able to get the application up and
running in just a couple of months and now it’s allowing us
to analyze millions of tweets in real time.
-Cécile Paris, CSIRO
”“
• We Feel is a project that explores the use of social
media to monitor large-scale changes in mood
• We Feel was developed by Australian institutions
Commonwealth Scientific and Industrial Research
Organization (CSIRO) and the Black Dog Institute, with
the use of AWS credits
• We Feel uses Amazon Kinesis to analyze the emotional
content of approximately 27 million tweets/day
• Visitors to the website are able to drill into the results by
gender, location, and emotional quality and to link shifts
in mood to current events and social context
From website wefeel.csiro.au:Distribution of
tweets related to
“sadness” on a
single day in
March, 2016
Harvard University uses AWS to develop workflow
for personalized medicine studies
The team of researchers tapped the Amazon cloud for a scalable framework to test the limits of genomics
workflows. Supported by a grant from Amazon Web Services, they were able to show how a custom workflow they developed was able to reduce the cost of analysts of
whole-genome data by 10x.
• The Laboratory for Personalized Medicine at Harvard University focuses on assessing the value of new genetic tests for use in preventative healthcare
• They were looking for an efficient way to handle large amounts of DNA sequence data without troubleshooting computing technology
• The lab used AWS credits to develop COSMOS, a workflow management system for genomic sequence data
• COSMOS takes advantage of AWS spot-instance pricing to dramatically reduce the cost of analysis
• COSMOS is available to researchers across the world who want to take advantage of highly scalable, low-cost computing solutions
-The Next Platform article, July 2015
”
“
Penn State uses AWS to make biotech research available to the scientific community
Penn State teaches students to be leaders with a global perspective, conducts research that improves lives, and
contributes millions to the economy and sharing expertise.
We can create an algorithm that can take 32, 64, 128 different
cores, and make it accessible to anyone in the world.
• The Biological Engineering Department wanted to give biotech researchers an easy way to share data and run computationally intensive simulations on DNA
• Moved from an office server to AWS to make Penn State’s design methods and optimization algorithms available to researchers all over the world
• Scaled to meet the demands of more than 6,000 users, who have designed more than 50,000 synthetic DNA sequences
• Reduced time to market so that research findings are more immediately available to the scientific community
• Eliminated the need to buy, power, and maintain expensive hardware
Howard SalisPh.D., Assistant Professor, Penn State
”
“
• San Francisco State University researchers working on a molecular research project, FEATURE, in collaboration with the Stanford Helix Group
• FEATURE uses machine learning to predict functional sites in proteins and other three-dimensional (3D) molecular structures
• Shared university resources led to long delays or impacted the scope of the research
• AWS removes size and scope limitations for research experiments while reducing costs
AWS helps accelerate protein research for the FEATURE machine learning project
With AWS high performance computing, experiments that used to take weeks now run
overnight
-Dragutin Petkovic, San Francisco State University ”
“
*NSF funded PI
Code.org scales to support 20 million students for Hour of Code campaign on AWS
Running on AWS kept our application running when traffic spiked from zero to 20 million
student programmers in one hour.” -Geoffrey Elliott, Code.org
”
“ • Code.org wanted to launch its website to coincide CSEdWeek and promote its Hour of Code campaign
• The company turned to AWS to build an environment with high availability and redundancy
• AWS provided the elasticity to keep the website running when traffic spiked to 20 million student programmers in one hour
• AWS also provided the scalability to spin down instances even as traffic grew and save on costs
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Caffe: Deep Learning Frameworkdeveloped at UC Berkeley
FEATURE Project Architecture
San Francisco State University Featurize Concept