AWS at Mendeley (London, September 27th 2011)
13
Dan Harvey Systems Architect Revolutionising the world of research with Amazon Web Services
-
Upload
dan-harvey -
Category
Technology
-
view
1.771 -
download
5
description
How Mendeley uses Amazon web services achieve our vision of improving the world of research. This talk was giving at an AWS event in London.
Transcript of AWS at Mendeley (London, September 27th 2011)
Dan HarveySystems Architect
Revolutionising the world of research
with Amazon Web Services
Mendeley helps researchers work smarter
Mendeley makes science morecollaborative and transparent
Sync between computersand peers around the world
1 million researchers...
Uploaded 130 million citations
40 million unique
17 million papers
16 TB of data
Document Storage
• Backed on to S3
• Previews for 16TB of PDFs?
PDF Previews
ElasticBeanstalk S3
Process Queue
Load PDFs
Store PNGServe
via Cloud Front
Render to PDF
System Overview
EMREC2
S3
EBS
RDS
VPC
Article Search
• Based on Solr (open source search)• 40GB index• Variable usage
Day of the month
H-pRequests
Solr Layout
SolrSlave Solr
SlaveSolr
Slave
EBS
ElasticLoad Balancer
SolrMaster
SearchQueries
Inside VPC
Outside VPC
• Machine learning on Hadoop
• Personalised article recommendations• Collaborative filtering based
• Running on Elastic Map Reduce
Summary
• Not all or nothing
• Focus on your problemnot “Undifferentiated heavy lifting”
- Werner Vogels
• Learn the building blocks AWS provide
Enjoy what you’ve seen?
We’re hiring!
Senior Java Engineers
chat to me afteror e-mail/tweet
