IMC Summit 2016 Innovation - Dennis Duckworth - Lambda-B-Gone: The In-memory Case Study for Faster,...
-
Upload
in-memory-computing-summit -
Category
Data & Analytics
-
view
68 -
download
1
Transcript of IMC Summit 2016 Innovation - Dennis Duckworth - Lambda-B-Gone: The In-memory Case Study for Faster,...
![Page 1: IMC Summit 2016 Innovation - Dennis Duckworth - Lambda-B-Gone: The In-memory Case Study for Faster, Smarter and Simpler Answers](https://reader036.fdocuments.in/reader036/viewer/2022070516/5872ec2a1a28abfa548b7451/html5/thumbnails/1.jpg)
LAMBDA-B-GONE: IN-MEMORY CASE STUDY FOR FASTER, SMARTER AND SIMPLER ANSWERS
DENNIS DUCKWORTHVOLTDB See all the presentations from the In-Memory
Computing Summit at http://imcsummit.org
![Page 2: IMC Summit 2016 Innovation - Dennis Duckworth - Lambda-B-Gone: The In-memory Case Study for Faster, Smarter and Simpler Answers](https://reader036.fdocuments.in/reader036/viewer/2022070516/5872ec2a1a28abfa548b7451/html5/thumbnails/2.jpg)
VOLTDB An operational database purpose-built to run 100% in-
memory at web scale In-Memory Relational, SQL, fully ACID compliantScale-out on commodity hardwareReliability, HA, fault tolerant Integration with OLAP, Hadoop, DW Best use cases: operational and transactional workloads
![Page 3: IMC Summit 2016 Innovation - Dennis Duckworth - Lambda-B-Gone: The In-memory Case Study for Faster, Smarter and Simpler Answers](https://reader036.fdocuments.in/reader036/viewer/2022070516/5872ec2a1a28abfa548b7451/html5/thumbnails/3.jpg)
ORIGINAL LAMBDA ARCHITECTURE
![Page 4: IMC Summit 2016 Innovation - Dennis Duckworth - Lambda-B-Gone: The In-memory Case Study for Faster, Smarter and Simpler Answers](https://reader036.fdocuments.in/reader036/viewer/2022070516/5872ec2a1a28abfa548b7451/html5/thumbnails/4.jpg)
VOLTDB-IMPROVED VERSION OF LAMBDA ARCHITECTURE
![Page 5: IMC Summit 2016 Innovation - Dennis Duckworth - Lambda-B-Gone: The In-memory Case Study for Faster, Smarter and Simpler Answers](https://reader036.fdocuments.in/reader036/viewer/2022070516/5872ec2a1a28abfa548b7451/html5/thumbnails/5.jpg)
CASE STUDY
5
Content Delivery Network Service Provider
Business challenges:- Real-time analytics for customers- Data accuracy: over/under billing- Scalability
![Page 6: IMC Summit 2016 Innovation - Dennis Duckworth - Lambda-B-Gone: The In-memory Case Study for Faster, Smarter and Simpler Answers](https://reader036.fdocuments.in/reader036/viewer/2022070516/5872ec2a1a28abfa548b7451/html5/thumbnails/6.jpg)
SQL interface unlike Trident or Spark-Streaming Merges the good things of the old-world like SQL and transactions
with the good things of the new world like ‘no-locks’, ‘k-factor’ HA, etc….
Very simple and intuitive API and usage k-factor + logs + snapshots eliminates the need to backup the
system Fast query performance Horizontal scalability
MAXCDN FINDINGS: VOLTDB ADVANTAGES
![Page 7: IMC Summit 2016 Innovation - Dennis Duckworth - Lambda-B-Gone: The In-memory Case Study for Faster, Smarter and Simpler Answers](https://reader036.fdocuments.in/reader036/viewer/2022070516/5872ec2a1a28abfa548b7451/html5/thumbnails/7.jpg)
MAXCDN FINDINGS: VOLTDB ADVANTAGES
![Page 8: IMC Summit 2016 Innovation - Dennis Duckworth - Lambda-B-Gone: The In-memory Case Study for Faster, Smarter and Simpler Answers](https://reader036.fdocuments.in/reader036/viewer/2022070516/5872ec2a1a28abfa548b7451/html5/thumbnails/8.jpg)
MAXCDN RESULTS
8
Simplified system architecture
1/10th the compute resources
100% budget accuracy, eliminated $$$ under/over spending
Faster time to value“We chose to go with VoltDB over other streaming aggregate solutions (like Trident) for its SQL interface, real-time Ad-Hoc queries over our raw data, and simpler overall design”Behzad Pirvali, Architect, MaxCDN
![Page 9: IMC Summit 2016 Innovation - Dennis Duckworth - Lambda-B-Gone: The In-memory Case Study for Faster, Smarter and Simpler Answers](https://reader036.fdocuments.in/reader036/viewer/2022070516/5872ec2a1a28abfa548b7451/html5/thumbnails/9.jpg)
REAL-TIME IN-MEMORY OLTP AND ANALYTICS WITH APACHE IGNITE ON AWS BY BABU ELUMALAI
HTTP://BLOGS.AWS.AMAZON.COM/BIGDATA/POST/TX3RS3V80XNRJH3/REAL-TIME-IN-MEMORY-OLTP-AND-ANALYTICS-WITH-APACHE-IGNITE-ON-AWS
![Page 10: IMC Summit 2016 Innovation - Dennis Duckworth - Lambda-B-Gone: The In-memory Case Study for Faster, Smarter and Simpler Answers](https://reader036.fdocuments.in/reader036/viewer/2022070516/5872ec2a1a28abfa548b7451/html5/thumbnails/10.jpg)
AMAZON DYNAMODB
![Page 11: IMC Summit 2016 Innovation - Dennis Duckworth - Lambda-B-Gone: The In-memory Case Study for Faster, Smarter and Simpler Answers](https://reader036.fdocuments.in/reader036/viewer/2022070516/5872ec2a1a28abfa548b7451/html5/thumbnails/11.jpg)
AMAZON DYNAMODB + AWS LAMBDA + KINESIS FIREHOSE + S3 + REDSHIFT
![Page 12: IMC Summit 2016 Innovation - Dennis Duckworth - Lambda-B-Gone: The In-memory Case Study for Faster, Smarter and Simpler Answers](https://reader036.fdocuments.in/reader036/viewer/2022070516/5872ec2a1a28abfa548b7451/html5/thumbnails/12.jpg)
AMAZON DYNAMODB + AWS LAMBDA + KINESIS FIREHOSE + S3 + REDSHIFT + SPARK + SPARK STREAMING +
![Page 13: IMC Summit 2016 Innovation - Dennis Duckworth - Lambda-B-Gone: The In-memory Case Study for Faster, Smarter and Simpler Answers](https://reader036.fdocuments.in/reader036/viewer/2022070516/5872ec2a1a28abfa548b7451/html5/thumbnails/13.jpg)
AMAZON DYNAMODB + AWS LAMBDA + KINESIS FIREHOSE + S3 + REDSHIFT + SPARK + SPARK STREAMING IGNITE + KCL
![Page 14: IMC Summit 2016 Innovation - Dennis Duckworth - Lambda-B-Gone: The In-memory Case Study for Faster, Smarter and Simpler Answers](https://reader036.fdocuments.in/reader036/viewer/2022070516/5872ec2a1a28abfa548b7451/html5/thumbnails/14.jpg)
VOLTDB-IMPROVED VERSION OF LAMBDA ARCHITECTURE
![Page 15: IMC Summit 2016 Innovation - Dennis Duckworth - Lambda-B-Gone: The In-memory Case Study for Faster, Smarter and Simpler Answers](https://reader036.fdocuments.in/reader036/viewer/2022070516/5872ec2a1a28abfa548b7451/html5/thumbnails/15.jpg)
VOLTDB-IMPROVED VERSION OF LAMBDA ARCHITECTURE
![Page 16: IMC Summit 2016 Innovation - Dennis Duckworth - Lambda-B-Gone: The In-memory Case Study for Faster, Smarter and Simpler Answers](https://reader036.fdocuments.in/reader036/viewer/2022070516/5872ec2a1a28abfa548b7451/html5/thumbnails/16.jpg)
ONE LAST THOUGHT: WORD OF THE DAY
IDEMPOTENCETHE PROPERTY OF CERTAIN OPERATIONS IN MATHEMATICS AND
COMPUTER SCIENCE, THAT CAN BE APPLIED MULTIPLE TIMES WITHOUT CHANGING THE RESULT BEYOND THE INITIAL
APPLICATION.
![Page 17: IMC Summit 2016 Innovation - Dennis Duckworth - Lambda-B-Gone: The In-memory Case Study for Faster, Smarter and Simpler Answers](https://reader036.fdocuments.in/reader036/viewer/2022070516/5872ec2a1a28abfa548b7451/html5/thumbnails/17.jpg)
DONE - THANK YOU
If you want to talk about how real transactions (and idempotence) can help you, come see us.
http://voltdb.com
all images from wikimedia w/ cc license unless otherwise noted