Riak - From Small to Large
-
Upload
rusty-klophaus -
Category
Technology
-
view
5.167 -
download
0
description
Transcript of Riak - From Small to Large
![Page 1: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/1.jpg)
Rusty Klophaus (@rklophaus)Basho Technologies
Riak from Small to Large
Strange Loop · October 2010
![Page 2: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/2.jpg)
Riak is a Dynamo-inspired,open-sourced, key/value datastorebuilt to scale predictably and easily.
2
![Page 3: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/3.jpg)
Riak is a Dynamo-inspired,open-sourced, key/value datastorebuilt to scale predictably and easily.
3
Your ops guy: calm, relaxed, and wearing a party hat.
![Page 4: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/4.jpg)
NoSQL
4
![Page 5: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/5.jpg)
5http://www.flickr.com/photos/ghewgill/129950124
![Page 6: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/6.jpg)
NoSQL has changed like Keanu hasn’t.
6http://www.youtube.com/watch?v=nEubt6HpGhs
![Page 7: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/7.jpg)
Companies, Funding, Customers, Awareness...
7http://www.flickr.com/photos/free-stock/4791385567
![Page 8: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/8.jpg)
“NoSQL? Yeah, it’s this really cool technology, you probably haven’t heard of it yet.”
8http://www.flickr.com/photos/slava/285199203
![Page 9: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/9.jpg)
“Johnson! Get me a memo on this ‘No sequal’ thing!”
9http://www.flickr.com/photos/proimos/4045973322
![Page 10: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/10.jpg)
Why?
10
![Page 11: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/11.jpg)
Because NoSQL is driven by narcissism, voyeurism, and
materialism.
![Page 12: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/12.jpg)
The catalyst for NoSQL is the data
generated or consumedby humans.
12
![Page 13: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/13.jpg)
13
Photos, music, and video...
![Page 14: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/14.jpg)
14
Constantly connected devices...
![Page 15: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/15.jpg)
15http://www.flickr.com/photos/hanspoldoja/5001818922
Social Networks...
![Page 16: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/16.jpg)
16http://www.flickr.com/photos/mega/7358278
Your Grandmama...
![Page 17: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/17.jpg)
Requirements That Confound an RDBMS
17
Large Data ObjectsBLOBs?
High-Availability SystemsBi-Directional Replication? Log shipping?
High-Capacity SystemsShards?
Vertical Partitioning?
Changing SchemasAuto-generated DDL?
Generic Columns?
Non-Standard AccessRecursive SQL?
![Page 18: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/18.jpg)
The roots of NoSQL are practical, not academic.
18
![Page 19: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/19.jpg)
Scratch an Itch Build a Company
Turning Point
19
➮
![Page 20: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/20.jpg)
Four Important Things
20
1. Broad KnowledgeLearn from the past, both DB *and* non-DB worlds.
• VP Technical Services, Akamai • Senior Architect, Akamai• Chief of Operations, Snapfish• Release Manager, Engine Yard• Bob Ippolito (CTO/CoFounder MochiMedia)• Dr. Eric Brewer (CAP Theorum)
![Page 21: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/21.jpg)
Four Important Things
21
2. Strong CommunityStay in touch with real problems.
Mark PhillipsBasho Community Manager
![Page 22: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/22.jpg)
Four Important Things
22
3. AgilityEvolve to solve new problems.
![Page 23: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/23.jpg)
Four Important Things
23
4. RestraintNo solution can do it all well.
![Page 24: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/24.jpg)
Back to Riak
24
![Page 25: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/25.jpg)
What characteristics of Riak become important at different cluster sizes?
25
![Page 26: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/26.jpg)
Single Box Riak
26
Simplicity
![Page 27: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/27.jpg)
Single Box RiakSimple Key/Value Store, Flexible Schema
27
client = RiakClient(host, port)bucket = client.bucket('mybucket')
obj = bucket.new('foo', 'bar')obj.store()
obj2 = bucket.get(‘foo’)
![Page 28: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/28.jpg)
Clients in Many Languages
Single Box Riak
28
![Page 29: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/29.jpg)
Predictable Development Interface
Single Box Riak
29
Riak
Riak Riak
RiakRiak
Riak
===
Development Production
![Page 30: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/30.jpg)
Configurable Buckets
Single Box Riak
30
!=
Profile
Bucket
Audio
Bucket
![Page 31: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/31.jpg)
Links (Lightweight Data Relations)
Single Box Riak
31
Profile Projects Comments
![Page 32: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/32.jpg)
Small Riak Cluster
32
Parallelism and Power
![Page 33: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/33.jpg)
Small Riak ClusterExpanding the Cluster
33
bin/riak-admin join riak@hostname
![Page 34: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/34.jpg)
Small Riak ClusterExpanding the Cluster
34
bin/riak-admin join riak@hostname
![Page 35: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/35.jpg)
Small Riak ClusterExpanding the Cluster
34
bin/riak-admin join riak@hostname
Hello ladies, can your datastore grow like mine?
*monocle smile*
![Page 36: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/36.jpg)
Small Riak ClusterDistributed Queries
35
![Page 37: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/37.jpg)
Small Riak ClusterDistributed Queries
36
![Page 38: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/38.jpg)
Small Riak ClusterJavascript-Based Map/Reduce
37
Keys
Map
Map
Map
Map
Map
Reduce
Reduce
Reduce
...
Output
![Page 39: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/39.jpg)
Small Riak ClusterWell-Behaved HTTP
38
Riak Riak
RiakRiak
Riak
nginx
Squid
ClientClient
Client
Browser
nginx proxy config - http://gist.github.com/323048
![Page 40: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/40.jpg)
Small Riak ClusterRiak Full-Text Search
39nginx proxy config - http://gist.github.com/323048
bin/search-cmd install mybucket
Riak Riak
RiakRiak
Riak
"converse AND category:shoes"
![Page 41: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/41.jpg)
Large Riak Cluster
40
Operations
![Page 42: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/42.jpg)
No Special Nodes
Large Riak Cluster
41
SpecialNodes
![Page 43: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/43.jpg)
Laugh at Machine Failure
Large Riak Cluster
42
Riak Riak
RiakRiak
RiakX
![Page 44: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/44.jpg)
Scale by Adding Nodes
Large Riak Cluster
43
Riak Riak
RiakRiak
Riak
Riak
Riak
Riak
Riak
Riak
Riak
Riak
![Page 45: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/45.jpg)
Packages or Self-Contained Installation
Large Riak Cluster
44
sudo dpkg -i riak_0.13.0-2_amd64.deb
-or-
unzip riak.zipriak/bin/riak start
![Page 46: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/46.jpg)
$0.00
![Page 47: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/47.jpg)
$0.00Order now, and we’ll include an
extra Riak for free!*
*Offer limited to open-source editions of Riak.
![Page 48: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/48.jpg)
• On-Call Support 24x7x365 • Management Tools• SNMP Monitoring• Multi-site Replication
Enterprise ($$$ / ~10’s of boxes)
47
![Page 49: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/49.jpg)
• On-Call Support 24x7x365• Management Tools • SNMP Monitoring• Multi-site Replication
Enterprise ($$$ / ~10’s of boxes)
48
![Page 50: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/50.jpg)
Thanks! Questions?Next Steps
Read more at http://wiki.basho.com
Download binaries from http://downloads.basho.com
Get source code from http://hg.basho.com
Join the public mailing list at [email protected]
Mailing list archives at http://riak.markmail.org
Thanks!
Rusty Klophaus (@rklophaus)
Mark Phillips (@pharkmillups)
49
![Page 51: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/51.jpg)
Additional Slides
![Page 52: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/52.jpg)
TutorialGet Riak
Connect with a Client
Store Data
Store an Object with Links
Linkwalking
Map/Reduce
51
![Page 53: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/53.jpg)
Get RiakDownloadhttp://downloads.basho.com/riak
Start Riakcd riakbin/riak start
52
![Page 54: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/54.jpg)
Connect with Python# Code is at http://hg.basho.comimport riak
# Connectclient = riak.RiakClient('127.0.0.1', 8098)
53
![Page 55: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/55.jpg)
Store Datamybucket = client.bucket('mybucket')
# Create an object...obj = mybucket.new('myobject')obj.set_data({ 'foo' : 1, 'bar' : 2 })obj.store()
# Read the object...obj = mybucket.get('myobject')print obj.get_data()
# Or, open a web browser...http://127.0.0.1:8098/riak/mybucket/myobject
54
![Page 56: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/56.jpg)
Store an Object with Linksbands = client.bucket('bands')albums = client.bucket('albums')members = client.bucket('members')
# Store a band, link to album and members...obj = bands.new('Winger') \ .add_link(albums.new('Pull', 1275922).store()) \ .add_link(albums.new('IV', 542731).store()) \ .add_link(albums.new('Karma', 200170).store()) \ .add_link(members.new('Kip Winger').store()) \ .add_link(members.new('Reb Beach').store()) \ .add_link(members.new('John Roth').store()) \ .add_link(members.new('Rod M.').store()) \ .store()
55
![Page 57: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/57.jpg)
Linkwalking# Get the albums...albums = obj.link('albums').run()
# Get the songs (assumes data is present)...songs = obj.link('albums').link('songs').run()
# Get the members...members = riak.MapReduce(client) \ .add('bands', 'Winger') \ .link('members') \ .run()
56
![Page 58: Riak - From Small to Large](https://reader034.fdocuments.in/reader034/viewer/2022052522/554bc03fb4c9053a298b508f/html5/thumbnails/58.jpg)
Map/Reduce# Count the number of sales...result = obj \ .link('albums') \ .map("function(v) { return [v.values[0].data]; }") \ .reduce("Riak.reduceSum") \ .run()
57