Riak Core: Building Distributed Applications Without Shared State

Rusty Klophaus (@rklophaus)Basho Technologies

Riak Core: Building Distributed

Applications Without Shared State

Commercial Users of Functional Programming Baltimore, MD · October 2010

Wednesday, October 6, 2010

2

You suddenly feelan uncontrollable desire to

learn Erlang.

http://www.flickr.com/photos/procsilas/18014203




What is Riak Core?How does it work?How can you use it?

3


Distributed, scalable, failure-tolerant.

4



No central coordinator. Easy to setup/operate.

5



Horizontally scalable; add commodity hardware

to get more X.

6



Always available. No single point of failure.

Self-healing.

7


Basho Technologies

8

Riak KVDistributed, scalable, failure-tolerant key/value datastore.

Started as a “Dynamo clone”.

Map/Reduce, Lightweight Data Relations, Client APIs

Riak SearchDistributed, scalable, failure-tolerant full-text search engine.

Near Realtime, Riak KV Integration, Solr Support


Riak

KV

Riak

Search

Riak

Core


Riak Core is an Erlang librarythat helps you build

distributed, scalable, failure-tolerant applications.

11


Amazon Dynamo


“We Generalized the Dynamo Architecture and Open-Sourced the Bits.”

13


Wait, doesn’t *Erlang* let you build distributed, scalable, failure-tolerant

applications?

14


Client

Service A Service B

Resource D

Service C

Queue E

Erlang makes it easy to connect the components of your application.


Service

Node A

Node E

Node I

Node M

Node B

Node F

Node J

Node N

Node C

Node G

Node K

Node O

Node D

Node H

Node L

. . .

Riak Core helps you build a service that harnesses the power of many nodes.


“People use languagesother than Erlang?!?I find that hilarious.”-Al Gore actually said this to me.

http://www.flickr.com/photos/scobleizer/2216445692




How does Riak Core work?

18


Command ObjectName, Payload


Predictable Routing

20


Hash the Object Name


SHA1(ObjName), Payload

0 to 2^160


A Naive Approach



Node A Node B Node C Node D


A Naive Approach

Command


Node A Node B Node C Node D Node E

ObjectName, Payload


"All problems in computer science can be solved by

another level of indirection." - David Wheeler

24


Add VNodes

25

WhatVirtual Node. Logical subdivision of the cluster.

Handles incoming commands, does work, replies.

For Parallelism# of VNodes = maximum concurrent requests

For Rebalancing the ClusterSmallest block that can be shifted to a new node.

For ResilianceThe system restarts failed VNodes.


Routing with Consistent Hash



VNode 0 VNode 1 VNode 2 VNode 3 VNode 4 VNode 5 VNode 6 VNode 7

Node A Node B Node C Node D


Adding a Node






Removing a Node






The Ring

Hash Location


The Ring

Preflist


Writing Replicas (n_val)

Preflist when N=3


Routing Around Failures

Preflist when N=3and node 0 is down.

X


Location of the Routing Layer

33


Router in the Middle

Client Client Client

Router

VNode

0


VNode

1

VNode

3

VNode

4

VNode

2

VNode

5

VNode

6

VNode

7


Riak Core - Router on Each Node


Router Router Router RouterRouter

VNode

0


VNode

1

VNode

3

VNode

4

VNode

2

VNode

5

VNode

6

VNode

7


Eventually - Router on the Client


VNode

0


VNode

1

VNode

3

VNode

4

VNode

2

VNode

5

VNode

6

VNode

7

Router RouterRouter


No Shared State

Router Router Router RouterRouter

VNode

0


VNode

1

VNode

3

VNode

4

VNode

2

VNode

5

VNode

6

VNode

7


Gossip

Local Ring State

IncomingRing State

Are rings equivalent?Strictly descendent?Or different?


Handoff

WhenNodes is added to the system.Node is removed from the system.Node has temporarily failed.

WhatShip the data backing a VNode from one node to another.

39


Not MentionedVector ClocksMerkle TreesBloom Filters

40


Distinguishedgentlemen

prefer Erlang.

http://www.flickr.com/photos/rebcal/3987226359




How do you use Riak Core?

42


Two Things to Think About

43

CommandCommand = ObjectName, PayloadThe commands/requests/operations that you will send through the system.

VNode ModuleThe callback module that will receive the commands.


VNode Module

44

Startup/Shutdowninit([Partition]) ->

{ok, State}

terminate(State) ->

ok

Commandshandle_command(Cmd, Sender, State) ->

{noreply, State1} | {reply, Reply, State1}

handle_handoff_command(Cmd, Sender, State) ->

{noreply, State1} | {reply, ok, State1}


VNode Module

45

Handoff Coordinationhandoff_starting(Node, State) ->

{Bool, State1}

encode_handoff_data(Data, State) ->

<<Binary>>.

handle_handoff_data(Data, Sender, State) ->

{reply, ok, State1}

handoff_finished(Node, State) ->

{ok, State1}


Start the riak_core application

riak_core

riak_core_vnode_sup riak_core_handoff_*

riak_core_ring_*

riak_core_node_*

riak_core_gossip_*X_vnode

. . .

X_vnode

X_vnode

46

application:start(riak_core).



riak_core


riak_core_ring_*

riak_core_node_*


. . .

X_vnode

X_vnode

47

Supervise vnode processes.



� � � � � � �

� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �

� � � � � � � � � � � � �

� � � � � � � � � � � � �

� � � � � � � � � � � � � � ��

� � � � �

� � � � � �

� � � � � �

48

Start, coordinate, and supervise handoff.



riak_core


riak_core_ring_*

riak_core_node_*


. . .

X_vnode

X_vnode

49

Maintain cluster membership information.



riak_core


riak_core_ring_*

riak_core_node_*


. . .

X_vnode

X_vnode

50

Monitor node liveness, broadcast to registered modules.



riak_core


riak_core_ring_*

riak_core_node_*


. . .

X_vnode

X_vnode

51

Send ring information to other nodes.Reconcile different views of the cluster.

Rebalance cluster when nodes join or leave.


In your application...

riak_core


riak_core_ring_*

riak_core_node_*


. . .

X_vnode

X_vnode

52

Start the vnodes for your application.Master = { riak_X_vnode_master, { riak_core_vnode_master, start_link, [riak_X_vnode] }, permanent, 5000, worker, [riak_core_vnode_master]},{ok, { {one_for_one, 5, 10}, [Master]} }.


In your application...

riak_core


riak_core_ring_*

riak_core_node_*


. . .

X_vnode

X_vnode

53

Tell riak_core that your applicationis ready to receive requests.

riak_core:register_vnode_module(riak_X_vnode),riak_core_node_watcher:service_up(riak_X, self())


In your application...riak_core


riak_core_ring_*

riak_core_node_*


. . .

X_vnode

X_vnode

riak_core


riak_core_ring_*

riak_core_node_*


. . .

X_vnode

X_vnode

54

Join to an existing node in the cluster.

riak_core_gossip:send_ring(ClusterNode, node())


Start Sending Commands

55

# Figure out the preflist...{_Verb, ObjName, _Payload} = Command,PrefList = riak_core_apl:get_apl(ObjName, NVal, riak_X),

# Send the command...riak_core_vnode_master:command(PrefList, Command, riak_X_vnode_master)


Review

Riak CoreOpen source Erlang library for building distributed, scalable, failure tolerant applications.

Continual improvement in the coming months.

Riak KVKey/Value datastore with map/reduce based on Riak Core.

Riak SearchFull-text, near real-time search engine based on Riak Core.

56


http://www.flickr.com/photos/jurvetson/469492885

At the center of the universe, beneath the bottom-most turtle,

is a cluster of Erlang nodes.





At the center of the universe, beneath the bottom-most turtle,

is a cluster of Erlang nodes.With 100% uptime.




Thanks! Questions?

Learn MoreMore Information: http://wiki.basho.comAmazon’s Dynamo Paper

Get the Codehttp://hg.basho.com/riak_core

Get in TouchTwitter : @rklophaus, @basho/teamEmail: [email protected]

59


http://wiki.basho.com/display/RIAK/Riak

http://wiki.basho.com/display/RIAK/Riak

http://hg.basho.com/riak_core

http://hg.basho.com/riak_core

END


Riak Core: Building Distributed Applications Without Shared State

Technology

Transcript of Riak Core: Building Distributed Applications Without Shared State