Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad:...
Transcript of Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad:...
![Page 1: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/1.jpg)
1/31
Kademlia: A Peertopeer Information System Based on the XOR Metric
Based on slides by Amir H. Payberah ([email protected])
![Page 2: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/2.jpg)
2/31
![Page 3: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/3.jpg)
3/31
Kademlia Basics
•Kademlia is a keyvalue(object) store.
•Each object is stored at the k closest nodes to the object's ID.
•Distance between id1 and id2: d(id1, id2) = id1 XOR id2 If ID space is 3 bits:
d(1, 4) = d(0012, 1002) = 0012 XOR 1002
= 1012 = 5
![Page 4: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/4.jpg)
4/31
Kademlia Routing Table
P
Node
KBucket List
KBucket
•Kbucket: each node keeps a list of references to nodes (contacts) of distance between 2i and 2i+1 for i=1 to i=N.
•Each Kbucket has max k entries.
[1, 2)
[2, 4)
[4, 8)
[8, 16)
[16, 32)
[32, 64)
[64, 128)
[128, 256)
![Page 5: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/5.jpg)
5/31
Kademlia Tuning Parameters
•B is the size in bits of the keys used to identify nodes and store and retrieve data; in basic Kademlia this is 160, the length of an SHA1 digest (hash).
•k is the maximum number of contacts stored in a Kbucket; this is normally 20.
•alpha () represents the degree of parallelism in network calls, usually 3.
•Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live
(TTL) from the original publication date tRefresh = 3600s, after which an otherwise unaccessed bucket must be refreshed tReplicate = 3600s, the interval between Kademlia replication events, when a node
is required to publish its entire database tRepublish = 86400s, the time after which the original publisher must republish a
key/value pair
![Page 6: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/6.jpg)
6/31
FIND_NODE in Kademlia
P
Node
KBucket List
Lookup Q
closest nodes to Q are stored here
•Closest nodes in ID space
![Page 7: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/7.jpg)
7/31
FIND_NODE in Kademlia
P
Node
KBucket List
closest nodes to Q are stored here
A B C
... and select nodes from the appropriate kbucket
Lookup Q
![Page 8: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/8.jpg)
8/31
FIND_NODE in Kademlia
FIND_NODE(Q)
P
A
B
C
FIND_NODE(Q)
FIND_NODE(Q)
![Page 9: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/9.jpg)
9/31
FIND_NODE in Kademlia
A
Find k closest nodes to Q
Find k closest nodes to Q
B
Find k closest nodes to Q
Find k closest nodes to Q
C
Find k closest nodes to Q
Find k closest nodes to Q
![Page 10: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/10.jpg)
10/31
FIND_NODE in Kademlia
Returns k closest nodes to Q
P
A
B
C
Returns k closest nodes to Q
Returns k closest nodes to Q
![Page 11: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/11.jpg)
11/31
FIND_NODE in Kademlia, Update Kbuckets
Received responses from A, B and C
P
When P receives a response from a node, it updates the appropriate Kbucket for the sender’s node ID.
KBucket List
M
P issues up to new requests to nodes it has not yet queried from the set of nodes received in the responses
N
O
![Page 12: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/12.jpg)
12/31
FIND_NODE in Kademlia
FIND_NODE(Q)
P
M
N
O
FIND_NODE(Q)
FIND_NODE(Q)
![Page 13: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/13.jpg)
13/31
FIND_NODE in Kademlia
Received information in round n1
P
Received information in round n
Repeats this procedure iteratively until received information in round n1 and n are the same.
![Page 14: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/14.jpg)
14/31
FIND_NODE in Kademlia
P
T S XReceived information in round n R
P resends the FIND_NODE to k closest nodes it has not already queried ...
![Page 15: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/15.jpg)
15/31
Let's Look Inside Kademlia
![Page 16: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/16.jpg)
16/31
Node State
•Kbucket: each node keeps a list of information for nodes of distance between 2i and 2i+1.
0 <= i < 160
Sorted by time last seen.
110
111
100101
000011 010 001
[1, 2)
[2, 4)
[4, 8)
![Page 17: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/17.jpg)
17/31
Node State
•Kbucket: each node keeps a list of information for nodes of distance between 2i and 2i+1.
0 <= i < 160
Sorted by time last seen.
110
111
100101
000011 010 001
[1, 2) Two first bits in common
[2, 4) First bit in common
[4, 8) No common prefix
![Page 18: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/18.jpg)
18/31
Kademlia RPCs
•PING Probes a node to see if it is online.
•STORE Instructs a node to store a <key, value> pair.
• FIND_NODE Returns information for the k nodes it knows about closest to the target ID. It can be from one kbucket or more.
• FIND_VALUE Like FIND_NODE, ... But if the recipient has stored they <key, value>, it just returns the stored value.
![Page 19: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/19.jpg)
19/31
Store Data
• The <key, value> data is stored in k closest nodes to the key.
![Page 20: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/20.jpg)
20/31
Lookup Service
001
000
011010
110 100 111
[1, 2)
[2, 4)
[4, 8)
110
111
100
000011 010 001
[1, 2)
[2, 4)
[4, 8)
100
101
110111
011001 000 010
[1, 2)
[2, 4)
[4, 8)
Step1
Step2
Step3
![Page 21: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/21.jpg)
21/31
Maintaining Kbucket List (Routing Table)
•When a Kademlia node receives any message from another node, it updates the appropriate kbucket for the sender’s node ID.
![Page 22: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/22.jpg)
22/31
Maintaining Kbucket List (Routing Table)
•When a Kademlia node receives any message from another node, it updates the appropriate kbucket for the sender’s node ID.
• If the sending node already exists in the kbucket: Moves it to the tail of the list.
![Page 23: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/23.jpg)
23/31
Maintaining Kbucket List (Routing Table)
•When a Kademlia node receives any message from another node, it updates the appropriate kbucket for the sender’s node ID.
• If the sending node already exists in the kbucket: Moves it to the tail of the list.
•Otherwise: If the bucket has fewer than k entries:
• Inserts the new sender at the tail of the list. Otherwise:
• Pings the kbucket’s leastrecently seen node:• If the leastrecently seen node fails to respond:
– it is evicted from the kbucket and the new sender inserted at the tail.• Otherwise:
– it is moved to the tail of the list, and the new sender’s contact is discarded.
![Page 24: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/24.jpg)
24/31
Maintaining Kbucket List (Routing Table)
•Buckets should generally be kept constantly fresh, due to traffic of requests travelling through nodes.
•When there is no traffic: each peer picks a random ID in kbucket's range and performs a node search for that ID.
![Page 25: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/25.jpg)
25/31
Join
•Node P contacts an already participating node Q.
•P inserts Q into the appropriate kbucket.
•P then performs a node lookup for its own node ID.
![Page 26: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/26.jpg)
26/31
Leave And Failure
•No action!
• If a node does not respond to the PING message, remove it from the table.
![Page 27: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/27.jpg)
27/31
Kademlia vs. Chord
![Page 28: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/28.jpg)
28/31
Kademlia vs. Chord
• like Chord When = 1 the lookup algorithm resembles Chord's in term of message
cost.
•Unlike Chord XOR metric is symmetric, while Chord's metric is asymmetric.
![Page 29: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/29.jpg)
29/31
Summary
001
000
011010
110 100 111
[1, 2)
[2, 4)
[4, 8)
110
111
100
000011 010 001
[1, 2)
[2, 4)
[4, 8)
100
101
110111
011001 000 010
[1, 2)
[2, 4)
[4, 8)
Step1
Step2
Step3
![Page 30: Kademlia: A Peertopeer Information System Based on the ... · •Other constants used in Kad: tExpire = 86400s, the time after which a key/value pair expires; this is a time-to-live](https://reader033.fdocuments.in/reader033/viewer/2022041614/5e39bfe133374209b03c3554/html5/thumbnails/30.jpg)
30/31
References
•Kademlia Specification http://xlattice.sourceforge.net/components/protocol/kademlia/specs.html
• Petar Maymounkov and David Mazieres, "Kademlia: A Peer-to-Peer Information System Based on the XOR Metric", IPTPS '02
http://www.cs.rice.edu/Conferences/IPTPS02/109.pdf
•Daniel Stutzbach and Reza Rejaie, "Improving Lookup Performance over a Widely-Deployed DHT", INFOCOM '06
http://www.barsoom.org/~agthorr/papers/infocom-2006-kad.pdf
•Raul Jimenez, Flutra Osmani and Bjorn Knutsson, “Sub-Second Lookups on a Large-Scale Kademlia-Based Overlay”, P2P '11.
http://people.kth.se/~rauljc/p2p11/jimenez2011subsecond.pdf