236601 - Coding and Algorithms for Memories Lecture 11
description
Transcript of 236601 - Coding and Algorithms for Memories Lecture 11
![Page 1: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/1.jpg)
1
236601 - Coding and Algorithms for
MemoriesLecture 11
![Page 2: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/2.jpg)
2
Array Codes and Distributed Storage
![Page 3: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/3.jpg)
Large Scale Storage Systems
3
• Big Data Players: Facebook, Amazon, Google, Yahoo,…
Cluster of machines running Hadoop at Yahoo! (Source: Yahoo!)
• Failures are the norm
![Page 4: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/4.jpg)
Node failures at Facebook
4
Date
XORing Elephants: Novel Erasure Codes for Big Data M. Sathiamoorthy, M. Asteris, D. Papailiopoulos, A. G. Dimakis, R. Vadali, S. Chen, and D. Borthakur, VLDB 2013
![Page 5: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/5.jpg)
• 3x Replication: » Easily implemented and maintained » Can tolerate any 2 disk failures » Large storage overhead of 300% - A Big Problem!
• More sophisticated schemes:» Reed-Solomon (RS) Codes» The repair problem
State-of-the-Art Storing Schemes
5
1 2 3 4 5 6 7 9 108
1 2 3 4 5 6 7 9 108
1 2 3 4 5 6 7 9 108
Widely used
![Page 6: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/6.jpg)
6
Problem Setup
• Disks are stored together in a group (rack)• Disk failures should be supported• Requirements:– Support as many disk failures as possible– And yet…
• Optimal and fast recovery• Low complexity
![Page 7: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/7.jpg)
7
Problem Setup
• Question 1: How many extra disks are required to support a single disk failure?
Answer: 1, How?• Question 2: How many extra disks are
required to support two disk failures? Answer: 2, How?• Question 3: How many extra disks are
required to support 3 disk failures?Answer: 3, How?
![Page 8: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/8.jpg)
8
Problem Setup
• Question 1: How many extra disks are required to support a single disk failure?
• Question 2: How many extra disks are required to support two disk failures?
• Question 3: How many extra disks are required to support 3 disk failures?
A B C A+B+C
A B C A+B+C
A+B+C
A B C A+B+C
A+B+C
’A+’B+’
C
![Page 9: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/9.jpg)
9
Problem Setup
• Question 1: How many extra disks are required to support a single disk failure?
• Question 2: How many extra disks are required to support two disk failures?
• Question 3: How many extra disks are required to support d disk failures?
A B C A+B+C
A B C A+B+C
A+B+C
A B C A+B+C
A+B+C
’A+’B+’C
{(x1,x2,x3,x4): x1+x2+x3+x4= 0 }
{(x1,x2,x3,x4,x5): x1+x2+x3+x4=0 x1+x2+x3+x5=0 }
{(x1,x2,x3,x4,x5,x6): x1+x2+x3+x4=0 x1+x2+x3+x5=0
’x1+’x2+’x3+x6=0}
![Page 10: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/10.jpg)
10
Problem Setup• Question 1: How many extra disks are required to support
a single disk failure?
• Question 2: How many extra disks are required to support two disk failures?
• Question 3: How many extra disks are required to support d disk failures?
A B C A+B+C
A B C A+B+C
A+B+C
A B C A+B+C
A+B+C
’A+’B+’C
{(x1,x2,x3,x4): x1+x2+x3+x4= 0 }
{(x1,x2,x3,x4,x5): x1+x2+x3+x4=0 x1+x2+x3+x5=0 }
{(x1,x2,x3,x4,x5,x6): x1+x2+x3+x4=0 x1+x2+x3+x5=0
’x1+’x2+’x3+x6=0}
{(x1,x2,x3,x4): H1∙(x1,x2,x3,x4)T=0}
H1 = (1,1,1,1)
{(x1,x2,x3,x4,x5): H2∙(x1,x2,x3,x4,x5)T=0}
H2= (1,1,1,1,0; ,,,0,1)
{(x1,x2,x3,x4,x5,x6):H3∙(x1,x2,x3,x4,x5,x6)T=0} H3= (1,1,1,1,0,0; ,,,0,1,0; ’,’,’,0,1,0)
![Page 11: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/11.jpg)
11
Problem Setup
• Question 2: How many extra disks are required to support two disk failures?
• Question: What is the requirement on H2?
Answer: Every 2x2 sub-matrix has rank two• Question: What is the requirement on H3?
Answer: Every 3x3 sub-matrix has rank three
A B C A+B+C
A+B+C
{(x1,x2,x3,x4,x5): x1+x2+x3+x4=0 x1+x2+x3+x5=0 }
{(x1,x2,x3,x4,x5): H2∙(x1,x2,x3,x4,x5)T=0}
H2= (1,1,1,1,0; ,,,0,1)
![Page 12: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/12.jpg)
12
Problem Setup
• Question: How many extra disks are required to support d disk failures?Answer: d, How?
{(x1,x2,…,xn-1,xn):H∙(x1,x2,…,xn-1,xn)T=0}, n=k+d• What is the requirement on H?• Answer: Every sub-matrix of size dxd has
rank d• Is it possible to construct such matrices?
![Page 13: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/13.jpg)
13
Reed Solomon Codes
• A code with parity check matrix of the form
Where is a primitive element at some extension field and O() > n-1Claim: Every sub-matrix of size dxd has full rank
![Page 14: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/14.jpg)
14
Vandermonde Matrices
![Page 15: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/15.jpg)
15
Reed Solomon Codes
• Advantages:– Support the maximum number of disk failures– Are very comment in practice and have
relatively efficient encoding/decoding schemes
• Disadvantages – Require to work over large fields– Need to require all the disks in order to recover
even a single disk failure – not efficient rebuild
![Page 16: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/16.jpg)
16
EVENODD Codes
• Designed by Mario Balum, Jim Brady, Jehoshua Bruck, and Jai Menon
• Goal: Construct array codes correcting 2 disk failures using only binary XOR operations– No need for calculations over extension fields
• Code construction:– Every disk is a column– The array size is (m-1)x(m+2), m is prime– The last two arrays are used for parity
![Page 17: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/17.jpg)
17
EVENODD Codes
• Code construction:– Every disk is a column– The array size is (m-1)x(m+2), m is prime– The last two arrays are used for parity
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
![Page 18: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/18.jpg)
18
EVENODD Codes
• Redundancy Calculation:– First parity drive – a simple XOR of the first m-1 disks
for 0 ≤ l ≤ m-2– Second parity drive – S=1
for 0 ≤ l ≤ m-2
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
1 0 1 1 0 1
0 0 0 1 1 0
0 0 0 0 1 1
1 1 1 0 1 0
![Page 19: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/19.jpg)
19
EVENODD Codes
• Redundancy Calculation:– First parity drive – a simple XOR of the first m-1 disks
for 0 ≤ l ≤ m-2– Second parity drive – S=1
for 0 ≤ l ≤ m-2
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 1 0 1
0 0 0 0 1 1 0
1 0 0 0 0 1 1
0 1 1 1 0 1 0
0 0 0 0 0 0 0
![Page 20: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/20.jpg)
20
Decoding of EVENODD Codes
• Observation: the value of S is the bits sum on the last two columns
S = 1
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 ? 0 ?
0 0 0 0 ? 1 ?
1 0 0 0 ? 1 ?
0 1 1 1 ? 1 ?
![Page 21: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/21.jpg)
21
Decoding of EVENODD Codes
• Observation: the value of S is the bits sum on the last two columns
S = 1
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 ? 0 ?
0 0 0 0 ? 1 ?
1 0 0 0 ? 1 ?
0 1 1 1 ? 1 ?
![Page 22: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/22.jpg)
22
Decoding of EVENODD Codes
• Observation: the value of S is the bits sum on the last two columns
S = 1
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 ? 0 ?
0 0 0 0 ? 1 0
1 0 0 0 ? 1 ?
0 1 1 1 ? 1 ?
![Page 23: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/23.jpg)
23
Decoding of EVENODD Codes
• Observation: the value of S is the bits sum on the last two columns
S = 1
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 ? 0 ?
0 0 0 0 1 1 0
1 0 0 0 ? 1 ?
0 1 1 1 ? 1 ?
![Page 24: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/24.jpg)
24
Decoding of EVENODD Codes
• Observation: the value of S is the bits sum on the last two columns
S = 1
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 ? 0 ?
0 0 0 0 1 1 0
1 0 0 0 ? 1 ?
0 1 1 1 ? 1 0
![Page 25: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/25.jpg)
25
Decoding of EVENODD Codes
• Observation: the value of S is the bits sum on the last two columns
S = 1
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 ? 0 ?
0 0 0 0 1 1 0
1 0 0 0 ? 1 ?
0 1 1 1 0 1 0
![Page 26: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/26.jpg)
26
Decoding of EVENODD Codes
• Observation: the value of S is the bits sum on the last two columns
S = 1
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 ? 0 1
0 0 0 0 1 1 0
1 0 0 0 ? 1 ?
0 1 1 1 0 1 0
![Page 27: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/27.jpg)
27
Decoding of EVENODD Codes
• Observation: the value of S is the bits sum on the last two columns
S = 1
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 ? 0 1
0 0 0 0 1 1 0
1 0 0 0 ? 1 ?
0 1 1 1 0 1 0
![Page 28: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/28.jpg)
28
Decoding of EVENODD Codes
• Observation: the value of S is the bits sum on the last two columns
S = 1
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 1 0 1
0 0 0 0 1 1 0
1 0 0 0 ? 1 ?
0 1 1 1 0 1 0
![Page 29: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/29.jpg)
29
Decoding of EVENODD Codes
• Observation: the value of S is the bits sum on the last two columns
S = 1
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 1 0 1
0 0 0 0 1 1 0
1 0 0 0 ? 1 1
0 1 1 1 0 1 0
![Page 30: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/30.jpg)
30
Decoding of EVENODD Codes
• Observation: the value of S is the bits sum on the last two columns
S = 1
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 1 0 1
0 0 0 0 1 1 0
1 0 0 0 0 1 1
0 1 1 1 0 1 0
![Page 31: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/31.jpg)
31
EVENODD Codes
• Redundancy Calculation:– First parity drive – a simple XOR of the first m-1 disks
for 0 ≤ l ≤ m-2– Second parity drive – S=1
for 0 ≤ l ≤ m-2
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 1 0 1
0 0 0 0 1 1 0
1 0 0 0 0 1 1
0 1 1 1 0 1 0
![Page 32: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/32.jpg)
32
![Page 33: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/33.jpg)
33
Decoding of EVENODD Codes
• Observation: the value of S is the bits sum on the last two columns
S = 1
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 ? 0 ?
0 0 0 0 ? 1 ?
1 0 0 0 ? 1 ?
0 1 1 1 ? 1 ?
![Page 34: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/34.jpg)
34
Decoding of EVENODD Codes
• Observation: the value of S is the bits sum on the last two columns
S = 1
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 ? 0 ?
0 0 0 0 ? 1 ?
1 0 0 0 0 1 ?
0 1 1 1 ? 1 ?
![Page 35: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/35.jpg)
35
Decoding of EVENODD Codes
• Observation: the value of S is the bits sum on the last two columns
S = 1
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 ? 0 ?
0 0 0 0 ? 1 ?
1 0 0 0 0 1 1
0 1 1 1 ? 1 ?
![Page 36: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/36.jpg)
36
Decoding of EVENODD Codes
• Observation: the value of S is the bits sum on the last two columns
S = 1
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 1 0 ?
0 0 0 0 ? 1 ?
1 0 0 0 0 1 1
0 1 1 1 ? 1 ?
![Page 37: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/37.jpg)
37
Decoding of EVENODD Codes
• Observation: the value of S is the bits sum on the last two columns
S = 1
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 1 0 1
0 0 0 0 ? 1 ?
1 0 0 0 0 1 1
0 1 1 1 ? 1 ?
![Page 38: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/38.jpg)
38
Decoding of EVENODD Codes
• Observation: the value of S is the bits sum on the last two columns
S = 1
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 1 0 1
0 0 0 0 ? 1 ?
1 0 0 0 0 1 1
0 1 1 1 0 1 ?
![Page 39: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/39.jpg)
39
Decoding of EVENODD Codes
• Observation: the value of S is the bits sum on the last two columns
S = 1
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 1 0 1
0 0 0 0 ? 1 ?
1 0 0 0 0 1 1
0 1 1 1 0 1 0
![Page 40: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/40.jpg)
40
Decoding of EVENODD Codes
• Observation: the value of S is the bits sum on the last two columns
S = 1
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 1 0 1
0 0 0 0 1 1 ?
1 0 0 0 0 1 1
0 1 1 1 0 1 0
![Page 41: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/41.jpg)
41
Decoding of EVENODD Codes
• Observation: the value of S is the bits sum on the last two columns
S = 1
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 1 0 1
0 0 0 0 1 1 0
1 0 0 0 0 1 1
0 1 1 1 0 1 0
![Page 42: 236601 - Coding and Algorithms for Memories Lecture 11](https://reader036.fdocuments.in/reader036/viewer/2022062500/568152d3550346895dc0eeaa/html5/thumbnails/42.jpg)
42
EVENODD Codes
• Redundancy Calculation:– First parity drive – a simple XOR of the first m-1 disks
for 0 ≤ l ≤ m-2– Second parity drive – S=1
for 0 ≤ l ≤ m-2
0 1 1 0 1
0 0 1 1 0
0 0 0 1 1
1 1 0 1 0
0 1 0 1 1 0 1
0 0 0 0 1 1 0
1 0 0 0 0 1 1
0 1 1 1 0 1 0