Radu Pastia - Couchdoop - Connecting Hadoop with Couchbase
-
Upload
huguk -
Category
Technology
-
view
120 -
download
0
description
Transcript of Radu Pastia - Couchdoop - Connecting Hadoop with Couchbase
![Page 1: Radu Pastia - Couchdoop - Connecting Hadoop with Couchbase](https://reader033.fdocuments.in/reader033/viewer/2022052900/55626425d8b42aab1a8b4b38/html5/thumbnails/1.jpg)
![Page 2: Radu Pastia - Couchdoop - Connecting Hadoop with Couchbase](https://reader033.fdocuments.in/reader033/viewer/2022052900/55626425d8b42aab1a8b4b38/html5/thumbnails/2.jpg)
pas$aro.wordpress.com
@rpas$a
![Page 3: Radu Pastia - Couchdoop - Connecting Hadoop with Couchbase](https://reader033.fdocuments.in/reader033/viewer/2022052900/55626425d8b42aab1a8b4b38/html5/thumbnails/3.jpg)
![Page 4: Radu Pastia - Couchdoop - Connecting Hadoop with Couchbase](https://reader033.fdocuments.in/reader033/viewer/2022052900/55626425d8b42aab1a8b4b38/html5/thumbnails/4.jpg)
Building a connector – The Wrong Way
Mapper Reducer
![Page 5: Radu Pastia - Couchdoop - Connecting Hadoop with Couchbase](https://reader033.fdocuments.in/reader033/viewer/2022052900/55626425d8b42aab1a8b4b38/html5/thumbnails/5.jpg)
![Page 6: Radu Pastia - Couchdoop - Connecting Hadoop with Couchbase](https://reader033.fdocuments.in/reader033/viewer/2022052900/55626425d8b42aab1a8b4b38/html5/thumbnails/6.jpg)
Building a connector – The Right Way
Mapper Reducer Par$$oner
Input Split
Input Format
Record Reader
Record Writer
Output Format
![Page 7: Radu Pastia - Couchdoop - Connecting Hadoop with Couchbase](https://reader033.fdocuments.in/reader033/viewer/2022052900/55626425d8b42aab1a8b4b38/html5/thumbnails/7.jpg)
![Page 8: Radu Pastia - Couchdoop - Connecting Hadoop with Couchbase](https://reader033.fdocuments.in/reader033/viewer/2022052900/55626425d8b42aab1a8b4b38/html5/thumbnails/8.jpg)
![Page 9: Radu Pastia - Couchdoop - Connecting Hadoop with Couchbase](https://reader033.fdocuments.in/reader033/viewer/2022052900/55626425d8b42aab1a8b4b38/html5/thumbnails/9.jpg)
![Page 10: Radu Pastia - Couchdoop - Connecting Hadoop with Couchbase](https://reader033.fdocuments.in/reader033/viewer/2022052900/55626425d8b42aab1a8b4b38/html5/thumbnails/10.jpg)
The InputFormat: From Input to Mapper --range 2014-09-01;2014-09-20
--number_of_mappers 4
2014-‐09-‐01 2014-‐09-‐02 2014-‐09-‐03
2014-‐09-‐04
2014-‐09-‐05
… … …
2014-‐09-‐06
2014-‐09-‐20
2014-‐09-‐01
2014-‐09-‐02
2014-‐09-‐05
.
.
.
Input Split 1
(2014-‐09-‐01-‐A; record A)
(2014-‐09-‐01-‐B; record B)
(2014-‐09-‐01-‐…; record …)
(2014-‐09-‐02-‐A; record A)
(2014-‐09-‐02-‐B; record B)
(2014-‐09-‐02-‐…; record …)
(2014-‐09-‐05-‐A; record A)
(2014-‐09-‐05-‐B; record B)
(2014-‐09-‐05-‐…; record …)
Record Reader 1
Mapper
![Page 11: Radu Pastia - Couchdoop - Connecting Hadoop with Couchbase](https://reader033.fdocuments.in/reader033/viewer/2022052900/55626425d8b42aab1a8b4b38/html5/thumbnails/11.jpg)
![Page 12: Radu Pastia - Couchdoop - Connecting Hadoop with Couchbase](https://reader033.fdocuments.in/reader033/viewer/2022052900/55626425d8b42aab1a8b4b38/html5/thumbnails/12.jpg)
![Page 13: Radu Pastia - Couchdoop - Connecting Hadoop with Couchbase](https://reader033.fdocuments.in/reader033/viewer/2022052900/55626425d8b42aab1a8b4b38/html5/thumbnails/13.jpg)
![Page 14: Radu Pastia - Couchdoop - Connecting Hadoop with Couchbase](https://reader033.fdocuments.in/reader033/viewer/2022052900/55626425d8b42aab1a8b4b38/html5/thumbnails/14.jpg)
![Page 15: Radu Pastia - Couchdoop - Connecting Hadoop with Couchbase](https://reader033.fdocuments.in/reader033/viewer/2022052900/55626425d8b42aab1a8b4b38/html5/thumbnails/15.jpg)
![Page 16: Radu Pastia - Couchdoop - Connecting Hadoop with Couchbase](https://reader033.fdocuments.in/reader033/viewer/2022052900/55626425d8b42aab1a8b4b38/html5/thumbnails/16.jpg)
The InputFormat: From Input to Mapper --range 2014-09-01;2014-09-20
--number_of_mappers 4
2014-‐09-‐01 2014-‐09-‐02 2014-‐09-‐03
2014-‐09-‐04
2014-‐09-‐05
… … …
2014-‐09-‐06
2014-‐09-‐20
2014-‐09-‐01
2014-‐09-‐02
2014-‐09-‐05
.
.
.
Input Split 1
(2014-‐09-‐01-‐A; record A)
(2014-‐09-‐01-‐B; record B)
(2014-‐09-‐01-‐…; record …)
(2014-‐09-‐02-‐A; record A)
(2014-‐09-‐02-‐B; record B)
(2014-‐09-‐02-‐…; record …)
(2014-‐09-‐05-‐A; record A)
(2014-‐09-‐05-‐B; record B)
(2014-‐09-‐05-‐…; record …)
Record Reader 1
Mapper
![Page 17: Radu Pastia - Couchdoop - Connecting Hadoop with Couchbase](https://reader033.fdocuments.in/reader033/viewer/2022052900/55626425d8b42aab1a8b4b38/html5/thumbnails/17.jpg)
![Page 18: Radu Pastia - Couchdoop - Connecting Hadoop with Couchbase](https://reader033.fdocuments.in/reader033/viewer/2022052900/55626425d8b42aab1a8b4b38/html5/thumbnails/18.jpg)
![Page 19: Radu Pastia - Couchdoop - Connecting Hadoop with Couchbase](https://reader033.fdocuments.in/reader033/viewer/2022052900/55626425d8b42aab1a8b4b38/html5/thumbnails/19.jpg)
![Page 20: Radu Pastia - Couchdoop - Connecting Hadoop with Couchbase](https://reader033.fdocuments.in/reader033/viewer/2022052900/55626425d8b42aab1a8b4b38/html5/thumbnails/20.jpg)