Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF
-
Upload
jesse-yates -
Category
Business
-
view
1.707 -
download
2
description
Transcript of Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF
![Page 1: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/1.jpg)
Secondary Indexing in Phoenix
Jesse YatesHBase CommitterSoftware Engineer
HBase BoF – June 25, 2013
![Page 2: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/2.jpg)
HBase BoF - June 20132
Outline
• Motivation• History• HBase Consistent Indexing– Index Management– Recovery Mechanism
• Conclusion
![Page 3: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/3.jpg)
HBase BoF - June 20133
A quick note…
![Page 4: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/4.jpg)
HBase BoF - June 20134
Outline
• Motivation• History• HBase Consistent Indexing– Index Management– Recovery Mechanism
• Conclusion
![Page 5: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/5.jpg)
HBase BoF - June 20135
Why do we need them?
• Sorted by key– Great for accessing on that key
What if we want to access by another dimension!?Table Scan!
![Page 6: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/6.jpg)
HBase BoF - June 20136
A short example
• Easy to search by name of food
• Hard to search on another dimension
Name Type Date Received Manufacturer Current Count
Apple Macintosh 6/23/13 Good Farm Inc. 200
Turkey Breast 6/23/13 Tasty Meat Co. 42
Chicken Drumstick 6/18/13 Pretty Ok Food 3
Jam Strawberry 6/18/10 Mash It Up Inc. 700
![Page 7: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/7.jpg)
HBase BoF - June 20137
A short exampleName Type Date Received Manufacturer Current Count
Apple Macintosh 6/23/13 Good Farm Inc. 200
Turkey Breast 6/23/13 Tasty Meat Co. 42
Chicken Drumstick 6/18/13 Pretty Ok Food 3
Jam Strawberry 6/18/10 Mash It Up Inc. 700
Date Received Name Type Manufacturer Current Count
6/18/13 Jam Strawberry Mash It Up Inc. 700
6/18/13 Chicken Drumstick Pretty Ok Food 3
6/23/13 Apple Macintosh Good Farm Inc. 200
6/23/13 Turkey Breast Tasty Meat Co. 42
![Page 8: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/8.jpg)
HBase BoF - June 20138
Outline
• Motivation• History• HBase Consistent Indexing– Index Management– Recovery Mechanism
• Conclusion
![Page 9: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/9.jpg)
HBase BoF - June 20139
HBase is “Special”…
• Partitioned Keys (“HRegion”)
• Scales because regions are independent
• Built-in data recovery mechanisms
![Page 10: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/10.jpg)
HBase BoF - June 201310
Hasn’t someone tried this?
• Omid
• Percolator
• Culvert
• Lily
• TrendMicro
• Client-coordinated
![Page 11: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/11.jpg)
HBase BoF - June 201311
We’ve gotten better…
• NGData– HBase-SEP– HBase-Indexer
• Intel– Lucene Full Text Indexing
![Page 12: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/12.jpg)
HBase BoF - June 201312
Still missing some things
• In-HBase index storage– Just another table in HBase
• Simple consistency guarantees– If X fails, then Y
• Minimal overhead for covered indexes– Network roundtrips
![Page 13: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/13.jpg)
HBase BoF - June 201313
Outline
• Motivation• History• HBase Consistent Indexing– Index Management– Recovery Mechanism
• Conclusion
![Page 14: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/14.jpg)
14
Two Major Components
• Index Management– Build index updates– Ensures index is ‘cleaned up’
• Recovery Mechanism– Ensures index updates are “ACID”
HBase BoF - June 2013
![Page 15: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/15.jpg)
HBase BoF - June 201315
Index Management
• Lives within a RegionCoprocesorObesrver• Access to the local Hregion• Specifies the mutations to apply to the index
tables
public interface IndexBuilder {public void setup(RegionCoprocessorEnvironment env);public Map<Mutation, String> getIndexUpdate(Put put);public Map<Mutation, String> getIndexUpdate(Delete delete);
}
![Page 16: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/16.jpg)
HBase BoF - June 201316
Index Management
![Page 17: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/17.jpg)
Key Observation #1
“We shouldn’t need to provide stronger guarantees than HBase - that is just asking for a bad time.”
- Jon Hsieh
HBase BoF - June 201317
* Paraphrased
*
![Page 18: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/18.jpg)
HBase BoF - June 201318
HBase ACID
• Does NOT give you:– Cross-row consistency– Cross-table consistency
• Does give you:– Durable data on success– Visibility on success without partial rows
![Page 19: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/19.jpg)
Key Observation #2
“Secondary indexing is inherently an easier problem than full transactions… secondary index updates are idempotent.”
- Lars Hofhansl
HBase BoF - June 201319
![Page 20: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/20.jpg)
HBase BoF - June 201320
Idempotent Index Updates
• Doesn’t need full transactions
• Replay as many times as needed
• Can tolerate a little lag– As long as we get the order right
![Page 21: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/21.jpg)
Taking a little ACID…
HBase BoF - June 201321
![Page 22: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/22.jpg)
HBase BoF - June 201322
![Page 23: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/23.jpg)
HBase BoF - June 201323
Durable Indexing: Standard Write Path
Client HRegion
RegionCoprocessorHost
WAL
RegionCoprocessorHost
MemStore
![Page 24: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/24.jpg)
HBase BoF - June 201324
Durable Indexing: Standard Write Path
Client HRegion
RegionCoprocessorHost
WAL
RegionCoprocessorHost
MemStore
![Page 25: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/25.jpg)
HBase BoF - June 201325
Durable Indexing
RegionCoprocessor
Host
WAL
RegionCoprocessorHost
Indexer IndexBuilder
WAL Updater
Durable!
IndexerIndex Table
Index TableIndex Table
![Page 26: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/26.jpg)
HBase BoF - June 201326
Failure Situations
• Before writing the WAL– Nothing is durable, nothing is visible
![Page 27: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/27.jpg)
HBase BoF - June 201327
Durable Indexing
Client HRegion
RegionCoprocessorHost
WAL
RegionCoprocessorHost
MemStore
Indexer
Indexer
Index TableIndex TableIndex Table
![Page 28: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/28.jpg)
HBase BoF - June 201328
Failure Situations
• Before writing the WAL– Nothing is durable, nothing is visible
✔
![Page 29: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/29.jpg)
HBase BoF - June 201329
Failure Situations
• Before writing the WAL– Nothing is durable, nothing is visible
• After writing WAL, before index update– WAL Replay updates the index table and the
primary table
✔
![Page 30: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/30.jpg)
HBase BoF - June 201330
Durable Indexing
Client HRegion
RegionCoprocessorHost
WAL
RegionCoprocessorHost
MemStore
Indexer
Indexer
Index TableIndex TableIndex Table
![Page 31: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/31.jpg)
HBase BoF - June 201331
Failure Situations
• Before writing the WAL– Nothing is durable, nothing is visible
• After writing WAL, before index update– WAL Replay updates the index table and the
primary table
✔
✔
![Page 32: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/32.jpg)
HBase BoF - June 201332
Failure Situations
• Before writing the WAL– Nothing is durable, nothing is visible
• After writing WAL, before index update– WAL Replay updates the index table and the
primary table• Mid-index update– WAL Replay finishes index update, primary table
update
✔
✔
![Page 33: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/33.jpg)
HBase BoF - June 201333
Durable Indexing
Client HRegion
RegionCoprocessorHost
WAL
RegionCoprocessorHost
MemStore
Indexer
Indexer
Index TableIndex TableIndex Table
![Page 34: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/34.jpg)
HBase BoF - June 201334
Failure Situations• Before writing the WAL– Nothing is durable, nothing is visible
• After writing WAL, before index update– WAL Replay updates the index table and the primary
table• Mid-index update– WAL Replay finishes index update, primary table
update
✔
✔
✔
![Page 35: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/35.jpg)
HBase BoF - June 201335
Failure Situations
• Before writing the WAL– Nothing is durable, nothing is visible
• After writing WAL, before index update– WAL Replay updates the index table and the primary
table• Mid-index update– WAL Replay finishes index update, primary table update
• After index updates, before primary– WAL Replay restores primary state, idempotently
applies index updates
✔
✔
✔
![Page 36: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/36.jpg)
HBase BoF - June 201336
Durable Indexing
Client HRegion
RegionCoprocessorHost
WAL
RegionCoprocessorHost
MemStore
Indexer
Indexer
Index TableIndex TableIndex Table
![Page 37: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/37.jpg)
HBase BoF - June 201337
Failure Situations
• Before writing the WAL– Nothing is durable, nothing is visible
• After writing WAL, before index update– WAL Replay updates the index table and the primary
table• Mid-index update– WAL Replay finishes index update, primary table update
• After index updates, before primary– WAL Replay restores primary state, idempotently
applies index updates
✔
✔
✔
✔
![Page 38: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/38.jpg)
HBase BoF - June 201338
Special Note: Failed Index Updates
• Index is corrupted– Index Table does not exist– Index table does not have write schema– Etc.
• Fail-fast behavior– Kill the whole server– Forces WAL Replay to enforce correctness– Modular enough to support alternative schemes
![Page 39: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/39.jpg)
HBase BoF - June 201339
Key Points
• Custom KeyValues to enable index durability in primary table WAL
• Custom WALEdit Codec for index update with WAL Replay
• Will see index updates before primary– Only a little bit of lag and never ‘wrong’– Matches HBase consistency
• Fail-fast behavior to enforce correctness
![Page 40: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/40.jpg)
HBase BoF - June 201340
Upcoming Work
• Performance testing
• Standard covered index managers
• Index cleanup on compaction
![Page 41: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/41.jpg)
HBase BoF - June 201341
Outline
• Motivation• History• HBase Consistent Indexing– Index Management– Recovery Mechanism
• Conclusion
![Page 42: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/42.jpg)
HBase BoF - June 201342
Conclusion
• Fully transparent to client
• Easy to build custom index maintenance
• Meets current HBase consistency guarantees
• Supports HBase 0.94.9+– Coming to 0.96/0.98 soon!
![Page 43: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/43.jpg)
hbase-index
HBase BoF - June 201343
https://github.com/forcedotcom/phoenix/tree/master/contrib/hbase-index
![Page 44: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/44.jpg)
Detailed Blog Post
HBase BoF - June 201344
http://jyates.github.io/2013/06/11/hbase-consistent-secondary-indexing.html
![Page 45: Secondary Indexing in Phoenix - Hadoop Summit 2012 - HBase BoF](https://reader036.fdocuments.in/reader036/viewer/2022062703/555097cab4c90595208b46db/html5/thumbnails/45.jpg)
HBase BoF - June 201345
Bonus!
• Usable as a standalone module
• Coming to phoenix*– Built-in support
• Future: added to HBase core (?)
* https://github.com/forcedotcom/phoenix