Gizzard, DAL and more
-
Upload
fulin-tang -
Category
Documents
-
view
4.067 -
download
4
description
Transcript of Gizzard, DAL and more
![Page 1: Gizzard, DAL and more](https://reader034.fdocuments.in/reader034/viewer/2022051611/54b7b1c34a7959ba688b45d9/html5/thumbnails/1.jpg)
gizzard, DAL and more
@tangfl
2011.2.22
11年2月28日星期一
![Page 2: Gizzard, DAL and more](https://reader034.fdocuments.in/reader034/viewer/2022051611/54b7b1c34a7959ba688b45d9/html5/thumbnails/2.jpg)
gizzard 1
• sharding framework
• extract from FlockDB
• written in Java/Scala
11年2月28日星期一
![Page 3: Gizzard, DAL and more](https://reader034.fdocuments.in/reader034/viewer/2022051611/54b7b1c34a7959ba688b45d9/html5/thumbnails/3.jpg)
gizzard 2
• middle ware :
• like mysql-proxy ?
• any backends
• like mytrigger ?
• adapter ?
11年2月28日星期一
![Page 4: Gizzard, DAL and more](https://reader034.fdocuments.in/reader034/viewer/2022051611/54b7b1c34a7959ba688b45d9/html5/thumbnails/4.jpg)
gizzard 3
• rules: forwarding table
• maps key ranges to partitions
• replication tree
11年2月28日星期一
![Page 5: Gizzard, DAL and more](https://reader034.fdocuments.in/reader034/viewer/2022051611/54b7b1c34a7959ba688b45d9/html5/thumbnails/5.jpg)
gizzard 4
• eventually consistent
• idempotent : f(f(n)) = f(n)
• commutative : f(a) + f(b) = f(b) + f(a)
• retry queue
11年2月28日星期一
![Page 6: Gizzard, DAL and more](https://reader034.fdocuments.in/reader034/viewer/2022051611/54b7b1c34a7959ba688b45d9/html5/thumbnails/6.jpg)
gizzard 5
• gracefully failures
• read : route to remain ones
• write : make high priority to remain replicas if any in a shard are unavailable
• retry queue
11年2月28日星期一
![Page 7: Gizzard, DAL and more](https://reader034.fdocuments.in/reader034/viewer/2022051611/54b7b1c34a7959ba688b45d9/html5/thumbnails/7.jpg)
gizzard 6
• gracefully migrations
• replica tree
• WriteOnly shard
11年2月28日星期一
![Page 8: Gizzard, DAL and more](https://reader034.fdocuments.in/reader034/viewer/2022051611/54b7b1c34a7959ba688b45d9/html5/thumbnails/8.jpg)
gizzard 7
• summary
• nothing new
• Get work done
11年2月28日星期一
![Page 9: Gizzard, DAL and more](https://reader034.fdocuments.in/reader034/viewer/2022051611/54b7b1c34a7959ba688b45d9/html5/thumbnails/9.jpg)
DAL 1
• Data Access Layer
• from iMobile
• by longker, simaliu and fulin
• Not only sharding
11年2月28日星期一
![Page 10: Gizzard, DAL and more](https://reader034.fdocuments.in/reader034/viewer/2022051611/54b7b1c34a7959ba688b45d9/html5/thumbnails/10.jpg)
DAL 2
• data table, hash by pri id, less index
• index tables, hash by any “where field” on demand
• front php devs : I don’t care !
11年2月28日星期一
![Page 11: Gizzard, DAL and more](https://reader034.fdocuments.in/reader034/viewer/2022051611/54b7b1c34a7959ba688b45d9/html5/thumbnails/11.jpg)
DAL 3
• insert / update / delete : auto update index table
• query : analyze sql, choose best index table
11年2月28日星期一
![Page 12: Gizzard, DAL and more](https://reader034.fdocuments.in/reader034/viewer/2022051611/54b7b1c34a7959ba688b45d9/html5/thumbnails/12.jpg)
DAL 4
• build-in cache : php don’t care about cache anymore
11年2月28日星期一
![Page 13: Gizzard, DAL and more](https://reader034.fdocuments.in/reader034/viewer/2022051611/54b7b1c34a7959ba688b45d9/html5/thumbnails/13.jpg)
DAL 5
• any backend
• treat Lucene just like mysql : select field from table where condition
• no replica
• backend do the job
11年2月28日星期一
![Page 14: Gizzard, DAL and more](https://reader034.fdocuments.in/reader034/viewer/2022051611/54b7b1c34a7959ba688b45d9/html5/thumbnails/14.jpg)
DAL 6
• summary
• nothing new
• Get work done
11年2月28日星期一
![Page 15: Gizzard, DAL and more](https://reader034.fdocuments.in/reader034/viewer/2022051611/54b7b1c34a7959ba688b45d9/html5/thumbnails/15.jpg)
more• Discuss
• fans_all list in redis
• vary from 0 to 500w
• single hash at most 5000
• auto resharding
• async
11年2月28日星期一
![Page 16: Gizzard, DAL and more](https://reader034.fdocuments.in/reader034/viewer/2022051611/54b7b1c34a7959ba688b45d9/html5/thumbnails/16.jpg)
Thanks
• @tangfl http://t.sina.com.cn
• http://blog.fulin.org
11年2月28日星期一