Using Apache Calcite for enabling SQL and JDBC access to Apache ...
Apache Calcite: One planner fits all
-
Upload
julian-hyde -
Category
Documents
-
view
306 -
download
0
Transcript of Apache Calcite: One planner fits all
![Page 1: Apache Calcite: One planner fits all](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ad7d2b1a28abfd798b47a2/html5/thumbnails/1.jpg)
One planner fits all (How Apache Calcite makes it easier to write a DBMS)
Lightning talk at XLDB 2015 Stanford, California
Julian Hyde (Hortonworks)
![Page 2: Apache Calcite: One planner fits all](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ad7d2b1a28abfd798b47a2/html5/thumbnails/2.jpg)
–Mike Stonebraker (2005)
“One size fits all” is an idea whose time has come and gone
![Page 3: Apache Calcite: One planner fits all](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ad7d2b1a28abfd798b47a2/html5/thumbnails/3.jpg)
• Hadoop and other open source technologies have deconstructed the DBMS
• Query parser/API + catalog + authorization + algorithms + scheduler + engine + data format + storage
![Page 4: Apache Calcite: One planner fits all](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ad7d2b1a28abfd798b47a2/html5/thumbnails/4.jpg)
image credit: http://oliviaobryon.com
![Page 5: Apache Calcite: One planner fits all](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ad7d2b1a28abfd798b47a2/html5/thumbnails/5.jpg)
image credit: http://oliviaobryon.com
Interesting
Boring
![Page 6: Apache Calcite: One planner fits all](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ad7d2b1a28abfd798b47a2/html5/thumbnails/6.jpg)
Conventional DB
Parser
Algebra
Catalog
Data
Algorithms
![Page 7: Apache Calcite: One planner fits all](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ad7d2b1a28abfd798b47a2/html5/thumbnails/7.jpg)
Apache Calcite DB framework
Parser
Algebra
Engine
Data
Engine
Data
Engine
Data
Schema SPI Operators, Rules,
Statistics, Cost model
![Page 8: Apache Calcite: One planner fits all](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ad7d2b1a28abfd798b47a2/html5/thumbnails/8.jpg)
SELECT products.name, COUNT(*) FROM salesJOIN products USING (productId) WHERE sales.discount IS NOT NULLGROUP BY products.name ORDER BY COUNT(*) DESC
scan [products]
scan [sales]
join
filter
aggregate
sort
scan [products]
scan [sales]
filter’
join’
aggregate
sort
FilterIntoJoinRule
translate SQL to relational
algebra
![Page 9: Apache Calcite: One planner fits all](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ad7d2b1a28abfd798b47a2/html5/thumbnails/9.jpg)
Relational algebra• Robust
• Allows re-use
• Complex cost-based optimization
• Multiple front-ends & back-ends
• Not just for “flat” relations
SQL OtherQL API
AlgebraRules
Engine A
Engine B
![Page 10: Apache Calcite: One planner fits all](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ad7d2b1a28abfd798b47a2/html5/thumbnails/10.jpg)
Thank you!Download: http://calcite.incubator.apache.org
Use Calcite to build your next database!
Calcite powers Apache Hive, Drill, Phoenix, Kylin
An Apache Incubator projectsince May 2014
@julianhyde
What’s in the box?
• SQL parser & AST • JDBC/ODBC framework • Built-in operators (project,
filter, …) • In-memory engine • 100+ rules • Planning engines • Adapters (CSV, JDBC,
Mongo, …) • Streaming SQL • Materialized views
ApacheCalciteApache Calcite