Final Project: Milestone 2scohen/cs327e_spr18/slides/04-02-2018.pdfApr 02, 2018 · 2) Does Dremel...
Transcript of Final Project: Milestone 2scohen/cs327e_spr18/slides/04-02-2018.pdfApr 02, 2018 · 2) Does Dremel...
![Page 1: Final Project: Milestone 2scohen/cs327e_spr18/slides/04-02-2018.pdfApr 02, 2018 · 2) Does Dremel enforce a schema on the records of a table? A) Yes, records must satisfy a schema](https://reader033.fdocuments.in/reader033/viewer/2022041712/5e48c7df7ae13e5eff0e6632/html5/thumbnails/1.jpg)
Final Project: Milestone 2
CS 327EApril 2, 2018
![Page 2: Final Project: Milestone 2scohen/cs327e_spr18/slides/04-02-2018.pdfApr 02, 2018 · 2) Does Dremel enforce a schema on the records of a table? A) Yes, records must satisfy a schema](https://reader033.fdocuments.in/reader033/viewer/2022041712/5e48c7df7ae13e5eff0e6632/html5/thumbnails/2.jpg)
Announcements:
Demo Day: Friday 04/27Demo Schedule: https://tinyurl.com/yd68gutt
![Page 3: Final Project: Milestone 2scohen/cs327e_spr18/slides/04-02-2018.pdfApr 02, 2018 · 2) Does Dremel enforce a schema on the records of a table? A) Yes, records must satisfy a schema](https://reader033.fdocuments.in/reader033/viewer/2022041712/5e48c7df7ae13e5eff0e6632/html5/thumbnails/3.jpg)
1) What is a key characteristic of Dremel’s data model?
A) It is a flat relational modelB) It is a graph-like data model C) It is a nested record data model
![Page 4: Final Project: Milestone 2scohen/cs327e_spr18/slides/04-02-2018.pdfApr 02, 2018 · 2) Does Dremel enforce a schema on the records of a table? A) Yes, records must satisfy a schema](https://reader033.fdocuments.in/reader033/viewer/2022041712/5e48c7df7ae13e5eff0e6632/html5/thumbnails/4.jpg)
2) Does Dremel enforce a schema on the records of a table?
A) Yes, records must satisfy a schema that applies to the tableB) No, records can have different structures in the same table
![Page 5: Final Project: Milestone 2scohen/cs327e_spr18/slides/04-02-2018.pdfApr 02, 2018 · 2) Does Dremel enforce a schema on the records of a table? A) Yes, records must satisfy a schema](https://reader033.fdocuments.in/reader033/viewer/2022041712/5e48c7df7ae13e5eff0e6632/html5/thumbnails/5.jpg)
3) Dremel stores records in a ________ format.
A) row formatB) column formatC) JSON format
![Page 6: Final Project: Milestone 2scohen/cs327e_spr18/slides/04-02-2018.pdfApr 02, 2018 · 2) Does Dremel enforce a schema on the records of a table? A) Yes, records must satisfy a schema](https://reader033.fdocuments.in/reader033/viewer/2022041712/5e48c7df7ae13e5eff0e6632/html5/thumbnails/6.jpg)
4) What is a key performance benefit of column striping?
A) columns that are not needed by the query are not read from disk.B) rows that are not needed by the query are not read from disk.
![Page 7: Final Project: Milestone 2scohen/cs327e_spr18/slides/04-02-2018.pdfApr 02, 2018 · 2) Does Dremel enforce a schema on the records of a table? A) Yes, records must satisfy a schema](https://reader033.fdocuments.in/reader033/viewer/2022041712/5e48c7df7ae13e5eff0e6632/html5/thumbnails/7.jpg)
5) What is Dremel’s query language?
A) Standard SQL + path expressions for querying nested fieldsB) MapReduceC) CypherD) None of the above
![Page 8: Final Project: Milestone 2scohen/cs327e_spr18/slides/04-02-2018.pdfApr 02, 2018 · 2) Does Dremel enforce a schema on the records of a table? A) Yes, records must satisfy a schema](https://reader033.fdocuments.in/reader033/viewer/2022041712/5e48c7df7ae13e5eff0e6632/html5/thumbnails/8.jpg)
BigQuery & UNION ALL
SELECT c1, c2, c3
FROM `project.dataset.T1`
[WHERE c1 > c2]
UNION ALL
SELECT c1, c2, c3
FROM `project.dataset.T2`
[WHERE c2 > c3]
[ORDER BY c1];
Note: UNION ALL is not available in legacy SQL and requires turning on the Standard SQL option in BigQuery.
![Page 9: Final Project: Milestone 2scohen/cs327e_spr18/slides/04-02-2018.pdfApr 02, 2018 · 2) Does Dremel enforce a schema on the records of a table? A) Yes, records must satisfy a schema](https://reader033.fdocuments.in/reader033/viewer/2022041712/5e48c7df7ae13e5eff0e6632/html5/thumbnails/9.jpg)
![Page 10: Final Project: Milestone 2scohen/cs327e_spr18/slides/04-02-2018.pdfApr 02, 2018 · 2) Does Dremel enforce a schema on the records of a table? A) Yes, records must satisfy a schema](https://reader033.fdocuments.in/reader033/viewer/2022041712/5e48c7df7ae13e5eff0e6632/html5/thumbnails/10.jpg)
![Page 11: Final Project: Milestone 2scohen/cs327e_spr18/slides/04-02-2018.pdfApr 02, 2018 · 2) Does Dremel enforce a schema on the records of a table? A) Yes, records must satisfy a schema](https://reader033.fdocuments.in/reader033/viewer/2022041712/5e48c7df7ae13e5eff0e6632/html5/thumbnails/11.jpg)
BigQuery & SQL Functions
Examples:
CAST()
CONCAT()
LENGTH()
ENDS_WITH()
SUBSTR()
CURRENT_DATE()
DATE_DIFF()
ROUND()
RAND()
STDDEV()
APPROX_QUANTILES()
Reference Documentation: https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators
Types of functions:
• Datatype conversion functions• String functions• Date functions• Time functions• Aggregate functions• Approximate Aggregate functions• Statistical functions• Mathematical functions• Array functions
![Page 12: Final Project: Milestone 2scohen/cs327e_spr18/slides/04-02-2018.pdfApr 02, 2018 · 2) Does Dremel enforce a schema on the records of a table? A) Yes, records must satisfy a schema](https://reader033.fdocuments.in/reader033/viewer/2022041712/5e48c7df7ae13e5eff0e6632/html5/thumbnails/12.jpg)
Tip: Create views in BigQuery by running the query and choosing the Save View option.
![Page 13: Final Project: Milestone 2scohen/cs327e_spr18/slides/04-02-2018.pdfApr 02, 2018 · 2) Does Dremel enforce a schema on the records of a table? A) Yes, records must satisfy a schema](https://reader033.fdocuments.in/reader033/viewer/2022041712/5e48c7df7ae13e5eff0e6632/html5/thumbnails/13.jpg)
Subqueries: General Form
SELECT c1, c2, c3
FROM (Table Subquery)
WHERE c1 >= c2;
SELECT c1, c2, c3
FROM T1
WHERE c1 IN (Column Subquery);
SELECT c1, c2, (Scalar Subquery)
FROM T1
WHERE c1 = (Scalar Subquery);
Note: Subqueries require turning on the Standard SQL option in BigQuery.
![Page 14: Final Project: Milestone 2scohen/cs327e_spr18/slides/04-02-2018.pdfApr 02, 2018 · 2) Does Dremel enforce a schema on the records of a table? A) Yes, records must satisfy a schema](https://reader033.fdocuments.in/reader033/viewer/2022041712/5e48c7df7ae13e5eff0e6632/html5/thumbnails/14.jpg)
Note: A subquery is required for finding the listings that have bothWasher and Dryer amenities.
![Page 15: Final Project: Milestone 2scohen/cs327e_spr18/slides/04-02-2018.pdfApr 02, 2018 · 2) Does Dremel enforce a schema on the records of a table? A) Yes, records must satisfy a schema](https://reader033.fdocuments.in/reader033/viewer/2022041712/5e48c7df7ae13e5eff0e6632/html5/thumbnails/15.jpg)
Note: A subquery is required for computing the max price.