Google BigQuery - The first step -

2016/05/28 Mulodo Vietnam Co., Ltd.

What’s BigQueryOfficial site : https://cloud.google.com/bigquery/docs/

BigQuery is Google's fully managed, petabyte scale, low cost analytics data warehouse.

BigQuery is NoOps—there is no infrastructure to manage and you don't need a database administrator—so you can focus on analyzing data to find meaningful insights, use familiar SQL, and take advantage of our pay-as-you-go model.

→ WDH: SQL like (easy to use), Petabyte scale(for Huge data)

Features of BigQueryColumn oriented Database

Full Scan instead of using Index

Charge for scanned data size

http://static.googleusercontent.com/media/research.google.com/ja//pubs/archive/36632.pdf https://www.flickr.com/photos/mairin/galleries/72157624181197793/https://cloud.google.com/blog/big-data/2016/03/google-bigquery-cuts-historical-data-storage-cost-in-half-and-accelerates-many-queries-by-10x

The first step - overview -1. Preparation (create account)

2. Try BigQuery with prepared data

3. Upload your data to your BigQuery storage and try it.

Preparation steps1. Preparation (create account)

1. Create your Google account

2. Move to Google Cloud platform (has no account)

3. “Try IT Free”

4. “Sign up for free trial”

5. Account registration

6. Move to Google Cloud Platform (login)

7. Select ‘BigQuery’ from left menu (top-left icon)

Create your google account1. Preparation (create account)

1. Create your Google account

https://accounts.google.com/signUp

Move to Google Cloud Platform1. Preparation (create account)

2. Go to Google Cloud platform (has no account) 3. “Try IT Free”

https://cloud.google.com

Move to Google Cloud Platform1. Preparation (create account)

2. Go to Google Cloud platform (has no account) 3. “Try IT Free”

https://cloud.google.com

Move to Google Cloud Platform1. Preparation (create account)

2. Go to Google Cloud platform (has no account) 3. “Try IT Free”

https://cloud.google.com

Account registration1. Preparation (create account)

4. “Sign up for free trial” 5. Account registration

Account registration1. Preparation (create account)

4. “Sign up for free trial” 5. Account registration

<- Credit Card required

Move to Google Cloud Platform1. Preparation (create account)

6. Move to Google Cloud Platform (login) 7. Select ‘BigQuery’ from left menu (top-left icon)

https://cloud.google.com

Move to Google Cloud Platform1. Preparation (create account)

6. Move to Google Cloud Platform (login) 7. Select ‘BigQuery’ from left menu (top-left icon)

https://cloud.google.com

Move to Google Cloud Platform1. Preparation (create account)

6. Move to Google Cloud Platform (login) 7. Select ‘BigQuery’ from left menu (top-left icon)

https://cloud.google.com

Now you are ready to use BigQuery

Credit card ....

I can hardly check how much I have as free credit.

Credit card ....

I can hardly check how much I have as free credit.

Okay. Anytime you can check your free creit and trial days easily.

... I wanna try BigQuery

Try BigQuery with public data

Try BigQuery with public data

Query

Query

SELECT repository_name FROM [bigquery-public-data:samples.github_timeline] LIMIT 1000

Query

SELECT repository_name FROM [bigquery-public-data:samples.github_timeline] LIMIT 1000

Query

SELECT repository_name FROM [bigquery-public-data:samples.github_timeline] LIMIT 1000

Query

SELECT repository_name FROM [bigquery-public-data:samples.github_timeline] LIMIT 1000

Query

SELECT repository_name FROM [bigquery-public-data:samples.github_timeline] LIMIT 1000

EX) Query complete (2.1s elapsed, cached) Query complete (3.8s elapsed, 61.4 MB processed)

More query

SELECT actor, actor_attributes_name , count( actor_attributes_name ) as amount FROM [bigquery-public-data:samples.github_timeline]

group by actor, actor_attributes_name order by amount desc LIMIT 1000

Much more query

You can check query syntax by yourself.

https://cloud.google.com/bigquery/query-reference

Upload your data

1. Create database

2. Create Table (with CSV)

3. check “Table ID”

4. QUERY!

Create new dataset

Create new dataset

Create new dataset

Create new dataset

Create newTable (with CSV file)

Create newTable (with CSV file)

Create newTable (with CSV file)

Create newTable (with CSV file)

nemo,47,nemo@mulodo.com foo,30,foo@example.com bar,25,bar@example.com

sample.csv

Create newTable (with CSV file)

nemo,47,nemo@mulodo.com foo,30,foo@example.com bar,25,bar@example.com

sample.csv

Check “Table ID”

Check “Table ID”

Check “Table ID”

QUERY for your own table!!

Now you are ready to use

BigQuery.

NextInstall GCP SDK

Use command line tools

Characteristic of Google BigQuery.

Especially, Tips for business use.

Thank you!