teddy@aftership.com

Teddy ChanCEO& Co-Founder

Shipment tracking APISupporting 270 290+ carriers

Free Shipping APIsSingle API for 5 carriers

How do we find you?

Google BigQuery is great

- Charge only when you do query

- Super cheap in storage

- Super cheap in sending the data

Perfect for log everything

3 billions records / month

1,200 records / sec

API Log BigQuery

Goal

API Log BigQuery

It does not work!

- http RESTful- BigQuery API call limit

API Log Redis Streamer BigQuery

How do we solve?

Why Redis?

- TCP persistent connection- Fast, really FAST

What does the STREAMER do?

- Read data from redis- Send BATCH data to BQ- Pre / Post process data

Demo

http://www.slideshare.net/aftership

FAQ

OPEN SOURCEhttps://github.com/AfterShip/demo_streamer2bigquery