BMDC: Utah Air Quality

@davidbgonzalezZiff.io

Boawp.comhttps://github.com/davidbgonzalez/bmdcfall2014data

??

??

WWWWW

Find: “\<this is what I'm looking for>”

Replace: %s/<leave me blank for the last thing I searched>/<replace with this>/

Scrape

txtwww

pdf

BIG

sql

BIG

● wc -l # 128438621 # Oh No's● Work with sample for flow● head -n 100000● Compress it

– Avros

– Parque

● Play with it hdfs

txt

Tools

● Beautifulsoup4 ← python● Vim + regex● Xlrd● CLI

– jq

– json2csv

– csvkit

– subsample