Viafoura's Big Data Use Case

16
BIG DATA OVERVIEW AND HOW WE MADE OUR CHOICES NO REALLY, IT’S NOT THAT BORING

description

 

Transcript of Viafoura's Big Data Use Case

  • 1. BIG DATA OVERVIEW AND HOW WE MADE OUR CHOICES NO REALLY, ITS NOT THAT BORING

2. WHO AM I ANYWAY? Engage, Discover, Monetize. Bio-Informatics Data Scientist Employee Retentions Analytics Data Warehouse Specialist System Operations / DevOps Founder & Lead Technologist Real Estate Investor and Broker Presenter, Speaker, Organizer Data Scientist & Architect 00101011110101101010100101010101101101010101000111011010010101111010110101010010101010110110101010100011101101101101010101010010101111010110101010010101010110110 3. A LITTLE BIT OF HISTORY Engage, Discover, Monetize. 00101011110101101010100101010101101101010101000111011010010101111010110101010010101010110110101010100011101101101101010101010010101111010110101010010101010110110 I used to walk barefoot, uphill both ways 4. THEORY OF EVOLUTION Engage, Discover, Monetize. Blade Servers Cheaper Storage Google, Youtube Cloud Computing Amazon Netflix SSD 00101011110101101010100101010101101101010101000111011010010101111010110101010010101010110110101010100011101101101101010101010010101111010110101010010101010110110 Move to available data for EVERYONE 5. WHOS ON DECK? Engage, Discover, Monetize. THE HALL OF FAMERS!!! MySQL (DB2, Oracle, Microsoft, etc) Teradata, SAS, SAP, Informatica GENERATION X, Y AND BEYOND!!! BerkelyDB (Oracle) Lotus Domino (IBM) Neo4J Memcached (and MemcacheDB) CouchDB (inspired by Lotus Domino) Google BigTable DynamoDB (Amazon) Redis Riak HP Vertica Cassandra Elasticache 00101011110101101010100101010101101101010101000111011010010101111010110101010010101010110110101010100011101101101101010101010010101111010110101010010101010110110 Rookie of the Year to Hall of Famer 6. 00101011110101101010100101010101101101010101000111011010010101111010110101010010101010110110101010100011101101101101010101010010101111010110101010010101010110110 BIG DATA OR DATA-WAREHOUSE??? Engage, Discover, Monetize. 7. 00101011110101101010100101010101101101010101000111011010010101111010110101010010101010110110101010100011101101101101010101010010101111010110101010010101010110110 HOW FAST DO YOU NEED TO KNOW? HOW SURE DO YOU WANT TO BE? Engage, Discover, Monetize. REAL TIME ANALYTICS / DISPLAY BUSINESS INTELLIGENCE HBase Mahout YARN Dremel Pregel Google Map Reduce Hadoop Amazon EMR 8. CAP THEOREM Engage, Discover, Monetize. 00101011110101101010100101010101101101010101000111011010010101111010110101010010101010110110101010100011101101101101010101010010101111010110101010010101010110110 9. 00101011110101101010100101010101101101010101000111011010010101111010110101010010101010110110101010100011101101101101010101010010101111010110101010010101010110110 HOW WE CHOSE A FIT Engage, Discover, Monetize. 10. 00101011110101101010100101010101101101010101000111011010010101111010110101010010101010110110101010100011101101101101010101010010101111010110101010010101010110110 HOW WE CHOSE A FIT Engage, Discover, Monetize. 11. 00101011110101101010100101010101101101010101000111011010010101111010110101010010101010110110101010100011101101101101010101010010101111010110101010010101010110110 HOW WE CHOSE A FIT Engage, Discover, Monetize. 12. 00101011110101101010100101010101101101010101000111011010010101111010110101010010101010110110101010100011101101101101010101010010101111010110101010010101010110110 HOW WE CHOSE A FIT Engage, Discover, Monetize. 13. 00101011110101101010100101010101101101010101000111011010010101111010110101010010101010110110101010100011101101101101010101010010101111010110101010010101010110110 HOW WE CHOSE A FIT Engage, Discover, Monetize. 14. 00101011110101101010100101010101101101010101000111011010010101111010110101010010101010110110101010100011101101101101010101010010101111010110101010010101010110110 Engage, Discover, Monetize. HOW WE CHOSE A FIT 15. 00101011110101101010100101010101101101010101000111011010010101111010110101010010101010110110101010100011101101101101010101010010101111010110101010010101010110110 WHATS ON THE HORIZON? Engage, Discover, Monetize. DATASTORE + SEARCHING ANALYTICS + INSIGHT RESEARCH & DEVELOPMENT 16. 00101011110101101010100101010101101101010101000111011010010101111010110101010010101010110110101010100011101101101101010101010010101111010110101010010101010110110 GET IN TOUCH Engage, Discover, Monetize.