Hadoop Israel - HBase Browser in Hue

of 37 /37
HBASE BROWSER IN HUE Abraham Elmahrek Hadoop Israel Apr 23, 2014

Embed Size (px)

description

Learn about the HBase browser in Hue, the UI for Apache Hadoop. Presented by Abraham Elmahrek at Hadoop Israel www.meetup.com/HadoopIsrael/events/161701092/ Find out everything you need about Hue at http://gethue.com

Transcript of Hadoop Israel - HBase Browser in Hue

  • HBASE BROWSER IN HUE Abraham Elmahrek Hadoop Israel Apr 23, 2014
  • WHAT IS HUE? WEB INTERFACE FOR MAKING HADOOP EASIER TO USE Suite of apps for each Hadoop component, like Hive, Pig, Impala, Oozie, Solr, Sqoop2, HBase...
  • VIEW FROM 30K FEET Hadoop Web Server You and even that friend that uses IE9 ;)
  • ECOSYSTEM PIG JO B BRO W SER JO B DESIG N ER O O ZIEH IVE IM PALA M ETASTO RE BRO W SER SEARC HH BASE BRO W SER SQ O O P ZO O KEEPER U SER ADM INDB Q U ERY SPARKH O M E ... FILE BRO W SER
  • YARN JobTracker Oozie Pig HDFS HiveServer2 Hive Metastore Cloudera Impala Solr HBase Sqoop2 Zookeeper LDAP SAML Hue Plugins APPS
  • TARGET OF HUE GETTING STARTED WITH HADOOP BEING PRODUCTIVE EXPLORING DIFFERENT ANGLES OF THE PLATFORM ! LET ANY USER FOCUS ON BIG DATA PROCESSING
  • OPEN SOURCE 3000+ COMMITS 35 CONTRIBUTORS 665 STARS 221 FORKS ! github.com/cloudera/hue
  • THE CORE TEAM PLAYERS team.gethue.com ABRAHAM ELMAHREK ROMAIN RIGAUX ENRICO BERTI CHANG BEER
  • TALKS Meetups and events in NYC, Paris, LA, Tokyo, SF, Stockholm, Vienna, San Jose, Singapore Coming up in London, West coast AROUND THE WORLD RETREATS Nov 13 Koh Chang, Thailand May 14 Curaao, Netherlands Antilles
  • FAST PACE LAST 30 DAYS 41 issues created and 38 resolved. Core team + Community
  • NEW APPS IN 6 MONTHS PIG JO B BRO W SER JO B DESIG N ER O O ZIEH IVE IM PALA M ETASTO RE BRO W SER SEARC HH BASE BRO W SER SQ O O P ZO O KEEPER U SER ADM INDB Q U ERY SPARKH O M E ... FILE BRO W SER
  • TREND: GROWTH gethue.com
  • HISTORY HUE 1 Desktop-like in a browser, did its job but pretty slow, memory leaks and not very IE friendly but denitely advanced for its time (2009-2010).
  • HISTORY HUE 2 The rst at structure port, with Twitter Bootstrap all over the place.
  • HISTORY HUE 2.5 New apps, improved the UX adding new nice functionalities like autocomplete and drag & drop.
  • HISTORY HUE 3 Transition to the new UI, major improvements and new apps.
  • HISTORY HUE 3.5+ Where we are now, new UI, several new apps, the most user friendly features to date.
  • BIG DATA MEANS HARDER SEARCH PROBLEMS. HBASE APPLICATION DEVELOPMENT WOES. UNDERSTANDING DATA BETTER.
  • PROBLEMS LACK OF ACCESSIBILITY LACK OF FAMILIARITY USABILITY CHALLENGES
  • DESIGN CHALLENGES PROBLEM HBase is a key-value store, not a traditional RDB SOLUTION Collapse Sparse Data across HBase Tables
  • FLEXIBLE SEARCH BAR PROBLEM Need high-level tool for complex data fetching & analysis SOLUTION Simple custom query language Supports HBase lter language Supports selection & Copy + Paste, gracefully degrades in IE Autocomplete Help Menu Row$Key$ Scan$Length$ Prex$Scan$ Column/Family$Filters$ Thri=$Filterstring$ Searchbar(Syntax(Breakdown(
  • DIVING DEEPER DOWN LOW Thrift with custom connection pooling with socket multiplexing to improve thrift calls. Router to streamline front end requests. UP TOP KnockoutJS, JQuery, DOM caching, etc. for a dynamic front end. client = thrift_util.get_client(TCLIService.Client, query_server['server_host'], query_server[server_port], ) client.mutateRow() client.scannerOpenWithScan() client.scannerGetList() ! api = HbaseApi() api.putColumn() api.deleteColumn() api.bulkUpload()
  • SCALE PROBLEM Performance on Millions of Columns SOLUTION Lazy loading & Truncation using Thrift FilterString Stream raw data cached MEM > DOM elements dynamically generated b64encode binary data to preserve during ASCII dump Detect schema on preview reading byte headers Capped, Lazy Loaded & Bound to DOM via MVVM Pattern IT DOES.
  • ZOOKEEPER BROWSER TECHNICAL DETAILS Communicate with Contrib. REST API View Node Stats View and Edit Node Data And and Remove ZNodes
  • QUERY HBASE WITH HIVE HIVE EDITOR
  • QUERY HBASE WITH PIG PIG EDITOR
  • INDEX HBASE AND SEARCH IT HUE LILY INDEXER CLOUDERA SEARCH
  • DEMO TIME
  • SDK
  • ROADMAP CDH 5 with Hue 3.5+ http://gethue.com/ hadoop-tutorial-new- impala-and-hive- editors/ CDH 5.1 with Hue 3.6: CHARTS, Yarn, Spark, SEARCH create index, Load data, Dashboard, UX ... your idea? Inter-app integration (e.g. schedule daily Hive query in one click), versioning, export/import. Oozie revamp, dashboard, reporting Come talk to us about your use cases and what you would like to see next in Hue! Hue 3.5 OSS release http://gethue.com/ hue-3-5-and-its- redesign-are-out/ DECEMBER 2013 Q1 2014 Q2 2014 AFTER
  • MISSED SOMETHING? learn.gethue.c
  • MISSED SOMETHING? demo.gethue.co
  • LINKS DEMO http://demo.gethue.com TWITTER @gethue USER GROUP [email protected] WEBSITE http://gethue.com LEARN http://learn.gethue.com
  • GET HUE Try in advance the latest and greatest but youll have to configure everything on your own. Get to play with Hue and various Hadoop components in 5 minutes. Its a self contained CDH environment ready to use. Newer version than HDP, close to the original 2.5 minus apps like HBase, Impala, Sqoop, Search. The newest addition, ships Hue 3.0 through the GreenButton products. Stable and highly tested releases perfectly integrated with the Hadoop ecosystem, automagically configured by Cloudera Manager. In HDP theres an old forked version of Hue 2.3. CLOUDERAS CDH TARBALL CLOUDERAS DEMO VM HORTONWORKS* MAPR* HP CLOUD* * YOUR MILEAGE MAY VARY. BIGTOP EMBEDDED/DEMO IN IND. COMPANIES
  • WHAT ARE YOUR USE CASES? WHICH COMPONENTS DO YOU USE? WHAT WOULD YOU LIKE TO SEE IN HUE? INTERESTED IN CONTRIBUTING? WANNA SAY HELLO? DO YOU WANT A TAILOR MADE TEAM RETREAT? [email protected] GETHUE.COM
  • ! THANK YOU www.gethue.com
  • HISTORY HUE 3 ALPHA Proposed design, didnt make it.