Big Data Perspective (Company Information)
-
Upload
avkash-chauhan -
Category
Technology
-
view
83 -
download
2
Transcript of Big Data Perspective (Company Information)
We Make Hadoop Fly
The TeamNinja is the SolutionOurTeamWhoweare?
AvkashChauhan,Founder- DeepexpertiseindistributeddataplatformsincludingHadoop,withvariouspublicationsandspeakingengagements.WorkedatMicrosoft,Platfora&experienceincludes:
§17+yearsintechindustry,workedwith15+fortune#100companiesworldwide§Largescaledistributedapplicationdevelopmentandperformanceexpert§CoreDeveloperandproductimplementationspecialistforMicrosoftAzure,HDInsight&Platfora.§Microsoftdistinguishedengineer,keyarchitectonHadoopimplementationforAzure.
HenryOhara,Architect(Backendengineer)- Hasacareerofdesigninganddevelopingscalablebackendsystemsforover21+years.
§FoundingmemberofYahoo’smobilesearchteamservicingover30millionusers.§DevelopedarecommendationsystemforYahoo'smediasite.§Developedtop#1applicationtomanagesecureenterprisecommunicationinJapanformobile
HamidBehnam,UXEngineer–• Over7+yearsoffrontendengineeringanddevelopmentexperiencewithastronginterestinJavaScript• Expertincreatingcomplex&multilayerwebapplicationusingJavaScriptframeworks• Amazingworkethic
SalSferlazza,principleinvestor&chairmanofboard– LaunchCapital.co• Overall21+yearsintechindustryfromdevelopertoCTOtoCEO• WorkedatAndersonConsulting,QuestSoftware,SonicWALL,Dell• Total5exitsinlast10years,3oftheexitsareinMSPspace
Board
Engineering
Ninja is the SolutionWhatwehavebuilt?
OperationalAnalyticsforBigData
HeterogeneousMonitoringSmartThresholdsUnifiedAlertingForecasting
Recommendations
BIGDATAAPM
Ninja is the SolutionWhy?
ThereisnoAPMforbigdatastack
Enterprise
BIVendors
ValleyStartups
ITOrganizations
3rd PartyApplications
Ninja is the SolutionTrueIntegrationsMarketPlacement
DataPerpetrationDataQuality
DataLakeHadoop
ETLBatchProcessing Business
Intelligence MachineLearningBigDataAnalytics
PlatformMonitoring&Analytics
BigDataPlatforms
EnvironmentswhereBigDataplatformisdeployed
BigData
Ninja is the SolutionTrueIntegrationsSupportedComponents
PlatformMonitoring&Analytics
Ec2ContainerService
• Multi Vendor Monitoring• Smart Thresholds • Unified Alerting • Forecasting • Recommendations
Ninja is the Solution
ProprietaryAndConfidential
PlatformFeatures
The TeamNinja is the SolutionSummaryArchitectureComplete
HadoopCluster
SystemCollectionServer
HadoopV1CollectionServer
HadoopV2CollectionServer
WebServer TimeSeriesDB
SchedulerServer
AnalysisServer
RESTful Interface
AJAX
• MajorHadoopDistributions• Amazon&HDInsight• ClusterDeploymentplatform• OnPremise• AnyHadoopFarm
HTML5,CSS,JavaScript,KnockoutAndD3
PostgresDB
ServerinteractionoverREST(admin/communication)
KAFKAStreams Messaging
Server
Noninvasiveappliancedesign&Selfservingdeploymentmodel
Jetty+JAX-RS
Ninja is the SolutionIntegrationsComplete
HadoopDistributions
CloudBasedHadoop
DistributedFileSystems
DistributeddataProcessing
DistributedServices
Cluster &containerManagement
MachineLearning
*
*PrototypingPhase
*
*
Ninja is the SolutionOurTeamFeaturesCompleted• DataCollectionMicro-servicesconnectedoverRESTinterface
• Microserversfordatacollection• MicroserversfortunnelingoverSSH• Distributedserverarchitecturetosupportdistributeddeployment
• HadoopDistributions• ApacheHadoop• Hortonworks• Pivotal
• DynamicViews• Cross-clusterandindividualclusterviewsthroughdeep-divenavigation• Dynamicwidgetscreationanddeploymentforanycollecteddata• DynamicpagegenerationforvariousHadoopdistributionsandversions• DynamicgraphdetailsforbothsystemandHadoopdatacombined
• Dashboards• Dynamicdashboardsgenerationfromanydatapoint• Dashboardsharing• DashboardscheduleddeliveryasPDF/Imageoveremail
• Reporting• ReportgenerationfromanypageordashboardinPDForimageformat• CustomComplexreportingformultipointdataacrossmulticlusterenvironment• ScheduleddeliveryforanyreportoveremailinPDF/Imageformat
• Processdataanalysisperprocess• Exampleprocessi.e.Kafka,Zookeeperetc
Ninja is the SolutionOurTeamFeaturesCompleted(Cont..)• Queryenginetoaccessanydatafromanytableatanyduration,examplequeryincludes:
• DisplaySystemCPUloadaveragelast2hours• DisplaySystemMemoryHeapMemorytoplast1day• DisplaySystemNetworkInterfacerx Byteseth0today• DisplaySystemDiskSpaceUsageAveragethisweek• DisplaySystemDiskIOTimeSpentIOxvda1last2hours
• MapReduce• MapReducejobsanalysisindividualorbatch• MapReducejobsbatchmodeSLA
• DatacollectionforMapReduce2andSparkoverYARN*• ClusterUtilization
• In-depthclusterutilizationmetrics• Dynamicscheduleddeliveryforclusterutilizationreportsandalert
• HDFS• HDFSdataanalysis,visualization&reportingatfolderlevel
• Alerting• Scheduledalertsandnotificationforanydatapointcollected
• Deployment• Selfservingdeployment&RemotePatching
• SystemDataAnalysispermachine• CPU• Memory• Networkdataatinterfacelevel• DiskI/Odataatatpartitionlevel• DiskSpaceatmounteddisklevel *Onlydatacollection
Ninja is the SolutionIntegrationsRoadmap
HadoopDistributions
CloudBasedHadoop
DistributedFileSystems
DistributeddataProcessing
DistributedServices
Cluster &containerManagement
MachineLearning
*
*PrototypingPhase
*
*
Ninja is the SolutionOurTeamFeaturesRoadmap
• UseODPi (http://www.odpi.org)toconnectwithsupportedbigdataplatform• Selfservicemodeltoslice&diceanydatacollection• MapReduce&SparkSupportforYARN• EMR/AMISupport• OtherHadoopvendorsSupport(Cloudera &MapR)• CartridgedesignforCassandradatacollection• Recommendation• Dockerbasedapplicationdeployment• CostAnalysisMatrix• Forecasting
The TeamNinja is the SolutionSummaryDemo
ClusterMonitoring HDFSMonitoring MapReduceJobMonitoring&Analysis
DataAnalysis,Graphs(D3/C3)
Alerts&Notifications On-demand&ScheduledReporting
Dashboards, HOD,UI,Control Panel,
TimeSpan
DataCollectionSystem
(Total500+datapoints)
Serverdesignandcommunication ClusterUtilization SelfManagement Troubleshooting
Ninja is the SolutionSummaryContact
Founder&PrincipalBigDataPerspectiveLLC657MissionSt.Suite602,SanFrancisco,CA94105E:[email protected] M:650-713-9055
AvkashChauhan