Log I am your father
-
Upload
hadoop-summit -
Category
Technology
-
view
345 -
download
0
Transcript of Log I am your father
![Page 1: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/1.jpg)
Copyright © 2016 Splunk Inc.
Log I am your father:search your data, you know it to be true
James Hodge,Principal Product Manager
![Page 2: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/2.jpg)
Spelunking:
Splunking:
to explore underground caves
to explore machine data
![Page 3: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/3.jpg)
Explore your IoT data
![Page 4: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/4.jpg)
4
The data we know and use
The available data we don’t know or use
![Page 5: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/5.jpg)
5
Nice, structured, tidy data
![Page 6: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/6.jpg)
It began with logs…
![Page 7: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/7.jpg)
7
Machine data time series, in motion, unstructured
![Page 8: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/8.jpg)
It can be big data…
![Page 9: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/9.jpg)
…and it is slightly crazy “data”…
![Page 10: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/10.jpg)
10
… and it is messy and lazy
![Page 11: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/11.jpg)
SECURITYIT OPERATIONS
CUSTOMER EXPERIENCERISK & COMPLIANCE
PRODUCT PERFORMANCEIOT
But it has many uses
and lots of untapped value
![Page 12: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/12.jpg)
There’s gold in that there data
![Page 13: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/13.jpg)
13
Make machine data accessible,usable and valuable to everyone.
13
![Page 14: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/14.jpg)
[2014-09-04-14.45.54.608000] proc_source="B24A", tmst_target="2013-09-04-14.45.54.724000", serv_id="ISS", proc_input="MAST", proc_target="B24H", interface_acq="BNET_1", interface_iss="02008", cod_msg="XJYZ", oper_rrn="XJYZ", card_id="52xxxxxxxxxxx", oper_amount="000000000050", oper_currency="978", oper_country="380", term_id="0059XXXX", circuito="", sett_merc="4722", bin_acq="XXXX", id_merc="32xxxxxxxxxx", prcode="XYZ", action_code="XXX", ......auth_rout_id="HISO_AUTH", msg_subst="", ndg="00000xxxxxxxx", station_acq="STA-BNET-MI1", acceptor="A COOL SHOP", tmst_ins="2013-09-04-14.48.56.277466", ...
Mastercard Circuit
Client IDMerchant name
AmountCard ID (masked)
Merchant category IDMerchant ID
There’s gold in that there data
![Page 15: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/15.jpg)
Turning Machine Data Into Business Value
15
Platform for Machine Data
ApplicationDelivery
ITOperations
Security,Complianceand Fraud
BusinessAnalytics Industrial
Data andInternet of
Things
Security,Complianceand Fraud
BusinessAnalytics Industrial
Data andInternet of
Things
![Page 16: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/16.jpg)
HA/DR Admin Data Security Apps SDKs/APIsScale
CollectData
IndexData
Enrich Data
Search & Explore
Analyze& Predict
Report &Visualize
Alert & Action
16
Fully Integrated Enterprise Platform
![Page 17: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/17.jpg)
Technology Telecommunications Travel and Leisure
Education
Healthcare
Energy and Utilities
Manufacturing
Financial Services and Insurance
Media
Proven at 11,000+ Customers in 100 CountriesMore Than 80 of the Fortune 100
Retail
Cloud and Online Services
Government
![Page 18: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/18.jpg)
Splunk Capabilities for IoT
Data Ingest at Scale
PartnerEcosystem
Advanced Analytics and Visualization
Sense and Respond
![Page 19: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/19.jpg)
IoT and Industrial Machine Data
DevelopVisualize PredictAlertSearch
Engineers Data Analysts
Security Analysts
Business Users
Native InputsTCP, UDP, Logs, Scripts, Wire, Mobile
SDKs and APIsJava, JS, C#, Python, Ruby, PHP
Modular InputsMQTT, AMQP, COAP, REST, JMS
HTTP Event CollectorToken Authenticated Events
Real-time
Technology PartnershipsKepware, ThingWorx, Cisco, Palo Alto
MaintenanceInfo
AssetInfo
DataStores
External Lookups/Enrichment
OT
Industrial Assets
IT
Consumer and Mobile Devices
![Page 20: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/20.jpg)
HTTP Event CollectorSupports DevOps and IoT data analysis needs at scale
20
• Standard API and logging libraries send events directly to Splunk• Libraries integrated with popular platforms and services
Scales to Millions of Events/Second
EVENT COLLECTOR API
![Page 21: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/21.jpg)
Advanced Analytics
Splunk ML Extensions• New/Enhanced Commands: Data Sampling,
Fit, Apply, Summary, Predict• Access to Python Data Science Library• Model storage and export to production
Splunk ML App• Step-wise guidance to create, test and deploy
custom ML models • Purpose-built visualizations• Sample data and best practices
Optimizing enterprise operations with predictive ML analytics
![Page 22: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/22.jpg)
ADVANCED VISUALIZATIONS
![Page 23: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/23.jpg)
How VW Visualizes Connected Car Data
VW Data LabsConnected Car program
Post-sales big data visualization
Customer loyalty & retention
IoT analysis & prediction of customer needs
![Page 24: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/24.jpg)
Sense and RespondUse Splunk Alerts and Custom Alert Actions to trigger & automate workflows
● Allows packaged integration with third-party applications
● Simple admin/user configuration● Developers can build, package, and
publish alert actions within an app● Growing list of integrations available
24
![Page 25: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/25.jpg)
Splunk’s IoT and Industrial Partner Ecosystem
25
SDKs UI
Ingest and Platforms
IoT and ICS SecurityAdvanced Analytics and ML Custom User Interfaces
Services and Delivery
![Page 26: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/26.jpg)
Customer Success
![Page 27: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/27.jpg)
Use Cases
27
Monitoring,
Diagnostics
Security, Safety
& Compliance
Preventative Maintenance
![Page 28: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/28.jpg)
28
Zebra Technologies CorporationFounded in 1969, Zebra offers customers a complete end-to-end solution – from mobile computers and scanners to specialty printers, RFID, software and services – for identifying, tracking and managing critical assets, people and transactions.
Headquarters: Lincolnshire, ILOffices: 122 offices across 81 countriesEmployees: 7000+ globally
Market Capitalization: $5,640M1
Revenue: $2,275M 1
Profit (EBITDA): $439M 1
Mobile ComputingPrintersData Capture (barcode scanners)
RFIDLocation solutionsWireless LAN
About
Global presence
Financials
Products
2,000+ NFL players tagged and tracked since 2014, generating more that 68+ billion bytes of player position data
1 Zebra corporate fact sheet, summer 2015
![Page 29: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/29.jpg)
29
Challenge for Zebra Printer DivisionLimited data analysis capabilities . . .
• Data not being collected for long-term use• Reports not being generated• Data not shareable across departments• Changes to traditional database took a long time• Minimal analytics capabilities (e.g., Excel)• Unable to scale to increasing data volumes
Reactive mindset: Teams were reacting to issues after they occurred
Little continuous improvement: Company was unable to leverage data to improve• Product design• Manufacturing• Repair and service
Resulted in . . .
Goal is to design a system that:• Captures complete device and test history• Is flexible and can be easily modified• Is capable of robust analytics, trending, & alerting capabilities• Can easily share data to enable better business decisions
![Page 30: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/30.jpg)
30
Solution developed by Zebra
Test HistoryHistory of the test performed by
the test utility on the device
Test DataAny data deemed necessary to
log and store
Device (Printer) DataRaw data output of device
(4 data formats)
1 manufacturing center8 service centers3 reconfig centers30+ applications600+ computers globally Allows for real-time view of production data to address low yields
and drive quality improvements
Communicates product performance over time, leveraging statistical methods to baseline product performance
Anomaly detection analytics utilized to capture known defect patterns that prevent product shipment
Dashboard
Reports
Analytics
![Page 31: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/31.jpg)
31
Value realized by Zebra
Quick access to test history and device data allows for enhanced customer experience
Faster customer resolution
Enhanced product development
Reduced return rate
Improved business decisions
Reduced cost
Data being captured early in development drives improvements in design
Anomaly detection analytics used to capture known defect patterns that prevent product shipment
Data being easily shared across Quality, Product, Technical Support, and Services enables data-driven decision making process
Quantified cost reduction of $90,000 per year through media reduction alone
1
2
3
4
5
![Page 32: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/32.jpg)
Planes, Trains and Automobiles(and Coke, Zombies, Floods, Buildings, Crops and Medicine)
![Page 33: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/33.jpg)
Data-driven RefreshmentAggregate machine data from
Coca-Cola freestyle® machines
Insights into customer interactions and decisions
Reduced Downtime andIncreased Consumer Satisfaction
Vending machine performance and diagnostics
![Page 34: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/34.jpg)
More people buy Coke just before The Walking
Dead
![Page 35: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/35.jpg)
35
How Gatwick Airport Ensures Better Passenger Experience With Splunk Cloud
On-time efficiency & dramatic queue reduction with 925 flights per day
Real-time, predictive airfield analytics - mobile app & CEO’s apple watch
Data from airport gates, board pass scans, x-ray, travel, passenger flow
![Page 36: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/36.jpg)
![Page 37: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/37.jpg)
![Page 38: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/38.jpg)
Saving The US Rail Industry A Billion Dollars And 250 Million Acres Of Trees in CO2
Train sensor dataFuel savingsBetter trained drivers
![Page 39: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/39.jpg)
24 Hour DB Hackathon
Highlight defect impact & rail construction issues
Predictive maintenance & reduced disruption
Transport, infrastructure, environment & journey data
How Deutsche Bahn Analyzed Tracks in 24 Hours
![Page 40: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/40.jpg)
![Page 41: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/41.jpg)
Top ten types of notifications about issues that have occurred in the transportation infrastructure
![Page 42: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/42.jpg)
What kinds of defects occur and quantify how big the deviations are
Correlated with materials of the track sleepers: concrete (“Beton”) vs. wood (“Holz”)
Track deviations with indication notifications. Blue bars denote “no known issues”
![Page 43: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/43.jpg)
Width of Sankey bar shows amount of track deviations between different destinationsFulda and Frankfurt has high track deviations (it is the widest bar)
Indicates the need for upcoming repair, maintenance and possible renewal
![Page 44: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/44.jpg)
Water Level Sensor
Water Level Sensor Data in
Monitors
Real-time
Data
In Advance of Future Impending Flood
Situations
Alert Mobile Devices
![Page 45: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/45.jpg)
Robot Analytics to Reduce Costs in the Supply Chain
4% Increased Throughput per Distribution Center
Aggregate machine data from robots
Failure pattern detection and reporting
Preventative maintenance scheduling
![Page 46: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/46.jpg)
46
Remote Freight TrainMonitoring
Energy Efficiency Calculations
Driving Strategy Recommendations
Over $1 Billion
Saved
![Page 47: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/47.jpg)
Sensor Data Delivering Millions of Dollars in Energy Savings
![Page 48: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/48.jpg)
+Content browsed, purchased and
watched. All tracked by time and MAC address
Customerbehavior analytics
Customerprofile and MAC address / device
assignments
Understanding Customer Behavior
![Page 49: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/49.jpg)
IoT WORKLOADS
Blurring the Lines Between Digital & Physical
Security Ops
Center
Business Ops
Center
IT Ops Center
CLOUD WORKLOADS ENTERPRISE IT WORKLOADS
ADVANCED ANALYTICSRAPID SOLUTION DEVELOPMENT
DATA INGEST AT SCALE
![Page 50: Log I am your father](https://reader031.fdocuments.in/reader031/viewer/2022030305/587287911a28abc7068b76fb/html5/thumbnails/50.jpg)
Thank You