A Brief History Of Data
-
Upload
damien-dallimore -
Category
Data & Analytics
-
view
425 -
download
6
description
Transcript of A Brief History Of Data
![Page 1: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/1.jpg)
Copyright © 2014 Splunk Inc.
A brief history of dataDamien DallimoreWorldwide Developer Evangelist @ Splunk
![Page 2: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/2.jpg)
Who Am I
2
Worldwide Developer Evangelist @ Splunk
I codeI talk about coding
Community, Collaboration, Open
![Page 3: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/3.jpg)
From Aotearoa (New Zealand)
3
![Page 4: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/4.jpg)
Where did the “BIG DATA” come from
4
![Page 5: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/5.jpg)
5
Let’s go on a journey
![Page 6: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/6.jpg)
20,000 BC Arithmetic
6
We start to count things
![Page 7: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/7.jpg)
15,000 BC Cave Painting
7
We start to record data visually
![Page 8: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/8.jpg)
3,500 BC Written Language
8
We start to record and “transmit” knowledge
![Page 9: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/9.jpg)
2,500 BC Sumerian Calendar
9
We start to organize and track time
![Page 10: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/10.jpg)
1,250 BC Library at Thebes
10
We start to store data in mass
![Page 11: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/11.jpg)
1,150 BC Egyptian Maps
11
The origin of Google Maps
![Page 12: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/12.jpg)
1000 BC Era Math, Computation, Logic
12
We start to develop understanding through numbers
500 BC Pythagoras300 BC Euclid
And how numbers can be used to compute data
250 BC Archimedes100 BC Antikythera Mechanism
We start to classify objects and use logic to derive insights
350 BC Aristotle
![Page 13: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/13.jpg)
0 - 1600
13
78 Pliny : “all” the world’s knowledge captured105 Paper : bulk recording of data340 Codices : making data browseable with sections and indexes1350 Nicole Oresme : turning data into picture1453 Guttenberg : mass distribution of data
![Page 14: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/14.jpg)
1600 - 1900
14
1640 Napier : before logs there were logarithms 1662 Graunt : father of statistics 1796 Watt : recording data with a machine1801 Jacquard : programming !!1830 Babbage : the first mechanical and programmable computer 1844 Morse : data encodings1850 Reuter : first “WAN” (the CSMA/CD was a bit messy)1876 Dewey : data classification
![Page 15: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/15.jpg)
1900 - 2000
15
1930’s Fisher : modern statistics1936 Turing : the universal computer1950’s Programming Languages : Fortran et al1962 Tomlinson : first standard for Geo Data1963 ASCII : a standard for representing letters and numbers1969 ARPANET and other Protocols1970’s RDBMS : ETL , BI , Data Warehouses , I am your father1970’s/ 80’s Personal Computing : the foundations for alot of today’s data1982 TCP/IP standardized and the Internet came to be1989 The Web and HTML blink tags1991 Unicode : all languages captured
![Page 16: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/16.jpg)
To infinity and beyond
16
Web 2.0Google
Social NetworkingIOT
![Page 17: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/17.jpg)
17
Data Today ?
![Page 18: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/18.jpg)
18
![Page 19: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/19.jpg)
19
![Page 20: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/20.jpg)
20
![Page 21: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/21.jpg)
21
![Page 22: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/22.jpg)
22
![Page 23: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/23.jpg)
23
![Page 24: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/24.jpg)
24
![Page 25: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/25.jpg)
25
![Page 26: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/26.jpg)
26
![Page 27: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/27.jpg)
Data Stats Candy
27
Every day 2.5 quintillion bytes of data (1 followed by 18 zeros) are created
A full 90 percent of all the data in the world has been generated over the last two years
2.7 Zetabytes of data exist in the digital universe today.
Facebook stores, accesses, and analyzes 30+ Petabytes of user generated data
Akamai analyzes 75 million events per day to better target advertisements.
Decoding the human genome originally took 10 years to process; now it can be achieved in one week.
Data production will be 44 times greater in 2020 than it was in 2009
![Page 28: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/28.jpg)
Data Characteristics
28
VOLUMEVARIETY
VERACITYVELOCITY
![Page 29: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/29.jpg)
29
Data Tomorrow ?
![Page 30: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/30.jpg)
30
![Page 31: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/31.jpg)
31
DATAUNDERSTANDING
INSIGHTSACTIONS
![Page 32: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/32.jpg)
32
![Page 33: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/33.jpg)
33
How are we going wrangle this data ?
![Page 34: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/34.jpg)
Release the Developers
34
![Page 35: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/35.jpg)
New approaches to data platforms are needed
35
Traditional ETL / Data Warehouses = schema at write timeTo cope with data today = schema at read time
A Data language , the new SQL
Platforms that support an ecosystem of developers , content creators , data knowledge sharing
Open and easily extensible to cope with a variety of data sources and use cases, API oriented
Elasticity
![Page 36: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/36.jpg)
36
Make machine data accessible, usable and valuable to everyone.
![Page 37: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/37.jpg)
Spelunking
37
![Page 38: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/38.jpg)
Platform for Machine DataAny Machine Data
HA Indexes and Storage
Search and Investigation
Proactive Monitoring
Operational Visibility
Real-time Business Insights
CommodityServers
Online Services Web
Services
ServersSecurity GPS
Location
StorageDesktops
Networks
Packaged Applications
CustomApplicationsMessaging
TelecomsOnline
Shopping Cart
Web Clickstreams
Databases
Energy Meters
Call Detail Records
Smartphones and Devices
RFID
![Page 39: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/39.jpg)
Powerful Platform for Enterprise Developers
39
REST API
Build Splunk Apps Extend and Integrate Splunk
Simple XML
JavaScript
Django
Web Framework
JavaJavaScriptPython
RubyC#PHP
Data Models
Search Extensibility
Modular Inputs
SDKs
![Page 40: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/40.jpg)
Enough talkingDEMO TIME !!
![Page 41: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/41.jpg)
The Developer Opportunity in Data
41
It’s fun to make cool thingsGet a job , build a business , make money !Promote yourself , Promote your companyGet involved in community projectsDo GoodThink of new data sources and tap into themDemocratize dataDiscover new things & drive society forward
We talk alot about the how , what , where and who ….. but what about the WHY
![Page 43: A Brief History Of Data](https://reader034.fdocuments.in/reader034/viewer/2022051322/53ed99e28d7f7289708b5da2/html5/thumbnails/43.jpg)
Thankyou !