Democratising data at the Financial Timesmatt.chadburn.co.uk/presentations/csv-conf-2016.pdf ·...
38
@commuterjoy Democratising data at the Financial Times Matt Chadburn, Principal Engineer, ft.com May 2016
Transcript of Democratising data at the Financial Timesmatt.chadburn.co.uk/presentations/csv-conf-2016.pdf ·...
@commuterjoy
Democratising data at the Financial Times
Matt Chadburn, Principal Engineer, ft.com May 2016
@commuterjoy
@commuterjoy
"It's about getting every part of the organisation to be more effective
with what they do and democratising data across the
organisation," he explained.
!"The challenge is to move beyond a centralised data function to a
decentralised one, where data becomes part of everyone's
responsibility.
!"It's much more than a technology challenge - creating reports and
saying 'there you go'. It's a training challenge, how we help
everyone understand the impact of what they do.
!"Certainly there's a core of experts who sit in the middle offering
expertise and rigour, but this is about having everybody learn
about our customers and ultimately this is a decentralised
capability."
@commuterjoy
source: opendefinition.org
@commuterjoy
• Users of data at the FT
• Systems we built to support them
@commuterjoy
@commuterjoy
@commuterjoy
@commuterjoy
@commuterjoy
@commuterjoy
Matthias Müller
Volkswagen
Germany
Emissions scandal
Frankfurt Stock Exchange
Automotive
located
headquartered inlisted on
industry
CEOabout
@commuterjoy
User
follows
USA
General Motors
lives in
works for
@commuterjoy
@commuterjoy
@commuterjoy
@commuterjoy
@commuterjoy
@commuterjoy
@commuterjoy
@commuterjoy
Lots of users !
Diverse use cases !
Single version of the truth !
No analytics system can do all this :(
@commuterjoy
Events Data WarehouseAPI
@commuterjoy
category:action
email:open
signup:subscribe
page:view
comment:post
page:scroll
payment:failure
…
account:update
@commuterjoy
Event
Context
User
Device
System
Category
Action
Time
@commuterjoy
@commuterjoy
Data WarehouseAPI
Client-side
Offline
Server (go, java, node)
Webhooks
AMP etc
@commuterjoy
Events API
SQS
Kinesis
Neo4J
Elastic Search
RedShift
Ontotext
Keen.io
Hadoop
Redis
Pusher.io
Google Sheets
@commuterjoy
Events Event streamAPI Enrichment
@commuterjoy
@commuterjoy
@commuterjoy
@commuterjoy
@commuterjoy
@commuterjoy
Events Event streamAPI Enrichment
Maxmind
Time
Device Atlas
Content API
AB Test
Propensity
Propensity
License API
RFV
myFT
@commuterjoy
caniuse.com
Events Event streamAPI Enrichment
Weather
Market prices
Offers
Demographics
QV
Freebase
timezone
Sentiment
sharedcount.com
@commuterjoy
Events Event streamAPI EnrichmentValidation
(in progress!)
@commuterjoy
• Focus on the users need
• Learnable (standards)
• Ease of use (APIs to get stuff in and out)
• Iterative (700+ production releases)
• Open for contribution
Democracy?
@commuterjoy
https://bit.ly/ft-csv-job
