Brian MoranWherein our intrepid explorer provides an account of many curious encounters with data...

28 September 2016

Adapted from http://www.anlytcs.com/2014/01/data-science-venn-diagram-v20.html

Data Science

Mathematics and Statistics

Research Methods

UNICORN

Machine Learning

Subject Matter Expertise

Programming Skills

Computer Science

What Does it Involve?

Business Understanding

Data Understanding

Data PreparationModelling

Evaluation

Deployment

What Does it Do?

The CRISP data mining process

Check email

Naive Bayes

Internet search

PageRank

Watch Netflix

Boltzman machine

Buy lunch

Artificial neural net

Use Sat-Nav

Dijkstra’s algorithm

Apply for a loan

Decision trees

Shop on Amazon

Matrix factorization

Get a letter

k-means clustering

Diagrams (but not text) from: http://machinelearningmastery.com/a-tour-of-machine-learning-algorithms/

Data Science Applications are Everywhere

Welcome back Suzy

Welcome back Jon

Recommend money advice

Alert

Close

82%Likelihood:

Recommend alternative

Alert

Close

90%Chance of broken appointment:

Better alternative:

2:00pm - 5:00pm

Recommend incentives

Advice

Close

Tenancy value:

Negative Low High

One click repair adactus

1. Repairrequest Delay Staff

checkStaffinput

Repairordered

2. Repairrequest

Repairordered

3. Repairrequest

MachineLearning

Staffcheck

Staffinput

Repairordered

RepairorderedProblem? N

Y

Options for Back-office Repairs Process

Proof of concept 1: Natural language processing with Naive Bayes…

TRADEREPAIR REQUEST

Check web form

Naive Bayes

“Our ba th room is leaking through the ceiling onto the stairs and the ceiling is wet through along with the walls where the taps are mounted. And puddle on the stairs”

PLUMBER

Check web form

Naive Bayes

Check web form

Naive Bayes

bathroomleaking

ceilingstairsceiling

wet

walls taps

stairs

PLUMBER

-£300,000

-£225,000

-£150,000

-£75,000

£0

£75,000

£150,000

£225,000

£300,000

10% channel shift 20% channel shift 30% channel shift 40% channel shift 50% channel shift 60% channel shift 70% channel shiftNet c

ost /

ben

efit o

f rep

airs

sel

f-ser

vice

90% accuracy

10% accuracy

50% accuracy

‘Friction Free’ Self Service: Avoiding Pyrrhic Victories

Proof of concept 2: Clustering with K-means…

CUSTOMISE SERVICES & INFORMATION

Assign to cluster

K-means clustering

Blocked means of escape

Alert

Close Blocked for one day

56 King StreetLeighWN7 4LJ

Unusual transaction

Alert

Close

Audit file ready

Advice

Close

Values beginning with

Highlighted for attention

1166

Proof of concept 3: Identifying unnatural numbers with Benford’s Law…

Find outliers

Benford’s Law

1 2 3 4 5 6 7 8 9

6’0”

5’0”

4’0”

3’0”

5’6”

4’6”

3’6”

6’0”

5’0”

4’0”

3’0”

5’6”

4’6”

3’6”I’m going to beat Benford’s Law

I’m going to beat Benford’s Law

I’m going to beat Benford’s Law I’m going to beat

Benford’s Law

6’0”

5’0”

4’0”

3’0”

5’6”

4’6”

3’6”

6’0”

5’0”

4’0”

3’0”

5’6”

4’6”

3’6”

BUSTED!

BUSTED!

BUSTED!

CRIMINAL GENIUS!

Payment cycle broken

Alert

Close

Proof of concept 4: Forecasting arrears with Holt-Winters…

Forecast time series data

Holt-Winters Triple Exponential Smoothing

Staffing forecast ready

Advice

Close

Recommend replacement

Advice

Close

97%Likelihood:

Boiler beyond economic repair

Optimal time to sell property

Alert

Close

Housing Association Costs

Proof of concept 5: Tenancy survival with Kaplan-Meier…

Survival Analysis

Kaplan-Meier Estimator

S(t)=1−F(t) = Prob(T≥t)

Histogram of Tenancy Length (Current Tenants)

A Simple Model of Tenancy Lengths

Tenancy Length (years)

Tenancy Length (years)

A Simple Model of Tenancy Lengths

Tenancy Length (years)

Unlikely to need component

replacements

Likely to needcomponent replacements

Even Simple Models can Lead to Interesting Thoughts

RECEPTIONCONTACT CENTRE

Getting Technology isn't the Same as Really Getting it

Digital front of office Digital back of office +

Really Getting Channel Shift Choice

Brian Moran

brian.moran@ adactushousing.co.uk