Applied Data Science - When Should We Change Our Website?
31
-
Upload
reid-ronnander -
Category
Data & Analytics
-
view
103 -
download
0
Transcript of Applied Data Science - When Should We Change Our Website?
Ronnander 2015
Ronnander 2015
Current situation:• We change our website when it “feels right”
Ronnander 2015
Current situation:• We change our website when it “feels right”
• Whenever we change our website, our
conversion rate increases
Ronnander 2015
Current situation:• We change our website when it “feels right”
• Whenever we change our website, our
conversion rate increases
• Over time the conversion rate decreases
Ronnander 2015
Our goals:
Ronnander 2015
Our goals:• We change the website when the data says
we should
Ronnander 2015
Our goals:• We change the website when the data says
we should
• We change the website to keep conversion
rates above a certain level
Ronnander 2015
Data:• Weekly conversion rate
Ronnander 2015
Data:• Weekly conversion rate
Science:• Test if the rate is below our threshold
Ronnander 2015
Data:• Weekly conversion rate
Science:• Test if the rate is below our threshold
Result:• Recommend the “right time” to change the website
Ronnander 2015
Ronnander 2015
Ronnander 2015
We decide to measure conversion rates by week
Ronnander 2015
We decide to measure conversion rates by weekWe decide on a threshold of 10%
Ronnander 2015
Each week we decide whether or not we should change the website
Ronnander 2015
Each week we decide whether or not we should change the website
If we have evidence that the conversion rate has dropped below 10%
we change the website
Ronnander 2015
Ronnander 2015
0%
2%
4%
6%
8%
10%
12%
14%
16%
18%
1 2 3 4
Co
nve
rsio
n R
ate
Week
Conversion Rate by WeekThreshold
Estimate
Bound
0%
2%
4%
6%
8%
10%
12%
14%
16%
18%
1 2 3 4
Co
nve
rsio
n R
ate
Week
Conversion Rate by Week
Threshold Estimate Bound
0%
2%
4%
6%
8%
10%
12%
14%
16%
18%
1 2 3 4
Co
nve
rsio
n R
ate
Week
Conversion Rate by Week
Threshold Estimate Bound
• Week 1 – We have no evidence to say that the conversion rate is less than 10%
0%
2%
4%
6%
8%
10%
12%
14%
16%
18%
1 2 3 4
Co
nve
rsio
n R
ate
Week
Conversion Rate by Week
Threshold Estimate Bound
• Week 1 – We have no evidence to say that the conversion rate is less than 10%
• Week 2 – We have no evidence to say that the conversion rate is less than 10%
0%
2%
4%
6%
8%
10%
12%
14%
16%
18%
1 2 3 4
Co
nve
rsio
n R
ate
Week
Conversion Rate by Week
Threshold Estimate Bound
• Week 1 – We have no evidence to say that the conversion rate is less than 10%
• Week 2 – We have no evidence to say that the conversion rate is less than 10%
• Week 3 – We have no evidence to say that the conversion rate is less than 10% however it appears we have a negative trend
0%
2%
4%
6%
8%
10%
12%
14%
16%
18%
1 2 3 4
Co
nve
rsio
n R
ate
Week
Conversion Rate by Week
Threshold Estimate Bound
• Week 1 – We have no evidence to say that the conversion rate is less than 10%
• Week 2 – We have no evidence to say that the conversion rate is less than 10%
• Week 3 – We have no evidence to say that the conversion rate is less than 10% however it appears we have a negative trend.
• Week 4 – We have evidence that the conversion rate is less than 10%
Ronnander 2015
Ronnander 2015
At week four we had the data and evidence to support a website change
Ronnander 2015
Ronnander 2015
How often will we need to change our website?
Ronnander 2015
How often will we need to change our website?
0%
2%
4%
6%
8%
10%
12%
14%
16%
18%
1 2 3 4 5 6
0%
2%
4%
6%
8%
10%
12%
14%
16%
18%
1 2 3 4 5
0%
2%
4%
6%
8%
10%
12%
14%
16%
18%
1 2 3 4 50%
2%
4%
6%
8%
10%
12%
14%
16%
18%
1 2 3 4
After four weeks After five weeks After six weeks After five weeks
Once we spot a trend we can adjust the development timeline to fit
Ronnander 2015
