Succeeding with OpenStack in the Enterprise (OpenStack Summit Austin 2016)
Succeeding with Enterprise Software Security Key Performance ...
Transcript of Succeeding with Enterprise Software Security Key Performance ...
![Page 1: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/1.jpg)
SESSION ID:
Succeeding with Enterprise Software Security Key Performance Indicators
ASEC-T08
Rafal M. Los Principal, Strategic Security Services
HP Enterprise Services @Wh1t3Rabbit
![Page 2: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/2.jpg)
Introduction to Key Performance Indicators (KPIs)
![Page 3: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/3.jpg)
#RSAC
Reporting on progress is tricky
3
![Page 4: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/4.jpg)
#RSAC
If you spend $1M, then…?
4
![Page 5: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/5.jpg)
#RSAC
First things first… Who here reports metrics?
5
![Page 6: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/6.jpg)
#RSAC
How many metrics do you track?
6
![Page 7: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/7.jpg)
#RSAC
I was once a victim of metrics
7
![Page 8: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/8.jpg)
#RSAC
Do your metrics give you insight?
8
![Page 9: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/9.jpg)
#RSAC
KPIs do.
9
![Page 10: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/10.jpg)
#RSAC
KPI = Key Performance Indicator
![Page 11: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/11.jpg)
#RSAC
A key performance indicator (KPI) is a measure of performance, commonly used to help an organization define and evaluate how successful it is, typically in terms of making progress towards its long-term organizational goals.
11
![Page 12: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/12.jpg)
#RSAC
…but implies you have long-term organizational goals!
12
![Page 13: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/13.jpg)
#RSAC
TL;DR: “Are you succeeding?”
13
![Page 14: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/14.jpg)
#RSAC
..and how much, relative to goals?
14
![Page 15: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/15.jpg)
#RSAC
Trademarks of good KPIs:
15
![Page 16: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/16.jpg)
#RSAC
1) Show relative distance to a goal
16
![Page 17: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/17.jpg)
#RSAC
2) Establish relevance to org
17
![Page 18: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/18.jpg)
#RSAC
3) Establish relevance to security
18
![Page 19: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/19.jpg)
#RSAC
>> context <<
19
![Page 20: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/20.jpg)
metrics vs KPIs
![Page 21: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/21.jpg)
#RSAC
How do you convey “improving”?
![Page 22: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/22.jpg)
#RSAC
Improvement as a result of effort
22
![Page 23: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/23.jpg)
#RSAC
Easy right? So why are we so bad at it?
23
![Page 24: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/24.jpg)
#RSAC
More importantly…
24
![Page 25: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/25.jpg)
#RSAC
..how do you define success?
25
![Page 26: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/26.jpg)
#RSAC
Study the following graph:
26
![Page 27: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/27.jpg)
#RSAC
27
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
Q1 2012 Q2 2012 Q3 2012 Q4 2012 Q1 2013 Q2 2013 Q3 2013 Q4 2013
Issues by OWASP Top 10
A5 A4 A3 A2 A1
![Page 28: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/28.jpg)
#RSAC
What does it show?
28
![Page 29: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/29.jpg)
#RSAC
Look again…
29
![Page 30: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/30.jpg)
#RSAC
30
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
Q1 2012 Q2 2012 Q3 2012 Q4 2012 Q1 2013 Q2 2013 Q3 2013 Q4 2013
Issues by OWASP Top 10
A5 A4 A3 A2 A1
A B D C
![Page 31: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/31.jpg)
#RSAC
A: Implemented mandatory testing
31
![Page 32: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/32.jpg)
#RSAC
B: Major acquisition
32
![Page 33: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/33.jpg)
#RSAC
C: Integration into primary dev cycle
33
![Page 34: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/34.jpg)
#RSAC
D: Switched s/w sec testing tools
34
![Page 35: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/35.jpg)
#RSAC
Clearly, the graph is inadequate
35
![Page 36: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/36.jpg)
#RSAC
Raw data:
36
Q1 2012 3575 135 4387 135 237
Q2 2012 3250 87 4357 31 219
Q3 2012 2978 12 3648 12 35
Q4 2012 4208 141 7989 47 187
Q1 2013 4189 109 6897 41 24
Q2 2013 2138 71 5867 39 23
Q3 2013 1378 14 2807 31 28
Q4 2013 2366 51 3879 38 31
A1 A2 A3 A4 A5
![Page 37: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/37.jpg)
#RSAC
Q1-Q2 2013: 49% decrease in A1 Q3-Q4 2013: 72% increase in A1
37
![Page 38: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/38.jpg)
#RSAC
Clearly this is data without context
38
![Page 39: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/39.jpg)
#RSAC
This shows no impact
39
![Page 40: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/40.jpg)
Defining effective KPIs
![Page 41: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/41.jpg)
#RSAC
What makes a good KPI?
41
![Page 42: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/42.jpg)
#RSAC
1. Relative distance to goal 2. Relevance to organization 3. Relevance to security
![Page 43: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/43.jpg)
#RSAC
Focus on 4 key SwSec areas
43
![Page 44: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/44.jpg)
#RSAC
“Impact to effort”
44
![Page 45: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/45.jpg)
#RSAC
Impact of a security item to the overall effort of the project
45
![Page 46: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/46.jpg)
#RSAC
[security item] [dev effort]
46
![Page 47: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/47.jpg)
#RSAC
Security items (examples) • static analysis process • dynamic analysis process • integrating testing tools • developer awareness
47
![Page 48: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/48.jpg)
#RSAC
Development effort • person-hours required to
complete existing task
48
![Page 49: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/49.jpg)
#RSAC
“By adding a dynamic testing process we initially added 25% effort but over 4 quarters now only add 10%” – AppSec Prog Mgr
49
![Page 50: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/50.jpg)
#RSAC
50
0%
5%
10%
15%
20%
25%
30%
35%
Q1 2012 Q2 2012 Q3 2012 Q4 2012 Q1 2013 Q2 2013 Q3 2013 Q4 2013
I2E (additional person-hours)
![Page 51: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/51.jpg)
#RSAC
We’re showing that we’re impacting the AppDev process less over time
51
![Page 52: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/52.jpg)
#RSAC
Doesn’t tell us if it’s helping security …
52
?
![Page 53: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/53.jpg)
#RSAC
“Impact to release”
53
![Page 54: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/54.jpg)
#RSAC
Impact of a security item to the release timeline
54
![Page 55: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/55.jpg)
#RSAC
[security item] [release timeline]
55
![Page 56: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/56.jpg)
#RSAC
Security items (examples) • integrating security testing early
in development • providing templates for ‘fixes’ • defining pre-built code modules
56
![Page 57: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/57.jpg)
#RSAC
Release timeline • person-hours required to
complete existing task
57
![Page 58: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/58.jpg)
#RSAC
“We were able to show that we could release faster if security was involved earlier on in development” – AppSec Prog Mgr
58
![Page 59: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/59.jpg)
#RSAC
59
0
10
20
30
40
50
60
70
80
90
100
Q1 2012 Q2 2012 Q3 2012 Q4 2012 Q1 2013 Q2 2013 Q3 2013 Q4 2013
I2R (hours additional avg/project)
![Page 60: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/60.jpg)
#RSAC
We’re showing that we’re impacting the release process less over time
60
![Page 61: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/61.jpg)
#RSAC
“Impact to uptime”
61
![Page 62: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/62.jpg)
#RSAC
Impact of a security item to the uptime of the application/service
62
![Page 63: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/63.jpg)
#RSAC
[security item] [uptime]
63
![Page 64: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/64.jpg)
#RSAC
Security items (examples) • continuous security monitoring • continuous/regular testing • remediation of exploitable vulns
64
![Page 65: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/65.jpg)
#RSAC
Uptime • an application/service event that
causes downtime due to security-related issue (configuration, attack, etc.)
65
![Page 66: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/66.jpg)
#RSAC
“We were able to prove that remediating all discovered SQL injection issues caused less application downtime” – AppSec Prog Mgr
66
![Page 67: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/67.jpg)
#RSAC
67
0
10
20
30
40
50
60
Q1 2012 Q2 2012 Q3 2012 Q4 2012 Q1 2013 Q2 2013 Q3 2013 Q4 2013
I2U (hours of effected uptime total)
![Page 68: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/68.jpg)
#RSAC
We’re showing that removing injection vulnerabilities, which are easily exploitable, reduces downtime.
68
![Page 69: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/69.jpg)
#RSAC
“Impact to residual risk”
69
![Page 70: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/70.jpg)
#RSAC
Impact of a security item to residual risk of an application or service
70
![Page 71: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/71.jpg)
#RSAC
[security item] [residual risk]
71
![Page 72: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/72.jpg)
#RSAC
Security items (examples) • mandatory peer review of code • required stage-gates to
production w/security sign*-off • accountability by LoB VP
72
![Page 73: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/73.jpg)
#RSAC
Residual risk • a level of residual risk in the
application as a result of security effort(s)
73
![Page 74: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/74.jpg)
#RSAC
“For each line of business that reported risk metrics up to the VP successfully, residual risk decreased.” – AppSec Prog Mgr
74
![Page 75: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/75.jpg)
#RSAC
75
0 20 40 60 80 100 120 140 160 180 200
App 1
App 1'
App 2
App 2'
App 3
App 3'
Residual Risk Charting
Cycle 4 Cycle 3 Cycle 2 Cycle 1
App x = Application w/o VP accountability App x’ = Application with VP accountability
![Page 76: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/76.jpg)
#RSAC
We’re showing that raising accountability to the LoB VP, residual risks fall greatly
76
![Page 77: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/77.jpg)
Defining a set of KPIs
![Page 78: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/78.jpg)
#RSAC
What is the goal of your effort?
![Page 79: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/79.jpg)
#RSAC
Minimize injection (A1) defects in new software releases
79
![Page 80: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/80.jpg)
#RSAC
“Let’s show progress”
![Page 81: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/81.jpg)
#RSAC
What security did: Introduced (self-service) static analysis tools into development cycle
81
![Page 82: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/82.jpg)
#RSAC
Impact it had: Initially the impact was prohibitive, but with effort became manageable.
82
![Page 83: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/83.jpg)
#RSAC
“Impact to effort”
83
![Page 84: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/84.jpg)
#RSAC
84
0
10
20
30
40
50
60
70
80
90
100
Q4 2012 Q1 2013 Q2 2013 Q3 2013 Q4 2013
Real I2E (hrs per dev per project)
Dev 1 Dev 2 Dev 3 Dev 4
Q4 2012 Baseline
Q1 2013 Initial rollout
Q2 2013 Product training
Q3 2013 IDE Automation
Q4 2013 Workstream integration
![Page 85: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/85.jpg)
#RSAC
“Impact to release”
85
![Page 86: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/86.jpg)
#RSAC
86
0
20
40
60
80
100
120
140
160
App 1 App 2 App 3 App 4
Real I2E (hrs per dev per project)
Q4 2012 Q1 2013 Q2 2013 Q3 2013 Q4 2013
Q4 2012 Baseline
Q1 2013 Initial rollout
Q2 2013 Product training
Q3 2013 IDE Automation
Q4 2013 Workstream integration
![Page 87: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/87.jpg)
#RSAC
“Impact to uptime”
87
![Page 88: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/88.jpg)
#RSAC
88
0
2
4
6
8
10
12
14
16
18
20
App 1 App 2 App 3 App 4
Security related downtime events
Q4 2012 Q1 2013 Q2 2013 Q3 2013 Q4 2013
Q4 2012 Baseline
Q1 2013 Initial rollout
Q2 2013 Product training
Q3 2013 IDE Automation
Q4 2013 Workstream integration
![Page 89: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/89.jpg)
#RSAC
“Impact to residual risk”
89
![Page 90: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/90.jpg)
#RSAC
90
0
50
100
150
200
250
Q4 2012 Q1 2013 Q2 2013 Q3 2013 Q4 2013
Impact to residual risk *Only A1 + A2 (OWASP Top 10)
App 1 App 2 App 3 App 4
Q4 2012 Baseline
Q1 2013 Initial rollout
Q2 2013 Product training
Q3 2013 IDE Automation
Q4 2013 Workstream integration
*based on organization’s basic IT risk’ calculation
![Page 91: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/91.jpg)
#RSAC
For the adventurous: “Impact to business”
91
![Page 92: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/92.jpg)
#RSAC
Is this approach perfect? No.
92
![Page 93: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/93.jpg)
#RSAC
Do these KPIs work everywhere? No.
93
![Page 94: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/94.jpg)
#RSAC
Better than existing metrics? Absolutely.
94
![Page 95: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/95.jpg)
#RSAC
Strive to do better.
95
Demonstrate meaningful
progress
![Page 96: Succeeding with Enterprise Software Security Key Performance ...](https://reader031.fdocuments.in/reader031/viewer/2022030323/589b07431a28abe6468beab3/html5/thumbnails/96.jpg)
#RSAC
Follow the wh1t3rabbit.
96
https://twitter.com/Wh1t3Rabbit