HDI Capital Area January 15 2014

2/5/14 ©2013 HDI. All rights reserved.

Welcome! Updates from HDI Capital Area

and HDI Corporate January 15, 2014

New! HDI Cer:fica:on

•  HDI announces a new cerBficaBon: HDI Problem Management Professional

•  The first offering will be at HDI 2014 in Orlando.

•  Learn more at www.ThinkHDI.com/PM

New! Corporate Membership

•  Offer your team the ability to benefit from an HDI membership!

•  Special pricing for medium to large organizaBons with twenty-‐five or more support staff.

Contact your HDI Account Manager

to learn more! 800.248.5667

HDI Desktop Support Prac:ces & Salary Survey

•  The HDI Desktop Support PracBces & Salary Survey is now open!

•  Contribute to this acclaimed research and gain insight into current desktop support processes, technologies, metrics, staffing, and salaries.

Take the survey | learn more at ThinkHDI.com/DS

New! Content to Share! “Show Me the Value: Support’s Mandate” – Roy Atkinson

SupportWorld: “Moments of Truth: The Future of the Customer Experience” – Charles Araujo

Most Recent Blog: “Are You Keeping the Lights On, or Are You Driving Change?” – Rob Stroud

#HDIStatToday: 15% of technical support Bckets are related to supporBng mobile devices.

Research Brief: “Mobile Device Support and BYOD: Where Are We Now? “

Current Survey: 4th annual Desktop Support PracBces & Salary Survey

Webcast: “Managing Change and Technology MigraBons” – Peter Jurhs

Coming up! SupportWorld App debuts in January !

HDI Forum Roundtables Take your networking power to the next level.

Join HDI for a Forum Roundtable event!

•  During the HDI Forum Roundtables, you will experience a taste of what happens during the HDI Forum meeBngs.

•  The HDI Forum program is designed to encourage, enhance and strengthen the members and their organizaBons by providing a vision for the future based on today’s actual innovaBons in support.

Register Today! www.ThinkHDI.com/2014Roundtable

HDI 2014 Conference & Expo

•  Register by January 24, 2014, and you could save up to $500!

Register today: www.HDIConference.com

Early Bird Discount $200Alumni Discount $100Member Discount (Gold and above) $200Save $500

Renew Online!

•  It’s fast, it’s simple, and you can do it in your pajamas!

•  You can renew your membership online up to ninety days before its expiraBon date through your MyHDI account.

Visit: www.ThinkHDI.com/Renew

Want to renew early? Call the HDI Customer Care Center at 800-‐248-‐5667!

Not a Member? Join Today! Become a Local Chapter member for just $75!

This individual local chapter membership is an opportunity to

connect, network, and learn in your own backyard.

Enjoy benefits like: •  Amend local chapter and vChapter meeBngs •  Digital subscripBon to SupportWorld magazine •  Apply for HDI awards •  Access to the HDI Job Board •  Regular e-‐newslemers and digests •  And much more!

Learn more at www.ThinkHDI.com/Join

or by calling 800.248.5667

l  Established in 1989 by Ron Muns as the Help Desk Ins:tute. l  1996 Sold to Ziff Davis l  1999 Bought back by Ron Muns under the company name

Think Service, Inc. (TSI) l  2000 Merged Help Desk Professional Associa:on into HDI l  2004 Rebranded to HDI l  2006 Acquired STI Knowledge (d.b.a. HelpDesk 2000) l  2008 Sold to UBM and became part of ThinkServices division l  2010 UBM restructured and HDI joined UBM TechWeb

1989

2000

2010 2002 2006 2012

HDI is 25 Years Old!

HDI Conferences Beat Expecta:ons

•  HDI Conference – April 1-‐4, 2014; Orlando, FL •  Fusion Conference – October 19-‐22, 2014; Washington, DC

HDI has signed with itSMF-‐USA to deliver FUSION through 2016.

©2013 HDI. All rights reserved.

Local Chapter Presos

Conference Session Props

Draw from the Community & Lead with Research

Webcast

Infographic

White Papers & Briefs

Support World

Custom Content w/ Partners

Research

Blogs, Tweets, AYN, FB, LI

Partner White Papers

Community – Forums

HDIConnectLCs

Partners Speakers


in 2014


HDI Social Stats 124 posts, 39 different writers Blog page views: 36,561 -‐ at an average of 2.1 minutes per visit 2000+ new visitors per month

HDI Professional Group – 7,409 members HDI Desktop Support Group – 2,625 members

#thinkhdi -‐ 2,141 followers

2,580 likes

Thank You 2014 Sponsors!!

HDI Capital Area Sponsors

•  Diamond – The MIL CorporaBon

•  PlaBnum Plus -‐ LanDesk

•  PlaBnum – Beyond20 – EasyVista

•  Gold –  Robert Half Technology

(Global Sponsor) –  Bomgar –  IssueTrak –  Cherwell

•  Silver –  Service Now –  Time Warner Cable

•  Web/Event –  IBnvolve –  RemedyForce –  ReACT –  StrataCom –  TechnoLava –  ArBsys

Call for Capital Area Board Nomina:ons

•  NominaBons due by January 27 – VP Programs – VP Membership – VP Finance – VP Special Programs & Vendor Liaison

Next Mee:ng

•  Who Moved My Service Desk? February 19, 12:00 pm -‐ 2:00 pm Marc Fey, Cherwell Sofware

•  We typically meet the 3rd Wednesday of the month and most meeBngs are free

•  Visit www.hdicapitalarea.com to register

Today’s Speaker

•  Dr. Mah Fourie

FINDING THE “ROOT” IN ROOT CAUSE ANALYSIS – A CLIENT CASE

MATT FOURIE THINKING DIMENSIONS

Some of our recent clients... Barclays IT Macquarie ITG Unisys Woolworths IT SGX IT SITA Global BT Financial McDonalds IT Queensland Police IT DBS IT Lockheed Martin Space Systems

•  Thinking Dimensions International - operating KEPNERandFOURIE RCA company initiatives for the last 25 years

•  Specializes in RCA Methodology for IT Incidents, Problems and Projects

AGENDA “Most incident

investigators ask the wrong

questions, so do not change your people

but change the questions they are

asking”

•  Introduction •  Introduce Client Situation •  The Three Skills sets •  The Common Process and Language •  Technical Cause Analysis •  Root Cause Components •  Client outcomes •  Questions & answers

Investigation Info

“It takes a company without a formal and effective Root Cause Analysis culture up to 23 days to repair service incidents”

Aberdeen Group – Boston 2010





asking”


Client Case - PM Improvement

International Investment Bank’s IT PM Division

2009-2012

•  Lack of Stakeholder commitment •  Poor management of information •  Working with poor quality information •  Poor problem investigation support •  Not really solving problems

permanently

•  For P1 & 2 Incidents a PM was assigned immediately •  All Incident & Problem Management Staff trained in some

common process •  Embedded the tools and templates into existing process to

make handover seamless •  Used in-house facilitators to lead PM teams in a very strong

and decisive way

Actions taken…





asking”


PAST NOW FUTURE

TECHNICAL CAUSE ANALYSIS INCIDENT RESTORATION ROOT CAUSE ANALYSIS

STANDARD





asking”


Common process •  Everybody uses the same

process for finding causes and solutions

•  The process determines which questions to ask at each step for each type of incident investigation approach

•  Designed for minimalistic information combined with a good focus to provide quick answers

Step 1: IdenBfy Problem SituaBon

Step 2: Gather Incident InformaBon

Step 3: Analyse Incident InformaBon

Step 4: Determine Conclusion





asking”


Extreme Focus With “Specificity” Specificity Rules • One object one fault • Single-minded & simplistic • Highly focused • Must find the correct entry point • Ask a question – expect an answer

Object Fault Servers Not communicaBng

Data not transferred Sent but not received by receiving servers

Data for Large Outlets Not received

Sales turnover numbers for Large Outlets

Not received

Extreme Focus With “Specificity” Specificity Rules • One object one fault • Single-minded & simplistic • Highly focused • Must find the correct entry point • Ask a question – expect an answer

Object Fault Murex Chip Produces latency

Transactions queuing up TX’s taking longer than 100 milliseconds

Transactions Takes longer to process

“Futures” transactions

Takes longer to process

Thinking more specifically… Incident Statement

Technical Cause

Root Cause

Internet Banking Degrading

New browser configuration issue

Integrative testing not done properly

Encrypted “hello” message not returned

‘Beta’ Certificate used

Policy requirements for “production” environment not adhered to

Thinking more specifically…

Incident Statement Technical Cause Root Cause

G-‐Force System Freezing

High volume Too many users allowed access

G-‐Force SQL DB thread count exceeding maximum count

G-‐Force program not closing out threads

Vendor implemented an untested program update

Crea:ng Intelligence

DATA

IS

Internet Banking

Slow

APAC users

Started Oct 1

ConBnuous

INFORMATION

BUT NOT

Intranet Banking

Freezing

USA, UK

Before

Awer 4pm

KNOWLEDGE

WHY NOT

Different rouBng SSL handshake

Volume?

ADSL lines

New passwords

Different rouBng

Unexpected Outcomes • “BUT NOT” clarifies the facts • Creates a curious “contrast” • Looking at answers at a “granular level” • Stimulates deductive reasoning

Example of templates with questions

“Minimalistic principle”.. •  Only need to analyse the information that

would be relevant to the incident •  Worked questions within a customised

“factor analysis” framework •  Get a quick factual “snapshot” of the

characteristics of the incident and then use SME experience and gut feel to explain the snapshot

•  Test SME inputs against logic of snapshot

“Too much information can cause confusion.

The key is to get all the relevant information only

and that is normally substantially less than

gathering all the Information.”

Innovation – the FreeZone thinking experience. by Kepner & Fourie

Snapshot info for causes IS BUT

NOT WHY NOT

OBJECT

FAULT

USERS

WHERE

TIMING

PATTERN

CYCLE

OBJECT – What object and which other object(s) not?

FAULT – What fault and which other typical faults not?

USERS – Who has the problem and who does not?

WHERE – Where are these users and where could they have been but are not?

TIMING – When did it happen first time and when not?

PATTERN – What is the pattern of faults and what could it have been but is not?

CYCLE – In which cycle does the problem occur and in which cycle does it not occur?

CauseWise sample DIMENSION

IS BUT NOT WHY NOT Possible Causes & Tes:ng

Object

Fault

Loc of Object

Timing

Pahern

Life Cycle

Phase of Work

CauseWise sample DIMENSION


Object Fireburst V2.0 connecBon

Fault

Loc of Object

Timing

Pahern

Life Cycle

Phase of Work

DIMENSION



E-‐Express, Mango connecBons

Fault

Loc of Object

Timing

Pahern

Life Cycle

Phase of Work

DIMENSION



E-‐Express, Mango connecBons

F/B upgrade from V1 to V2, Poor tesBng issue

Fault

Loc of Object

Timing

Pahern

Life Cycle

Phase of Work

DIMENSION

IS BUT NOT WHY NOT Possible Causes & Testing

Object

Fireburst V2.0 connection

E-Express, Mango connections

F/B upgrade from V1 to V2, Poor testing issue

Fault dropping Freezing, slow Time out settings, configuration of drivers

Loc of Object

Timing

Pattern

Life Cycle

Phase of Work

DIMENSION


Object





Loc of Object

ANZ, USA, UK

Asia LAN, Proxy server issues, F/Wall rules

Timing

Pattern

Life Cycle

Phase of Work

DIMENSION


Object





Loc of Object

ANZ, USA, UK


Timing

Monday, Sept 2nd with SOB

Any time earlier than Sept 2nd

Java upgrade, Netscape upgrade

Pattern

Life Cycle

Phase of Work

DIMENSION


Object





Loc of Object

ANZ, USA, UK


Timing




Pattern

Continuous Sporadic, Periodic

Don’t know

Life Cycle

Phase of Work

DIMENSION


Object





Loc of Object

ANZ, USA, UK


Timing




Pattern


Don’t know

Life Cycle

When doing a transaction

“x” time into transaction

Operator error, Code error on a specific page

Phase of Work

DIMENSION


Object





Loc of Object

ANZ, USA, UK


Timing




Pattern


Don’t know

Life Cycle




Phase of Work

Just after logging in

Logging in or out

OS configuration issue, DNS issue

DIMENSION


Object




1. Proxy server tampered with during the Java upgrade on the LAN


Loc of Object

ANZ, USA, UK


Timing




Pattern


Don’t know

Life Cycle




Phase of Work


Logging in or out


DIMENSION


Object






Loc of Object

ANZ, USA, UK


2. Java upgrade caused driver incompatibility with Fireburst website V2.0

Timing




Pattern


Don’t know

Life Cycle




Phase of Work


Logging in or out


DIMENSION


Object






Loc of Object

ANZ, USA, UK



Timing




Pattern


Don’t know 3. Netscape upgrade caused driver incompatibility with Fireburst website V2.0

Life Cycle




Phase of Work


Logging in or out


DIMENSION


Object






X

Loc of Object

ANZ, USA, UK



Timing




Pattern



Life Cycle




Phase of Work


Logging in or out


DIMENSION


Object






X

Loc of Object

ANZ, USA, UK



Timing




√ √ X

Pattern



Life Cycle




Phase of Work


Logging in or out


DIMENSION


Object






X

Loc of Object

ANZ, USA, UK



Timing




√ √ X

Pattern



Life Cycle




√ √ A1 √ √ √ √

A1- Only if the staff in Asia did not upgrade to Netscape

Phase of Work


Logging in or out






asking”


Stair stepping… 3. Netscape upgrade caused driver incompatibility with Fireburst website V2.0

Installed the wrong driver WHY?

WHY? Driver Specs not correct

Don’t Know WHY?

COMPONENT CAUSAL FACTORS Decision Making DM Process and Collaboration for inputs

Implementation issues Resourcing Issues and Scope & Definition of project

Standard Operating Procedures

Applicability of SOP, Awareness of SOP and Documentation

Management Management of Work and Management of Staff

Measurement Key Performance Indicators (KPI’s) and Roles & Responsibilities

Support Internal Support and External Vendor support

Communications Clarity of communications and verbal instructions

Work Environment Task Interference and consequences of doing the task

Skills Complexity needed and applicability of person to task

Testing Practices Testing Procedures and Testing Requirements

Personal Aptitude for job and Attitude towards the job

Root Cause Analysis Components

Snapshot info for Solutions Key Requirements 1 Best data transfer rate

2 No loss of data

3 Improve system up-‐Bme

4 Improve trickle & purge

5 Reduce DR Bme

6 Capex < $2m

7 Implement < 3 mos

Four Question Drill •  What are the results you want to

achieve with this solution? •  What are the existing problems you

would like to remove with this solution?

•  What are the potential risks you would like to avoid with this solution?

•  What money and time do you have or do you need to preserve? What are the restrictions out of your control?

Snapshot info for Solutions Key Requirements 1 Best data transfer rate

2 No loss of data



5 Reduce DR Bme

6 Capex < $2m

7 Implement < 3 mos

Questions for Actions •  What action can we take today and

implement tomorrow that would meet the 1st requirement? (repeat)

•  What action can we take today and implement tomorrow that would meet the 2nd requirement? (repeat)

•  Looking at all the actions, which one(s) would satisfy the 1st requirement best?

•  Which actions has got nothing to do with the 1st requirement?


•  Repeat for each requirement…

Snapshot info for Solutions Key Requirements

1 2 3 4 5 6 7 8

1 Best data transfer rate

2 No loss of data



5 Reduce DR Bme

6 Capex < $2m

7 Implement < 3 mos









1 2 3 4 5 6 7 8


3 3

2 No loss of data



5 Reduce DR Bme

6 Capex < $2m

7 Implement < 3 mos









1 2 3 4 5 6 7 8


3 3 0 0 0 0 0

2 No loss of data



5 Reduce DR Bme

6 Capex < $2m

7 Implement < 3 mos









1 2 3 4 5 6 7 8


3 3 0 1 0 0 0 0

2 No loss of data



5 Reduce DR Bme

6 Capex < $2m

7 Implement < 3 mos









1 2 3 4 5 6 7 8


3 3 0 1 0 0 0 0

2 No loss of data 0 3 3 0 0 1 2 2


1 2 1 0 3 1 2 1


2 2 3 1 0 0 0 0

5 Reduce DR Bme 1 1 3 2 1 1 3 1

6 Capex < $2m 3 1 2 2 1 0 3 1

7 Implement < 3 mos 0 3 0 0 0 1 3 3









1 2 3 4 5 6 7 8


3 3 0 1 0 0 0 0

2 No loss of data 0 3 3 0 0 1 2 2


1 2 1 0 3 1 2 1


2 2 3 1 0 0 0 0

5 Reduce DR Bme 1 1 3 2 1 1 3 1

6 Capex < $2m 3 1 2 2 1 0 3 1

7 Implement < 3 mos 0 3 0 0 0 1 3 3

Develop the new solution •  Circle the best performing areas to

give you a visual impact of best performing actions.


1 2 3 4 5 6 7 8


3 3 0 1 0 0 0 0

2 No loss of data 0 3 3 0 0 1 2 2


1 2 1 0 3 1 2 1


2 2 3 1 0 0 0 0

5 Reduce DR Bme 1 1 3 2 1 1 3 1

6 Capex < $2m 3 1 2 2 1 0 3 1

7 Implement < 3 mos 0 3 0 0 0 1 3 3

Develop the new solution •  Circle the best performing areas to

give you a visual impact of best performing actions.

•  ASK: Which combination of actions would give you at least one [3] for each requirement? Look for at least 3-4 actions to combine

•  Lastly, which action(s) do we have to add to the mix to make the suggested solution even more effective?


1 2 3 4 5 6 7 8


3 3 0 1 0 0 0 0

2 No loss of data 0 3 3 0 0 1 2 2


1 2 1 0 3 1 2 1


2 2 3 1 0 0 0 0

5 Reduce DR Bme 1 1 3 2 1 1 3 1

6 Capex < $2m 3 1 2 2 1 0 3 1

7 Implement < 3 mos 0 3 0 0 0 1 3 3

The Solu.on! •  Which combinaBon of acBons makes

the best sense and would provide the best chance of success?

•  Check whether your suggested soluBon would saBsfy all the requirements. Also check impact on implementaBon costs

•  How would you suggest we implement these acBons?





asking”


Application Performance results

0

10

20

30

40

50

60

70

80

90

100

Mean-‐Bme-‐to-‐repair

Improvement m-‐t-‐t-‐r

Availability

Our client’s systems availability went from 76% at the end of 2009 to 88% end 2010 to eventually 95% at the end of 2012 – A gain of 19%

2009 2010 2012

Improvement in escalations

0

10

20

30

40

50

60

70

80

90

Sev 3 to Sev 2 Sev2 to Sev 1 Recurring problems

Vendor Interven:ons

In our client’s case the P2 to P1 escalations dropped by 38% Recurring incidents dropped by 21% P3 to P2 escalations dropped by 19%

Lessons learned.. •  Most of the recurring incidents and problems are caused by “out of date

procedures” and lack of proper documentation •  TCA is a “mental orientation” which people have to get trained in – “does not

come with experience” •  IT professionals need a “thinking approach” that could be applied in most

situations •  Rules of Engagement to become a standing order •  Encourage use in all incident investigation meetings – ask for the

paperwork/evidence •  Sponsors continuous TCA/RCA training •  Regular email communications to publish successes

Questions & Answers

I thank you for your time!

HDI Capital Area January 15 2014

Technology

Transcript of HDI Capital Area January 15 2014