Your approach towards Business...

BusinessMeetsIT: Business Continuity1 11/12/2014P.

Your approach towards Business Continuity


Who’s BELTUG?

BELTUG is the largest Belgian association of ICT managers, with a specialised focus on company networks, mobile communications, UC and Cloud.


BELTUG is the largest Belgian association of

ICT managers

Companies/Organizationsare member of BELTUG

ICT professionals aremember of BELTUG – plus 500 V-ICT-OR members (local governments)

Average yearly growth ofthe member base between2008-2013

Participants in BELTUGactivities in 2013


AGENDA


WHAT ?

Purpose of a BCP, in generic terms

Execute risk-management

• Mitigate risks

• Operate efficiently “when disaster strikes”

Start = Define (a) S.M.A.R.T. objective(s)

• Bottom up:

IT / facilities-focused

• Tools + strategies to minimize downtime and preserve the business data

• A subset of business continuity plans

• Top down:

Business-focused

• Strategy that outlines plans and procedures to keep operations100% available

• Framework, establishing contingency plans for various parts of the business

BCP and DRP in perspective

Overzicht van de overeenkomsten BCP – DRP doelstellingen

Datacenter 1 en 2

NetworkServer Farm

Storage

ICT GAP ANALYSE

Middelenvoor herstel

AS IS

DRP (= plan, een

document)• Inventaris techniek• Inventaris applicaties

• Detail Beschrijving van een theoretisch herstel

BUSINESS ProcessenKritsiche eerst !

Inventaris applicatiesVereiste RPO RTO …

Herstel Scenario’sAutomatisch-manueel ?

Op welke locatie ?Proces vereisten ?

Vertrek = DRP Process Kaart

PROJECTTESTEN DRP

BC

PD

RP

ICT

Voortschrijdend Inzicht

Impact op INFRA

ARCHITECTUURAPPLICATIESPROJECTEN

Wetgeving, Regulering

Security

date1 date2 1/4/2014 date3

Concretetesten

gedefinieerd

Het DRP Alternatief• In €• In Manpower• In Scope

It won’t happen to me ??

Spectacular Disastersare remembered

easier/longer

Exceptional or

Far from home

creates a false feeling

“THAT

won’t happen

to me anyway”

Risks ?

Avoid encurring unknown risks,

but never be afraid…

to take a managed-controlled risk !


All references are equal, but some are more equalThan the others !!

Resultaat van een BCP:

Het doordacht treffen van maatregelen

om bij “alle ernstige storingen”

de continuïteit van kritische bedrijfsprocessen,

die gebruik maken van informatiesystemen,

binnen de maximaal toegestane uitvaltijd

te waarborgen.

Van onbewust risico lopen… naar bewust risico nemen

Defining Disasters: Business Disruptions

Civil Unrest

1%

Fire

8%

Flood

8%

Power Outage

13%

Hardware

12%

Hurricane

14%

Terrorism

24%

Lightning

0%

Software

1%

Tornado

2%

Data Center Move

1%

Network

2%

Environment

2%

Miscellaneous

2%Bomb

3%Earthquake

7%

Source: Comdisco Vulnerability Index

« 2 out of 5 enterprises, experiencing a disaster, are out of business within 5 years » Gartner

Sources of downtime

source: Gartner Group, December 2002

unplannedplanned

application failure40%

20%

operator errors40%

application and

database65%

10%hardware, networks,operating systems, system software

batch application processing

13%

10%backup/

recovery

physical plant/ environment 2%

“people and processes account for over 80% of all downtime”

environmental factors, hardware,

operating systems, power, disasters

Omgaan met risiko’s…• Laat het

– Potentiële impact is te klein om iets te doen– Negatieve impact is nooit af te dekken tegen de vereiste kosten– Er bestaat gewoon niets om het risiko te verkleinen

• Monitor het– Geen proactief beheer, wel een reactief bewaken van het gevaar

• Ontwijk het– De oorzaak van het probleem/risiko gewoon elimineren, kan gaan van het wisselen

van leverancier tot een technische wonder. Niet altijd mogelijk, uiteraard !

• Verschuif het– Typisch geval van outsourcing naar een gespecialiseerde derde partij

• Verklein het– DIT is meestal de aangewezen oplossing: proactieve stappen nemen om het risiko te

ontlopen of te reduceren tot de impact op een aanvaardbaar niveau wordt teruggebracht

Last

Backup

Business activityNormal Level

Time

RPO

Recovered Minimum Level

Max amount

of “allowed”

DATA LOST

HW

SWDATA

backup

TEST

DATA

Recovered

since T=0

RTO

RMO

Max amount of allowed OUTAGE TIME T = 0

T - 1

T + 1

Back from T-1 point

Back at T+1 point,But in RMO mode

Plan for Disaster = Recover faster !!

RPO, RTO, RMO, MTO, MT(p)D, RTC, WRT, …

Maximum Tolerable Downtime, Work Recovery Time, Recovery Time CapabilityMaximum Tolerable Outage, Maximum Tolerable period of Disruption


The new Mini-guide on BCP-DRP is here

Which business continuity + recovery measures are in place today

Do they correspond to the business needs? have you verified ??

How to determine RTO, RPO (and RMO)

– Raise awareness / general information session

– Questionnaires– Interviews– Workshops– Identify interdependencies– Validate results– Management decision


WHY ?

Waarom zou u pessimistisch denken ?…

• 93 % van de bedrijven die een significante data loss ondergingen, verdwenen binnen de 5 jaar

• Het marktaandeel dat verloren gaat per 8 uur “out of business” wordt binnen de 3 jaar NIET teruggewonnen

• Per 6 uur downtime zal het bedrijf nog schadelijk gevolgen dragen gedurende een jaar lang !

Méér VÓÓRdenken, niet alléén NAdenken !

Improving Customer’s Core Business ! 25

Time to recover

K

O

S

T

VERLIES

Accepteerbare

downtime

GE

LD

Spendeer méér

Verlies minderBreak even

Spendeer minder

Verlies méér

Uw eigen pijnpunt€

Hoe sterk is de competitie ?

Hoe loyaal zijn uw klanten ?

Hoe gemakkelijk kunnen ze overlopen ?

Heb je zelf een alternatief verkoopskanaal ?

Improving Customer’s Core Business ! 26

« Continuity Services »

Gemiddelde kritikaliteit

NIET kritische Activiteit

« Betaalbare » Recovery Kosten

in Functie van vereiste recovery delay

Time4h 8h 24h Days

« Disast.Rec.Services »

Kost in M€ Kritische Activiteiten

- Dealing Room

- E-commerce

- Critical Process (SAP, weborders…)

Recovery Solution

BusinessVERLIES


Suppose your premises are simply isolated (a fire around the block)

• Up-to-date list of telephone nrs, “who to call first”, available outside premises

Top 10 customers, top 10 suppliers, contractors,..?

• Who are the key employees to call first ? Do they know what to do first ?

Do they have access to all required info, outside your premises ?

Do THEY trust their plans and updates ?

• Is there a prepared “positive” press announcement ? A framework for it ?

WHO is entitled to launch it ? Who to consult for content & consequences ?

• Do YOU know what to do if your absolute most important supplier has such a major incident tomorrow, and he is OUT for a week or more ?

BCP is not rocket science, it is pre-planned common sense

“WHY” ?.....The “24 x 7” enterprise

• Shareholders expect management to remain in

control through any crisis. If not ? See you in court !

• Regulatory agencies expect their rules to be met,

regardless of the conditions. BASEL II

• Customers need services and supplies to continue

without interruption. Or they’ll find their alternatives

• Suppliers expect agreements and payments to be

nonstop. If not ? See you in court

• Employees expect their livelihood to be protected.


WHEN ?

Approach, implementation plan

• Who are the stakeholders ?

• Which domains have to be covered ?

• Which processes are crucial - critical ?

• Which systems support these critical processes ?

• Requirements on reliability ?

• Threats & Risks ?

• Which measures @ company, individual level ?


“The customer has a choice at all times.”

what the business objective is

how much he is prepared to spend

when and how to implement

what functionality is required

what performance levels are needed

what quality and resilience are necessary

what risk levels are acceptable

how many service providers to use

what kind of organisation(s) he wishes to deal with

how to structure the provision of services

how to organise and manage the relationship(s)

what the criteria are for selection

whether to proceed at all….

ALL of this … is worth investing in the selection process !

ICT

NON-ICT

You have a choice, but .. Do YOU decide ??

Can you allow NOT

focussing on these?

ICT : Helping the business

to be(come) even more Efficient & Effective

Expert Class on ICT Procurement33 6-12-2012P.

Shifting from device/product/service-oriented towards end-user-oriented

SCOPE: Elapsed time = 1 month

CSF: Management of SLAs (SLM)

Outsourcing your Data Center

SW

OT

HARMFULTo achieving the objectives

HELPFULTo achieving the objectives

STRENGTHS WEAKNESSES

OPPORTUNITIES THREATS

• Ease of mind• Staffing• Physical security• Upfront investments• Maintenance is no kid’s play

• Quality of outsourced solution does it fit with the business

• We made the investments and they are not yet depreciated

• We need a total managed solution: data center + IT

• Avoid capital expenditure• More flexibility• Better connectivity • Sustainability

• Loss of control• Risk of outages• Rapid changes in IT-technology – cloud

In

tern

alfa

cto

rs

Exte

rn

al

facto

rs


BE SPECIFIC !! % availability, … per MONTH !!

“Level TIA 942”ISO 27001Uptime Institute

Availability “Allowed”downtime

Per MONTH

“Allowed” downtimePer YEAR

TIER 3 99, 982 % 8 min/m 94 min/y 1,6 hr/y

TIER 3+ 99, 99 % 4 min/m 52 min/y < 1 hr

TIER 4 99, 995 % 2 min/m 25 min/y 0,4 hr

No cumulated tolerances per Y

No repetitive breaches ! (= E.O.T)

http://www.datacenterchecklists.com/data-center-tier-4-requirements-templates

http://www.datacenterchecklists.com/data-center-tier-4-requirements-templates


HOW ?

Plan

in

LAYERS

38

Business Continuity Plan

Business Impact Analysis

Risk Analysis

Recovery Strategy

Group Plans and Procedures

Business Continuity Planning Initiation

Risk Reduction

ImplementStandby Facilities

Create Planning Organization

Testing

PROCESS

Change Management Education Testing Review

Policy ScopeResourcesOrganization

Ongoing Process

Project

What+How MuchIs at risk

Data

Back-up

Cycle

Normal Operations

RPO

Data Loss

RTO

Functionality Loss

Disaster

Systems & Data

RecoveryBusiness processes

operational

Business Impact Analysis Framework

1. What is the maximum elapsed time from start of disruption until minimum functionality is restored?

=> Recovery Time Objective (RTO)

2. What is the maximum accepted data loss (i.e., no data loss, one hour, one day, etc.)?

=> Recovery Point Objective (RPO)

3. What are the key operating resource dependencies that must be replicated to alternate recovery facilities, including people, vital records, communications, facilities, equipment and IT infrastructure.

=> Minimum Operating Requirements (MOR)

Answers to these questions

require two activities:

1. Evaluate the impact of

several disaster

scenarios

2. Determine what is

acceptable for

management

41

Business Impact AnalysisGoal

• to determine the impact of an outage or unavailability of a service on the business processes and operations

• to define the resulting business requirements for continuity

To be performed on a business level (using business terminology)

NOT: server A is down, impact = application B does not function anymore and processes X & Y are not operational anymore

BUT: what is the financial, operational and strategic impact of an unavailability of application XYZ

- Orders cannot be registered anymore- X euros in missed orders and revenue loss- We will not reach profit goals towards shareholders

42

• Dimensions of Business Impact

– Financial impact

• Revenue

• Costs

• Productivity impact

• Contractual penalties / fines

– Strategic impact

• Market share

• Brand name

• Regulatory oversight

– Operational impact

• Employee morale

• Internal controls

• …

• Dimensions of Business Impact

– Direct Impact:

• Business interruption

• Data loss

– Internal indirect impact

• Consequences on other business operations (longer shifts, reduced stock levels, …)

• More unfamiliar people at other sites, creating additional risks.

– External long-term impact

• Brand name

• Investor confidence

• …

BIA dimensions

43

The Cost of Downtime *

ApplicationTypical Downtime

Costs

Financial/Trading €40,000 / minute

Supply Chain €10,000 / minute

ERP €10,000 / minute

CRM €8,000 / minute

E-Commerce €8,000 / minute

E-Business €8,000 / minute

Business

Application

€5,000 / minute

Database €5,000 / minute

Messaging €1,000 / minute

Infrastructure €700 / minute

* Source: DRJ; Winter 2003 Issue; How Much Is Enough?

* Source: Gartner; High Availability Networking; September 2002

The Cost of Downtime *

Cost Description

Productivity

Loss

• Number of employees

affected x hours out x

burdened hourly rate

Revenue Loss

• Direct revenue loss

• Compensatory payments

• Lost future revenue

• Billing losses

• Investment revenue losses

Impaired

Financial

Performance

• Revenue recognition

• Cash flow

• Lost discounts (accounts

payable)

• Payment guarantees

• Credit rating

• Stock price

Damaged

Reputation

• Customers, Suppliers,

Financial Markets, Banks,

Business Partners

Other Expenses

• Temporary employees

• Equipment rental

• Overtime costs

Benchmarking the cost of downtime

Please

focus on the

PROCESS,

not

on the

technology

!!

Assessment

or

WHAT-IF

.

BUT … What if …

Checklist: prepare for the unthinkable

• Does your organization have an up-to-date business continuity plan for its mission critical activities and their

dependencies? What can’t you afford to lose in order to maintain critical business processes?

• Does the plan define how to revive those activities within a stated time frame? Aiming for ‘zero downtime’

could be very costly, and inappropriate for several areas of the business.

• Is business continuity adequately funded in your company? It may be less costly than you think, and can be

developed gradually. The economy, as well as the handy ubiquity, of high-speed IP networking must not be

overlooked.

• Who writes up the business continuity plan? If done by IT there is a risk that too much attention will be paid to

technology systems at the expense of business processes and people issues.

• Who is ultimately accountable for business continuity? Is the reporting line to that individual clear?

Responsibility for business continuity must be a board level issue.

• How regularly are ‘fire drills’ staged to test business continuity plans, and ensure they are up to speed with

recent changes in the organizations?

• Does your plan specify personnel roles and their accountability? Are they clear when they should invoke

business continuity plans?

• Does the business continuity plan specify the level of response required, according to the type of

emergency?

• How is the plan communicated to staff in the organization? How do you check that the message has got

across?

• How do you tackle the press and media following a crisis? The company’s standing can actually increase if

the publicity provoked by the crisis is properly managed.

Use the BELTUG Checklist

• Chapter 4 : what do you have to know upfront (= previous slides)

– Solve the IT & Telecom problems as they arise ?? N, plan !!

– Mitigate risks: eg. How much data can you afford to lose ?

– How far is far enough ? Ref. WTC I –WTC II

– Availability 99.9999 %, who will measure it ?

– Single points of failure: state them + how to eliminate ?

– Transparency & dependency on 1 single provider (2 paths)

• Chapter 5 : Operator assessment, questions !– Self assessment: 28 questions

– Provider assessment: 31 questions

If you want to understand it, never stop asking questions.

Socrates

50

Self Assessment: 28 questions

• Services

• Network routing

• Dependencies

• Diversity an separation

• New Services

• Changes to network structure

• Power

• Contact in crisis

51

Provider Assessment: 31 questions

• Standards

• SLA’s, contracts & due diligence

• Providing assurance

• Availability measures

• Understanding the threats

• Providing the right solutions

• Final questions

“Questionairy for Audits”ISO IEC 17799 2005

Information Security Audit tool

14.1 USE CONTINUITY MANAGEMENT TO PROTECT INFORMATION

14.1.1 ESTABLISH A BUSINESS CONTINUITY PROCESS FOR INFORMATION

14.1.2 IDENTIFY THE EVENTS THAT COULD INTERRUPT YOUR BUSINESS

14.1.3 DEVELOP AND IMPLEMENT YOUR BUSINESS CONTINUITY PLANS

14.1.4 ESTABLISH A BUSINESS CONTINUITY PLANNING FRAMEWORK

14.1.5 TEST AND UPDATE YOUR BUSINESS CONTINUITY PLANS

Audit “tools” ….(goal, guide, ctrl)

Highly

Available

Datacenter

People

Staffing Training

Process

Standardization Simplicity Documentation

Technology

Data Processing Communications Data Storage

Datacenter-critical Physical InfrastructurePower Racks &

Floorspace

Cooling Service &

MaintenanceMonitoring

& Control

Fire

ProtectionCablingSecurity

DCPI Quick Scan - Summary


Starting from RPO and RTO…..

….

With views from the real life

1 4 8 24hrs 2 3 days 1w

1 4

8

2

4h

rs 2

3

days

1w

RTO, max outage

RP

O, m

ax

lo

st

da

ta

Specific Needs per key business process

Production A

ERP

Pri

nt

se

rve

r

File server

Accounting

MAIL

WWWVoice mail

ONLineShop

Business functions have specific needs and

regulatory requirements (Basel II)

How much data can the business afford to loose?

How fast do you need to be up and running again?

Have all resources the same requirements?

All cost have to be considered

Financial impact (Loss of revenue, Additional costs)

Intangible impact (Image loss, Market share)

Plot RPO versus RTO

THAT will help you to select the most appropriate solution

Watch the axes

State your 'Acceptable Risk' :

Same story, focus on storage

RGO = recovery granularity objective

Four Major Availability Strategies

Standard

Availability

(99%)

Computing

Data

Network

Single

Server

Disaster

Recovery

(99.9%)

High

Availability

(99.99%)

Resilient

(99.999%)

Server with

Hot-Site

Subscription

Local

Cluster

Dispersed

Cluster with

Failover

Single

Storage

Device

Storage

Device with

Off-Site

Vaulting

Local

RAID Mirroring

Synchronous

Remote

Mirroring

Legacy

Network

Connectivity

2nd Center or

Trailored

solution

Unprotected

DWDM

Services

Protected

Metro Ring

Services

65

PRODUCTION + UAT Tier 1, HA + Log-shipping Tier 2, Log-shipping Tier 3, "DRP"

Datacenter 26 sec /m No ADW

99,999% 365*24 zero SPOFs ? Report measured results (top down) Today 11 SPOFs in WTC

Netwerk Backbone 26 sec /m No ADW

99,999% 365*24 zero SPOFs ? Report measured results (top down) Today 1 identified

Network Distribution 4 min/m + 43 min/m Backup ADSL->ISDN

99,99 % + 99,9 % Hundreds of SPOFs 99,99%PENALTY ? Measure ! Report ! Extra availability = extra cost

Serverpark HW ADW 99,95%99,95 % + 99,9 % 20 min

Applications-SW Poststation: 10,7% 99,75% 96,84 96,599,75 % + 99?? % Postacademy: 73% some 100 min some 22 hrs some 1 day

PostOffices Equipment

Relevant in E2E figures 1 day for the whole process

Hdesk-CallC-ControlR

Availability = per month Tier 1 Tier 2 Tier 3

99,999% 99,9% 99,0%26 sec 43 min 438 min = 7,3 hr

"Process", BCP back

in business RTO = per incident2 hrs <= 24hrs <= 72 hrs

= 99,72 % = 96,71 % = 90,14 %

"Back in time", allowed

Data Loss = ? RPO = per incident0 hrs <= 4 hrs <= 1 working day

= zero dataloss = accepted 4 hr data-loss = accepted 1 day data-loss

43 min/month

99,999% NO diversification between different applications

99,999% NO diversification between different applications

99,9%

Some applications run without any NW !

43 min/month

99,9%

Availability in % means “tangible €”

And in practice …

24 0-12 2412 36

Uren nodig voor het hervatten van de bedrijfsvoering (RT0)Uren van verloren transacties (RP0)

Transacties niet opgeslagen

Declaratie

Verplaatsing

Systeem boot

Database herstel

Transacties herstellen

Process 1

Process 2

Process 3

71

It is all about scales and scalability !!

Completely Duplicated/

Interconnected hot-site

Remote Disk Mirroring

Disk Mirroring

Shared Disk

Single Disk Copy

Electronic Vaulting

Tape On-site

Tape Back-up

Off-site (trucks)

Disk Consolidation

More

LessDelayed

Immediate

More

LessAmount

of Data

Recovery

Time

Importance of

Data

Different Types of Data

require

Different Levels of Protection

Source: CNT

Same story, focus on storage


WHO ?

“The danger of oversimplification”

Make things as simple as possible,

…. but never simpler !Albert Einstein

Never assume, it will make an ASS out of U and ME

SO…this info-session will (hopefully) comfort you

• Proving that you ARE indeed on the right track

• BUT I cannot … make BCP DRP less complex than it is !

WHO ? The one who answers on the philosophical view

Calamiteiten Management team ICT : Naam, Voornaam

Gedelegeerde van RvB

BELEIDSGROEP =Coordinatiecentrum Calamiteiten

-Communicatie-Coordinator Gas, Coordinator Electriciteit-Facilitair Bedrijf, BHV-Inform. Beveiliging ISO: -ICT:

CMT Gas

Etc.

CRT 1 Calamiteiten Reactie Team

Service Delivery INFRAServers, Werkplek, Operations, Svc

Desk,Netwerk, Verbindingen, Spraak

CRT 2 Calamiteiten Reactie Team

Service Delivery APPLICATIESAP, GIS, KIS, e-commerce,

ET&W BiLLiT, specials..

CMT Electriciteit

BCP IV

Beslissen

Adviseren

Uitvoeren

WHO ? ORG chart of a crisis team

BeleidsgroepCrisis Coordinatie

ICT Crisis Management Team

ICT Crisis Reactie Team

Afstemming met de BU’s

BU Management

Inf-Mgr, ICT-Mgr ?

BU rescue team ???Ook geldig voor alle oefeningen, organisatorische uitwijk, communicatie, …Per BU, ook geldig voor afstemming met bv. Facilitair Bedrijf, BedrijfsHulpVerlening, …

?

?

?


LinkedIn BCMIX


WHERE ?

BCP: business is “key”

• Digg & Delve deep where you must– Business critical processes

– Full analysis & Appropriate measures

• Be pragmatic where you can

– Most processes: create a baseline

for common requirements & measures

– Best practice !

BUT……

• Avoid paper tigers

• Create quick wins

• Adapt, step by step

Get STARTED !

Process control


CMMI applied in practice..

The Belgian department ISACA of the organization IIA–(the Institute of

Internal Auditors) uses an assessment in 8 domains.

Executive management support and sponsorship

Risk assessment and business impact analysis

Business continuity strategy and design

Business alignment

Plan development and strategy implementation

Training and awareness

Testing and plan maintenance

Compliance monitoring and auditing

These domains are ranked against 5 CMMI levels:

initial, repeatable, defined, managed & optimizing.


CMMI applied in practice..

initial, repeatable, defined, managed & optimizing.

Executive

Management

Support

and

Sponsorship

repeatable Senior management supports the

BCM

program; however limited

involvement

in process execution persists.

Although

coordination of CM, BC, and IT

disaster

recovery are assigned to middle

management, overall coordination

of BCM is ad-hoc or missing. Failure

events are recognized and

corrected after they occur.

Senior management is aware of the

need

for BCM capabilities. A BCM policy has

been created, and BCM efforts are

driven

based on the results of a BIA (formal or

informal).

Executive

Management

Support

and

Sponsorship

initial Senior management sponsorship of

BCM efforts is informal or absent.

At this stage, BCM capabilities rely

on individual efforts and “heroics,”

and mostly focus on IT systems

backup and restoration, and ER such

as building evacuation procedures.

These efforts are led by middle

management

And executed without proper funding

and sufficient resources. Consequently,

any existing continuity capabilities are

defined as tactical measures.

Characteristics of Capability Method of Achievement


Get Senior Management buy-in

Start creating awareness inside the company: SMART objectives !

Assess impact (in €) of business disruption at your level in your department

Get external consultant to assist internal process: objectivity

Contact BCP-DRP-partners in an early stage toensure a consistent overall design

Where to start tomorrow ?

Nobody can direct the wind..But we can…adjust the sails !

Your approach towards Business...

Documents

Transcript of Your approach towards Business...