JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

39
The most comprehensive Oracle applications & technology content under one roof The most comprehensive Oracle applications & technology content under one roof Real World Disaster Recovery Mark Elley

Transcript of JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

Page 1: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof The most comprehensive Oracle applications & technology content under one roof

Real  World    Disaster  Recovery  

Mark  Elley  

Page 2: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

What  is  a  Disaster?  Definition: dis·as·ter noun /diˈzastər/  disasters,  plural  

•  A sudden event, such as an accident or a natural catastrophe, that causes great damage or loss of life- 159 people died in the disaster

•  - disaster struck within minutes of takeoff •  An event or fact that has unfortunate

consequences- a string of personal disasters •  - reduced legal aid could spell financial disaster •  A person, act, or thing that is a failure- my perm is

a total disaster

Page 3: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Disaster  

Bad  Perm  

Page 4: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Types  of  Disaster  •  Natural  Disaster  

– Earthquake  – Tsunami  – LiquefacEon  

•  Other  Disaster  – Fire  – Flood  –  Infrastructure  failure  

Page 5: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

SENDAI    Japan    

March  2011    

Page 6: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Japan  -­‐  Earthquake  &  Tsunami  

Page 7: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Building  Damage  

Page 8: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Infrastructure  Damage  

Page 9: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Tsunami  

Page 10: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Christchurch    New  Zealand  

 September  2010  

&  February  2011  

 

Page 11: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Christchurch  -­‐  Earthquakes  

Page 12: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

 Building    Damage  

Page 13: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

DestrucEon  &  Human  Tragedy  

Page 14: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Infrastructure  Damage  

Page 15: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Brisbane    Australia  

 January  2011  

 

Page 16: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Inundated  

Page 17: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Stock  Recovery  

Page 18: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Server  Recovery  

Page 19: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

All  of  the  Disasters  above  have  occurred  in  the  last  

12  Months        

Page 20: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

What  about    Localised  Disasters?  

     

Page 21: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Disaster  -­‐  FIRE    

Page 22: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Disaster  FLOOD  

 

Page 23: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Infrastructure  Failure      

1998  Auckland  CBD    Three  Week  Outage    Container  Ships  plugged  into  the  grid  to  provide  some  supply…  

Page 24: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Could  Your  Business  Survive  A  Disaster  

Page 25: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Disaster  Recovery  

•  Focussed  Overview  of  Recent  Experience  •  Predominantly  with  JD  Edwards  in  mind  

– Purpose  •  Lessons  learned  from  the  field  • What  works  best  /  What  doesn’t?    • What  fits  your  business  expectaEons  for:  

–  Return  to  OperaEon  (RTO)  ?  –  How  long  can  the  business  survive  without  this  core  system  

•  Is  distance  a  factor?  

Page 26: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Business  ConsideraEons  •  Human  Resource  Issues  

–  Staff  dealing  with  personal  &  family  issues  –  Key  staff  oeen  have  to  step  up  beyond  the  call!  

•  Site  Access  – Days  to  get  an  engineer  to  give  the  OK  to  site  access  –  Building  irreparably  damaged  –  Servers  &  server  room  destroyed  –  Physical  constraints  –  i.e  Roading  – And  Many  More!!!  

Page 27: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Provoking  Thought  •  What  is  in  place  today  if  a  disaster  occurred  •  Do  you  have  a  true  DR  strategy  or  HA  only?  •  Could  you  be  confident  that  your  business  would  conEnue  to  operate  in  the  event  of  a  disaster?  

•  Can  you  put  a  cost  to  each  hour  of  outage  for  your  business?  

•  Can  you  keep  doing  business  with  system  users  in  other  ciEes?  

•  Does  your  business  need  up  to  the  Nano-­‐second  data  or  point  in  Eme?  

Page 28: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Typical  Infrastructure                Test  /  DR                Produc6on  

Page 29: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Scenario  One    •  Earthquake  –  city  wide  impact  –  huge  destrucEon  –  server  room  within  the  cordon  –  liquefacEon  in  the  server  room  –  building  structurally  unsound.    

•  Decision  made  early  to  switch  to  DR  soluEon  in  another  city.    

•  But!!!  

Page 30: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Scenario  One  -­‐  LiquefacEon    

Page 31: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Scenario  Two    •  Earthquake  –  city  wide  impact  –  huge  destrucEon  –  server  room  within  the  cordon  –  no  power  to  server  room  –  limited  access.    

•  DR  site  was  in  the  midst  of  change  –  switching  to  DR  might  be  painful.  

•  Generator  onsite  within  24  hours    •  Decision  made  to  stay  with  the  producEon  system  located  on  the  customer  site.    

•  But!!!  

Page 32: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Scenario  Two  –  No  Electricity    

Do  you  have  access  to  one  of  these?  

Page 33: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Scenario  Three    •  Earthquake  –  city  wide  impact  –  huge  destrucEon  –  server  room  within  purpose  built  data  centre  –  limited  outage.    

•  Cross  city  DR  site  did  not  need  to  be  invoked  •  Within  an  hour  all  other  sites  were  working  as  normal.    

•  Local  staff  could  focus  on  family  and  conEnue  to  access  systems  remotely  as  infrastructure  allowed  

Page 34: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Lessons  Learned    •  Do  you  have  split  ProducEon  &  Test  /  DR  infrastructures?  –  Test  /  DR  ensures  that  your  DR  is  ready  for  acEon!  

•  Network  &  Infrastructure  are  constantly  used  (tested)  •  Do  you  have  a  DR  strategy?  –  GREAT!  

– Have  you  tested  it  recently?  – Did  you  deploy  a  JDE  full  package?  

•  A  ‘Purpose  Built’  data  centre  is  best  for  ProducEon  -­‐  Why?  

•  Test  your  DR  strategy  regularly    

Page 35: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

What  Worked  –  What  Didn’t?    •  Data  ReplicaEon  –  sorted  J  •  Server  images  or  backups  in  hand?  

– Where  is  the  last  full  backup  of  this  machine?  – What  do  you  mean  you  backed  up  the  structures  ONLY!!!  

– Where  are  the  JDE  installaEon  CDs?  •  Don’t  neglect  your  deployment  server  •  Ensure  your  client  technologies  are  covered    •  Any  business  criEcal  3rd  party  systems?  

 

Page 36: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Is  Distance  a  Factor?    •  Absolutely  –  you  don’t  want  your  DR  too  close  –  RIGHT?  

•  Too  far  apart  –  shipping  consideraEons    – Comms  costs  consideraEons  for  ANZ    

•  Too  close  and  both  instances  are  affected  

•  20km+    =    Ideal  minimum  separaEon  

 

Page 37: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

In  Summary    

The  solu6on  that  I  have  seen  work  best  is  as  follows.    

•  Split  ProducEon  and  Non  ProducEon  (Test/DR)  infrastructures  •  ProducEon  hosted  in  a  purpose  built  hosted  facility  where  security,  

power  circuit  redundancy,  generators  and  diesel  fuel  stocks  are  available  to  ensure  24x7  security  and  electricity  supply.  

•  Non  producEon  either  hosted  in  an  alternate  facility  or  on  site  at  the  customer’s  premises  20+  kms  from  the  producEon  data  centre.    

•  100Mb  metropolitan  LAN  or  equivalent  available  to  service  communicaEons  requirements  between  the  two  sites  

•  Test  the  soluEon  regularly  &  ensure  your  backups  are  appropriate  

 

Page 38: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Build  Your  DR  SoluEon    to  match  your    

Business  ExpectaEons    

No  CEO/CIO/CFO  affected  by  the  Christchurch  earthquakes  has  wanted  to  reduce  their  disaster  

recovery  protecEon  aeer  recent  events.      

Most  have  requested  a  full  DR  review  Many  have  implemented  stronger  soluEons.    

 

Page 39: JD Edwards & Peoplesoft 2 _ Mark Elley _ Real word experiences disaster recovery.pdf

The most comprehensive Oracle applications & technology content under one roof

Disaster  Recovery  ARE  YOU  READY