So What is FlashCopy Doing? - GSE...
-
Upload
truongtuyen -
Category
Documents
-
view
235 -
download
5
Transcript of So What is FlashCopy Doing? - GSE...
So What is FlashCopy Doing?
John Ticic
IntelliMagic
Date of presentation (1/11/2016)
Session DB
Agenda
• Observations
– Volume FlashCopy
– Data Set FlashCopy
• Summary
• Appendix
• What is FlashCopy?
– Features
– Differences
• FlashCopy Interfaces
– Commands
• Disk Back-end
John Ticic – Senior Technical Consultant
Started in Systems Programming in 1984.
Joined IntelliMagic in 2008 as a Senior Consultant
Specialties include:
Disk/Tape performance
z/OS Performance
z/OS, zSeries implementation
Presenting (I/O classes, SHARE, GSE,..)
What is FlashCopy?
FlashCopy®
• Creates image of volume or data set – With or without full physical copy
– Source & target are immediately available for use
– Does not consume resources outside DSS
Source
Target
FlashCopy – Many Features
• Copy / NoCopy
• Persistent or Temporary
Relationship
• Data set copy in addition to
volume copy
– DFSMSdss allocates and
catalogs the copy
• Multiple relationships:
– Associate single source with
more than one target
• Incremental FlashCopy:
– FC command triggers
transmission of changes since
previous FC command
• FlashCopy Consistency Group
– Create group of target
volumes that is consistent
• Reverse Restore – The relationship can be
reversed by copying modified tracks from the target to the source volume
FlashCopy – Even More Features
• Fast Reverse Restore – Used by Global Mirror
• Virtual Concurrent Copy – Concurrent Copy (CC) first
issues FlashCopy and then copies data
• Space Efficient FlashCopy – Space Efficient means target
disk space is only required when write data needs to be hardened
• Other vendors support
FlashCopy, but also have their
own point-in-time technology
• EMC
– Timefinder Mirror/Clone/Snap
• HDS/HP
– ShadowImage
Track SE not available on DS8880 family
FlashCopy - Copy
COPY option initiates full source to target copy
• Sequential process, no RAID-5 write penalty
On-demand copy when updating tracks not yet copied
• ODC rate probably decreases over time
FlashCopy relationship automatically ended when full copy is complete
Full copy & ODC may impact application I/O performance
• But, when it’s over, it’s over
FlashCopy - Copy
Process target
Process source
FlashCopy / Copy
initiated
copy
FC relationship ended (automatically)
On-demand Copy copy Copy tracks
FlashCopy - Copy
Copy activity is background, low priority.
Internal copy tasks exist per disk adapter and per disk rank.
The maximum number of tasks per adapter/rank are limited.
I/O is optimized (sequential).
FlashCopy - Nocopy
On-demand copy for tracks updated on source or target
• ODC rate fully depends on source / target write activity
• Worst case: multiple sequential write processes in parallel to volumes in same RAID array
FC relationship should be ended ‘manually’ when finished processing target
• Avoid unnecessary ODC activity
• Logical status of target is unpredictable
FlashCopy - Nocopy
On-demand Copy
Process target
Process source
FlashCopy / Nocopy
Initiated
FC relationship ended manually
FlashCopy - Nocopy
All I/O is based on destaging modified data.
Before the destage, data needs to be copied from source to target disk.
Can affect random I/O.
I/O is background and limited in concurrency.
Incremental FlashCopy
Requires persistent relationship & change recording must be on
Sets up 2 track bitmaps on source
• (1) preserves ‘t0’ status of target
• (2) keeps a record of updates after ‘t0’
Initial FC invocation results in full copy
• ODC may occur during full / delta copy; driven by (1)
Incremental FC results in ‘delta’ copy based on (2)
Incremental FlashCopy
Process target
Process source
Initial FC Incremental FC
copy
On-demand Copy copy Full Copy of tracks
Delta Copy of tracks
FlashCopy Interfaces
Interaction
• TSO/E Command – Note: Starting z/OS 2.3, only QUERY
commands remain supported.
• ANTRQST API
• REXX ANTTREXX
• DFSMSdss
• ICKDSF
• Web Service – DS8K interface
Note: RACF and/or IKJTSOxx changes may
be needed.
Sample Commands
FCESTABL Establish a FlashCopy relationship
FCESTABL SDEVN(2001) TDEVN(2804) MODE(COPY)
ANTF0001I FCESTABL COMMAND COMPLETED FOR DEVICE 2001, COMPLETION CODE: 00
Instantaneous reply. Command accepted and executed.
Copy initiated, but not finished.
When copy is complete, the relationship will be terminated. No
message will be generated.
Caution: target device will have source volume label. Possible IPL
considerations.
Sample Commands
FCWITHDR Withdraw a FlashCopy relationship
FCWITHDR TDEVN(2A06)
ANTF0001I FCWITHDR COMMAND COMPLETED FOR DEVICE 2A06,
COMPLETION CODE: 00
Terminates the FlashCopy relationship, regardless of the current status. i.e. does not wait for complete volume to be copied when COPY is specified.
Status of target volume is unknown.
But, it may have the source volume label! Possible IPL considerations.
Sample Commands
FCQUERY Query a device
FCQUERY DEVN(2001) SHOWRELS(ALL)
ANTF0421I FCQUERY Relationship 1
DEVN SSID LSS CCA CU SERIAL ACT MAX XC PC CC RV SE SEQNUM
2001 1000 00 01 2107 0000000DDK61 0 65534 N N N N NN 00000000
RELATIONSHIP DETAIL STARTING TRACK: 00000000
DEVICE LONG BUSY FOR CG: NO WRITE INHIBITED: NO
---------------------------------------------------
NO RELATIONSHIPS FOUND FOR SPECIFIED DEVICE OR STARTING ADDRESS
ANTF0001I FCQUERY COMMAND COMPLETED FOR DEVICE 2001, COMPLETION CODE: 00
Sample Commands
SSID + LSS + CCA + Serial Number uniquely identify a device.
SSID Disk Subsystem Identifier. This must be unique within an LPAR.
LSS Logical Subsystem. Unique within a disk system.
CCA Channel Connection Address. Device number within an LSS
Some customers have LSS + CCA = z/OS Device address. Nice, but not required.
Sample Commands
FCQUERY DEVN(2006) SHOWRELS(ALL)
ANTF0421I FCQUERY Relationship 1
DEVN SSID LSS CCA CU SERIAL ACT MAX XC PC CC RV SE SEQNUM
2006 1000 00 06 2107 0000000DDK61 1 65534 N P N N NN 00000000
RELATIONSHIP DETAIL STARTING TRACK: 00000000
DEVICE LONG BUSY FOR CG: NO WRITE INHIBITED: NO
---------------------------------------------------
PARTNER SOURCE TARGET S F C C P C T S F P
LSS CCA SSID START START O V O A R R W E S M
--- --- ---- -------- -------- - - - - - - - - - -
0A 06 100A 00000000 00000000 Y Y N N N N Y N N N
NO. OF TRACKS: 0006E0CD TRACKS TO COPY: 0002E16C
ESTABL: 2015/07/27 13:43:19 LAST INCR: 2015/07/27 13:43:19
ANTF0001I FCQUERY COMMAND COMPLETED FOR DEVICE 2006, COMPLETION CODE: 00
Sample Commands
FCQUERY DEVN(2006) SHOWRELS(ALL)
ANTF0421I FCQUERY Relationship 1
DEVN SSID LSS CCA CU SERIAL ACT MAX XC PC CC RV SE SEQNUM
2006 1000 00 06 2107 0000000DDK61 1 65534 N P N N NN 00000000
z/OS
Device
Disk
Device
(Source)
Sample Commands
FCQUERY DEVN(2006) SHOWRELS(ALL)
---------------------------------------------------
PARTNER SOURCE TARGET S F C C P C T S F P
LSS CCA SSID START START O V O A R R W E S M
--- --- ---- -------- -------- - - - - - - - - - -
0A 06 100A 00000000 00000000 Y Y N N N N Y N N N
NO. OF TRACKS: 0006E0CD TRACKS TO COPY: 0002E16C
Disk
Device
(Target)
Full
Volume See Appendix for full
description
30051 Cyl 12585 Cyl
Disk Back-end
What is Available?
• RMF (CMF) provide performance data –
– From the z/OS LPAR
• Based on Start Subchannels (SSCH)
• Collected per logical volume
• FICON channel data available
– From the Disk Cache statistics
• Not SSCH but Locate Records (LR)
– From the Disk internal Ranks/Arrays
SMF 74.1
SMF 73
SMF 74.5
SMF 74.8
Front-end activity Front-end activity is
the count of SSCH
issued by z/OS
(SMF74SSC.)
We need to
consolidate data from
all LPARs and isolate
single disk systems.
Will all of these I/O’s
create disk internal
I/O?
I/O Types
HDD (arrays)
Read hit Read miss
Random Write
Cache
Seq. Write
Front-end I/Os
Reads/Writes
Sequential/Random
Cache hits/misses
Back-end I/Os
Reads/Writes
Cache staging
Additional I/O for Parity
Optimized for sequential
writes
How
many?
Front-end vs Back-end activity Very application and
I/O dependent.
Sometimes the back-
end I/O rate (blue) is
higher than the front-
end (red).
Being able to see the
back-end activity
allows us to observe
FlashCopy activity.
Sample FlashCopy
Let’s look at some disk performance data that contains FlashCopy
activity.
Using RMF/CMF data, we can observe the z/OS initiated I/Os, and look
at the back-end rank data.
The back-end rank data will show us the results of z/OS initiated I/Os
(those that made it to the back-end) and any other I/O activity (e.g.
FlashCopy.)
Front-end FlashCopy is a disk
internal I/O
technology.
We will not see it in
the front-end
performance data.
But – we may suffer
performance due to
FlashCopy back-end
activity.
~14000 I/Os
Front-end and Back-end We can see significant
back-end activity
(blue).
Note. Both of these
are to the same scale.
What about read/write
MB/s? ~14,000 I/Os
~130,000 I/Os
Back-end MB/s We can also see the
significant throughput,
by drive tier (IBM
DS8870).
1200 GB 10K
RAID6
300 GB 15K
RAID6
400 GB SSD
RAID5
Let’s break it down
(Read/Write).
Back-end MB/s We see a very close
correlation between
read (red) and write
(blue) MB/s (we
expected it for
FlashCopy activity.)
Let’s break it down
further into the 3 Tiers.
Back-end Read MB/s Most of the MBs read
activity is from the
SSD Tier.
What about writes?
Back-end Write MB/s Very few MBs are
being written to SSD!
This is to be expected
but has implications.
What do you want to
use this data for?
A large amount is on
1200GB 10K HDD!
What about the I/O
breakdown?
Back-end Read I/Os This is similar to the
Read throughput
distribution (by Tier.)
What about write
I/Os?
Back-end Write I/Os This is similar to the
Write throughput
distribution (by Tier.)
Note: The scale is
different.
How do they
compare?
Let’s put them onto
the same scale.
Back-end I/O Comparison Many fewer write I/Os
(Red). But, the same
amount of data.
FlashCopy is not
simple I/O!
What does the back-
end balance look like?
Back-end Storage Pool Balance All 4 Extent Pools are
reasonable well
balanced.
It is very important to
know where the
secondary FlashCopy
devices will be.
Spread evenly to
minimize disruption.
What about z/OS
Response time?
z/OS Response Time We clearly see an
increase in z/OS
response time (e.g.
between 7:00 and
9:00 PM)
The breakdown shows
us that it is Disconnect
time.
Lets look a little
deeper.
Disconnect Time Breakdown Looking closer at
Disconnect time
shows us that our
peak is caused by
HDD read response
time.
But – is it
FlashCopy?
Conclusive Evidence?
• We’ve seen what we think is FlashCopy activity affecting our Disk
Back-end response times, and causing z/OS response time
increases.
• But, how can we be sure?
• The previously shown commands can show FlashCopy activity.
FlashCopy Sessions I’m using data
gathered by our z/OS
Collector to show
FlashCopy activity.
We clearly see that we
have FlashCopy
relationships.
And, they don’t end!
FlashCopy Relationships
• Each FlashCopy source to target activity is assigned to a
relationship (see FCQUERY output.)
• Multiple relationships per volume are possible
– COPY relationships dissolve automatically after data has been copied.
– Other relationships remain indefinitely.
FlashCopy Relationships
• Active relationships does not mean that data is always being copied.
• But, if relationships exists, data may be copied (on demand) when
not expected.
• How much data is being copied?
Sample FlashCopy - Cylinders to Copy The total number of
cylinders isn’t really
too significant, but
how long it takes until
all activity ceases can
be critical.
Note: Data is interval
based so peaks can
(will) be much higher.
What have we
observed?
FlashCopy Observations
• “Kicking off” a FlashCopy process can certainly generate a large
back-end load.
• This load is not visible from z/OS, but can be observed with the
RMF/CMF 74.8 records (ESS – rank statistics.)
• But, you can’t observe and control FlashCopy if you can’t tell what is
generating the I/O load.
• The Relationships (in this case) remain even after the data has been
copied.
Observations
Observations
• In this section, I’d like to look at some of the different FlashCopy
executions and what the results are.
• FlashCopy.
– Full Volume COPY.
– Full Volume NOCOPY.
– Incremental + Data set.
Full Volume COPY
• FlashCopy COPY
– We expect a lot of Back-end activity, but it will cease when the copies
are complete and the relationships will be dissolved.
– Let’s look at an example:
• 42 x 3390-27. 83% full
• Approximately 1200 Thousand Cylinders. (~850 GB)
• IBM DS8870. 5 minute intervals.
Full Volume COPY There are some
existing sessions, but
the 42 Full Volume
COPY sessions start
at 12:15 PM.
We are done by 4:15
PM.
How fast are the
cylinders copied?
Full Volume COPY A quick start and then
very linear until all
sessions are
completed.
Let’s look at the disk
performance data.
Full Volume COPY This is the I/O rate
generated from z/OS
to this disk system.
A spike around 3:00
PM, but very little I/O.
What does the back-
end look like?
Full Volume COPY Only 2 drive tiers.
Read MB/S.
Initial spike, but it
settles down quickly.
Initially, data is read
from the SSD, but
older data is read from
the 7.2K drives.
Note: 3:00 PM z/OS
spike not even visible!
Full Volume COPY Write MB/s.
Initially, SSDs are
used, but most of the
write I/O is to the 7.2K
drives.
This is where most of
the new data will
reside until EasyTier
moves it around!
You need to be aware
of this!
Full Volume COPY The z/OS response
time (red) isn’t
affected in this
sample.
This disk system is
not heavily loaded, so
it can cope well with
the FlashCopy load.
But it still took 4 hours
to complete (42 x
3390-27)
Full Volume NOCOPY
• FlashCopy NOCOPY
– We actually expect little activity. Only on-demand-copy.
– Note: The relationships will need to be manually terminated.
– Same example:
• 42 x 3390-27. 83% full
• Approximately 1200 Thousand Cylinders. (~850 GB)
Full Volume NOCOPY We see the additional
42 sessions starting
at 7:50.
At 10:25, the sessions
are manually
terminated.
So, how many
cylinders need to be
copied?
Full Volume NOCOPY Almost no data is
being copied.
How busy is the disk
system?
Full Volume NOCOPY Very little z/OS
initiated I/O.
How much work do
the internal disk drives
need to do?
Full Volume NOCOPY Almost no back-end
I/O!
FlashCopy NOCOPY
only generates I/O
when demand-on-
copy is required.
Combine with
INCREMENTAL(YES)
to only copy the tracks
that have changed
since the last
FlashCopy command.
Customer FlashCopy
COPY INCREMENTAL(YES)
• What does a real customer environment look like?
• IBM DS8870 hardware
• FlashCopy COPY INCREMENTAL(YES) and Data Set FlashCopy.
Data Set Copy We see very many
relationships.
It looks like it takes 1
½ hours until all
extents are copied.
Each data set extent
is defined by a
relationship.
How many cylinders
need to be copied?
Existing
COPY +
Incremental
Data Set FC
Data Set Copy Many Cylinders!
Around 14,500
Thousand Cylinders
(~ 60 x 3390-A)
Start time is around
9:10
End time is around
10:45
What does the back-
end look like?
Data Set Copy Back-end activity has
a clear starting time.
Start time is around
9:10
At 10:45, we still have
significant back-end
activity.
End time is around
11:50
Compare to Response
Time.
10:45
Data Set Copy The z/OS I/O
response time (red)
clearly spikes when
the back-end (blue)
activity increases
significantly.
It does recover, but it
will take time.
The main component
was Disconnect time
(back-end HDD
access) – not shown.
3 ms
Data Set Copy Note:
This disk systems has
a relatively low read
cache hit rate (%).
This means it will be
more influenced by
the back-end read
response times.
(i.e. more read I/Os
requiring HDD
access)
FlashCopy
Overhead
• How much overhead is there in obtaining FlashCopy information?
• The IBM DS8K ensures that host I/O activity has a higher priority
than the FlashCopy workload.
• The FCQUERY requests are also prioritized to not impact disk
response time and disk activity.
FCQUERY Time (Sec.) Be aware that the
FCQUERY command
may sometimes take a
long time to complete.
The disk system
needs to prioritize the
internal activity and
favor response time.
Other Observations
• Each data set extent to be copied is represented by a unique
relationship.
• If PPRC (Metro Mirror) replication is being used, you can query the
remote device.
– Referred to as being an “Inband” command.
– E.g.
FCQUERY DEVN(2603) +
REMOTE(YES) QRYDVC(DDK61 X'0A' X'03') +
QRYSSID(100A)
Summary
Summary
• Be aware of location of FlashCopy target volumes.
– It may be on a slower tier.
– This may be relevant when using these volumes.
• If you want to dump (e.g. to tape) the target devices, wait until a
Copy is completely finished. This will stress the back-end disk
system much less.
• Results may be different for other vendors.
Summary
• FlashCopy may impact your z/OS response time.
– Ensure that any “Pre-online” FlashCopy activities are finished well
before online start.
• Define FlashCopy source and target devices in different ranks.
– Especially important for COPY activity.
Documentation IBM DFSMS Advanced Copy Services SC23-6847
IBM IBM DS8870 Copy Services for IBM z Systems SG24-6787
IBM IBM System Storage DS8870 Performance
Whitepaper
WP102183
IBM FlashCopy in a zSeries Environment Charlie Burger
ATS
HDS ShadowImage User Guide MK-92RD8021
EMC Implementing Local Replication using EMC
Timefinder and Recoverpoint and coexistance of
RP/CDP with SRDF on Symmetrix VMAX
H8981.2
IntelliMagic Monitoring FlashCopy
Thank you
www.intellimagic.com
Note: See appendix for code
samples and additional information
Session feedback
• Please submit your feedback at
http://conferences.gse.org.uk/2016/feedback/DB
• Session is DB
Appendix
Sample TSO Command //*
//* ISSUE FC COMMANDS FOR DATA AND FTP VOLUMES
//*
//DATA EXEC PGM=IKJEFT1A
//SYSTSPRT DD SYSOUT=*
//SYSPRINT DD SYSOUT=*
//SYSTSIN DD *
FCESTABL SDEVN(2005) TDEVN(2039) MODE(COPY)
/*
Sample REXX /* REXX */
command = 'FCQUERY'
args.0 = 3
args.1 = 'DEVN(2000)'
args.2 = 'QRYINFO()'
args.3 = 'QRYSIZE(200)'
call anttrexx command,'args'
say result
say args.2
Sample Assembler ANTRQST ILK=ESSRVCS,REQUEST=FCQUERY,FORMAT=FFQMAP, X
DEVN=Dev_No,SUBCHSET=Subchan, X
QRYSIZE=FQRY_Size,QRYINFO=(4), X
RETCODE=Pret,RSNCODE=Prsn,RETINFO=Pretinfo, X
MF=(E,PLIST)
Sample DFDSS //DSCOPY EXEC PGM=ADRDSSU
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
COPY DATASET(INCLUDE(IMUSJT1.**) -
EXCL(IMUSJT1.BRODCAST, -
IMUSJT1.ZF*.**) ) -
RENAMEU(IMUSJT2 ) -
FR(REQ) FCNOCOPY -
NOPACKING(**) -
ADMIN -
CATALOG -
ALLDATA(*) ALLEXCP TOL(ENQF)
/*
Sample ICKDSF //*
//* FCQUERY
//*
//QUERY EXEC PGM=ICKDSF
//SYSPRINT DD SYSOUT=*
//VOL1 DD DISP=SHR,VOL=SER=JOHN01,UNIT=3390
//SYSIN DD *
FLASHCPY QUERY -
DDNAME(VOL1)
/*
Detailed Output - Data Set Copy Device 6020 SS=0 ABCDEF Source/Target SSID=6000/0000 CCA=20/00 LSS=60/00 Rels = 235
Rel 1 LSS=62 CCA=22 SSID=6200/6200
FC cap. PPRC Sess. FCPY Sess. SRC. Full Vol. Persist. INCR(YES).
SEQ= 0 TRKS= 3606120 LSIG= 0 MSIG 0
Established = 2016/06/13 17:59:18 Recent Increment = 2016/06/13 17:59:18
Source CCHH = 00000000 Target CCHH = 00000000
Source device 6020 SSID 6000 LSS 60 + CCA 20
Target device SSID 6200 LSS 62 + CCA 22
Full Volume. Incremental = yes.
Source device = 3,606,120 tracks (3390-A 229,763 Cylinders)
Data Set Copy Device 6020 SS=0 ABCDEF Source/Target SSID=6000/0000 CCA=20/00 LSS=60/00 Rels = 235
Rel 2 LSS=62 CCA=21 SSID=6200/6200
FC cap. PPRC Sess. FCPY Sess. SRC.
SEQ= 0 TRKS= 30 LSIG= 30 MSIG 0
Established = 2016/06/14 08:16:15 Recent Increment = 2016/06/14 08:16:15
Source CCHH = 00780009 Target CCHH = 00780009
Disk
Timestamp
Tracks = 30 30 Tracks will be copied
Data Set Copy Device 6020 SS=0 ABCDEF Source/Target SSID=6000/0000 CCA=20/00 LSS=60/00 Rels = 235
Rel 235 LSS=62 CCA=21 SSID=6200/6200
FC cap. PPRC Sess. FCPY Sess. SRC.
SEQ= 0 TRKS= 3446448 LSIG= 3446448 MSIG 0
Established = 2016/06/14 08:16:16 Recent Increment = 2016/06/14 08:16:16
Source CCHH = 2859000C Target CCHH = 2859000C
Disk
Timestamp
Tracks= 3,446,448 3,446,448 Tracks will be copied
ANTF042I
ANTF0421I FCQUERY Relationship 1
DEVN SSID LSS CCA CU SERIAL ACT MAX XC PC CC RV SE SEQNUM
2006 1000 00 06 2107 0000000DDK61 1 65534 N P N N NN 00000000
RELATIONSHIP DETAIL STARTING TRACK: 00000000
DEVICE LONG BUSY FOR CG: NO WRITE INHIBITED: NO
---------------------------------------------------
PARTNER SOURCE TARGET S F C C P C T S F P
LSS CCA SSID START START O V O A R R W E S M
--- --- ---- -------- -------- - - - - - - - - - -
0A 06 100A 00000000 00000000 Y Y N N N N Y N N N
SO is one of the following value
Y Indicates that this is a source relationship.
N Indicates that this is a target relationship.
ANTF042I
FV is one of the following values:
Y Indicates that this is a full volume relationship.
N Indicates that this is not a full volume relationship.
CO is one of the following values:
Y Indicates that this is a COPY relationship.
N Indicates that this is a NOCOPY relationship.
CA is one of the following values:
Y Indicates that the background copy is currently active for this relationship.
N Indicates that the background copy is not currently active for this relationship.
PR is one of the following values:
Y Indicates that this is a persistent relationship.
N Indicates this is not a persistent relationship.
ANTF042I
CR is one of the following values
Y Indicates that change recording is active for this relationship (incremental).
N Indicates that change recording is not active for this relationship.
TW is one of the following values:
Y Indicates that the target of the relationship is writable.
N Indicates that the target is write-inhibited.
SE is one of the following values:
N Indicates that the target of this relationship is not a space efficient volume.
F Indicates that the target of this relationship is space efficient and the relationship will be
failed if the repository becomes full.
I Indicates that the target of this relationship is space efficient and the source will become
write-inhibited if the repository becomes full.
ANTF042I
FS is one of the following values:
Y Indicates that the relationship is currently in a failed state.
N Indicates that the relationship is not currently in a failed state.
PM represents the Preserve Mirror state of the relationship. r one of the following
values:
N Indicates that the relationship was not established as a Preserve Mirror operation because
Preserve Mirror was not requested or Preserve Mirror Preferred was specified but could not be
accomplished.
P Indicates that the relationship was established as a Preserve Mirror operation because of a
Preserve Mirror Preferred request.
R Indicates that the relationship was established as a Preserve Mirror operation because of a
Preserve Mirror Required request.
S Indicates that this is a remote relationship established between two PPRC secondary devices as
part of a Preserve Mirror operation or the PM relationship type cannot be determined because of a
conflict in the source and/or target PPRC copy status.