IBM Spectrum Scale 5.0.2: Administration Guide€¦ · IBM Spectrum Scale V ersion 5.0.2 Administra...

IBM Spectrum ScaleVersion 5.0.2

Administration Guide

SC27-9288-03

IBM

NoteBefore using this information and the product it supports, read the information in “Notices” on page 753.

This edition applies to version 5 release 0 modification 2 of the following products, and to all subsequent releasesand modifications until otherwise indicated in new editions:v IBM Spectrum Scale ordered through Passport Advantage® (product number 5725-Q01)v IBM Spectrum Scale ordered through AAS/eConfig (product number 5641-GPF)v IBM Spectrum Scale for Linux on Z (product number 5725-S28)v IBM Spectrum Scale for IBM ESS (product number 5765-ESS)

Significant changes or additions to the text and illustrations are indicated by a vertical line (|) to the left of thechange.

IBM welcomes your comments; see the topic “How to send your comments” on page xxvi. When you sendinformation to IBM, you grant IBM a nonexclusive right to use or distribute the information in any way it believesappropriate without incurring any obligation to you.

© Copyright IBM Corporation 2014, 2019.US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contractwith IBM Corp.

Contents

Tables . . . . . . . . . . . . . . . xi

About this information. . . . . . . . xiiiPrerequisite and related information . . . . . xxivConventions used in this information . . . . . xxvHow to send your comments . . . . . . . . xxvi

Summary of changes . . . . . . . xxvii

Chapter 1. Configuring the GPFS cluster 1Creating your GPFS cluster . . . . . . . . . 1Displaying cluster configuration information . . . 1

Basic configuration information . . . . . . . 1Information about protocol nodes . . . . . . 2

Adding nodes to a GPFS cluster . . . . . . . . 2Deleting nodes from a GPFS cluster . . . . . . 3Changing the GPFS cluster configuration data . . . 4Security mode . . . . . . . . . . . . . 16Running IBM Spectrum Scale commands withoutremote root login . . . . . . . . . . . . 17

Configuring sudo . . . . . . . . . . . 17Configuring the cluster to use sudo wrapperscripts . . . . . . . . . . . . . . . 18Configuring IBM Spectrum Scale GUI to usesudo wrapper . . . . . . . . . . . . 19Configuring a cluster to stop using sudo wrapperscripts . . . . . . . . . . . . . . . 19Root-level processes that call administrationcommands directly . . . . . . . . . . . 20

Node quorum considerations . . . . . . . . 20Node quorum with tiebreaker considerations . . . 20Displaying and changing the file system managernode. . . . . . . . . . . . . . . . . 21Determining how long mmrestripefs takes tocomplete . . . . . . . . . . . . . . . 22Starting and stopping GPFS . . . . . . . . . 22Shutting down an IBM Spectrum Scale cluster . . . 23

Chapter 2. Configuring the CES andprotocol configuration . . . . . . . . 25Configuring Cluster Export Services . . . . . . 25

Setting up Cluster Export Services shared rootfile system. . . . . . . . . . . . . . 25Configuring Cluster Export Services nodes . . . 26Configuring CES protocol service IP addresses . 26CES IP aliasing to network adapters on protocolnodes . . . . . . . . . . . . . . . 28Deploying Cluster Export Services packages onexisting IBM Spectrum Scale 4.1.1 and later nodes 32Verifying the final CES configurations . . . . 33

Creating and configuring file systems and filesetsfor exports. . . . . . . . . . . . . . . 33Configuring with the installation toolkit . . . . . 33

Deleting a Cluster Export Services node from anIBM Spectrum Scale cluster . . . . . . . . . 34Setting up Cluster Export Services groups in an IBMSpectrum Scale cluster . . . . . . . . . . . 34

Chapter 3. Configuring and tuning yoursystem for GPFS . . . . . . . . . . 37General system configuration and tuningconsiderations . . . . . . . . . . . . . 37

Clock synchronization . . . . . . . . . . 37GPFS administration security . . . . . . . 37Cache usage . . . . . . . . . . . . . 38Access patterns . . . . . . . . . . . . 40Aggregate network interfaces . . . . . . . 40Swap space . . . . . . . . . . . . . 40

Linux configuration and tuning considerations . . 41updatedb considerations . . . . . . . . . 41Memory considerations . . . . . . . . . 41GPFS helper threads . . . . . . . . . . 41Communications I/O . . . . . . . . . . 42Disk I/O . . . . . . . . . . . . . . 42

AIX configuration and tuning considerations . . . 43GPFS use with Oracle . . . . . . . . . . 43

Chapter 4. Parameters for performancetuning and optimization . . . . . . . 45Tuning parameters change history . . . . . . . 47

Chapter 5. Ensuring high availability ofthe GUI service . . . . . . . . . . . 53

Chapter 6. Configuring and tuning yoursystem for Cloud services . . . . . . 55Designating the Cloud services nodes . . . . . 55Starting up the Cloud services software . . . . . 56Managing a cloud storage account. . . . . . . 57

Amazon S3 . . . . . . . . . . . . . 57Swift3 account . . . . . . . . . . . . 58IBM Cloud Object Storage . . . . . . . . 58Openstack Swift . . . . . . . . . . . . 59

Defining cloud storage access points (CSAP) . . . 60Creating Cloud services . . . . . . . . . . 61Configuring Cloud services with SKLM (optional) 62Binding your file system or fileset to the Cloudservice by creating a container pair set . . . . . 63Backing up the Cloud services database to the cloud 66Backing up the Cloud services configuration . . . 66Configuring the maintenance windows . . . . . 67Enabling a policy for Cloud data sharing exportservice . . . . . . . . . . . . . . . . 69Tuning Cloud services parameters . . . . . . . 70Integrating Cloud services metrics with theperformance monitoring tool . . . . . . . . 72

GPFS-based configuration . . . . . . . . 73

© Copyright IBM Corp. 2014, 2019 iii

||||||||

File-based configuration . . . . . . . . . 74Setting up Transparent cloud tiering service on aremotely mounted client . . . . . . . . . . 75Deploying WORM solutions . . . . . . . . . 77

Creating immutable filesets and files . . . . . 77Setting up Transparent cloud tiering for WORMsolutions . . . . . . . . . . . . . . 78

Chapter 7. Configuring file auditlogging . . . . . . . . . . . . . . 87Enabling file audit logging on a file system. . . . 87Disabling file audit logging on a file system . . . 87Disabling the message queue for the cluster . . . 87Actions taken when enabling the message queueand file audit logging . . . . . . . . . . . 88Enabling and disabling file audit logging using theGUI . . . . . . . . . . . . . . . . . 90Viewing file systems that have file audit loggingenabled with the GUI . . . . . . . . . . . 91Enabling file audit logging for a remotely mountedfile system. . . . . . . . . . . . . . . 91

Chapter 8. Configuring Active FileManagement . . . . . . . . . . . . 93Configuration parameters for AFM . . . . . . 93Parallel data transfer configuration parameters forAFM . . . . . . . . . . . . . . . . 97Configuration changes in an existing AFMrelationship . . . . . . . . . . . . . . 98

Gateway nodes in the cache cluster . . . . . 98The NFS server at the home cluster . . . . . 99

Chapter 9. Configuring AFM-based DR 101Configuration parameters for AFM-based DR . . 101Parallel data transfer configuration parameters forAFM-based DR . . . . . . . . . . . . . 103Changing configuration in an existing AFM DRrelationship . . . . . . . . . . . . . . 104

Changing NFS server at secondary . . . . . 104Changing gateway nodes in primary . . . . 104

Chapter 10. Tuning for Kernel NFSbackend on AFM and AFM DR . . . . 107Tuning the gateway node on the NFS client . . . 107Tuning on both the NFS client (gateway) and theNFS server (the home/secondary cluster) . . . . 107Tuning the NFS server on the home/secondarycluster or the NFS server . . . . . . . . . 108

Chapter 11. Performing GPFSadministration tasks . . . . . . . . 111Requirements for administering a GPFS file system 111

adminMode configuration attribute . . . . . 112Common GPFS command principles. . . . . . 113

Specifying nodes as input to GPFS commands 113Stanza files . . . . . . . . . . . . . 114Listing active IBM Spectrum Scale commands 116

Chapter 12. Verifying networkoperation with the mmnetverifycommand . . . . . . . . . . . . . 117

Chapter 13. Managing file systems 119Mounting a file system . . . . . . . . . . 119

Mounting a file system on multiple nodes . . . 120Mount options specific to IBM Spectrum Scale 120Mounting a file system through GUI . . . . 121Changing a file system mount point on protocolnodes . . . . . . . . . . . . . . . 122

Unmounting a file system . . . . . . . . . 122Unmounting a file system on multiple nodes 122Unmounting a file system through GUI . . . 123

Deleting a file system . . . . . . . . . . 123Determining which nodes have a file systemmounted . . . . . . . . . . . . . . . 124Checking and repairing a file system . . . . . 124Dynamic validation of descriptors on disk. . . . 126File system maintenance mode . . . . . . . 126Listing file system attributes . . . . . . . . 129Modifying file system attributes . . . . . . . 130Querying and changing file replication attributes 130

Querying file replication. . . . . . . . . 131Changing file replication attributes . . . . . 131

Using Direct I/O on a file in a GPFS file system 131File compression . . . . . . . . . . . . 132Setting the Quality of Service for I/O operations(QoS) . . . . . . . . . . . . . . . . 138Restriping a GPFS file system . . . . . . . . 140Querying file system space . . . . . . . . . 141Querying and reducing file system fragmentation 142

Querying file system fragmentation . . . . . 143Reducing file system fragmentation . . . . . 143

Protecting data in a file system using backup. . . 144Protecting data in a file system using themmbackup command . . . . . . . . . 144Backing up a file system using the GPFS policyengine . . . . . . . . . . . . . . . 150Backing up file system configurationinformation . . . . . . . . . . . . . 150Using APIs to develop backup applications . . 151

Scale Out Backup and Restore (SOBAR) . . . . 151Scheduling backups using IBM Spectrum Protectscheduler . . . . . . . . . . . . . . . 152Configuration reference for using IBM SpectrumProtect with IBM Spectrum Scale . . . . . . . 152

Options in the IBM Spectrum Protectconfiguration file dsm.sys . . . . . . . . 153Options in the IBM Spectrum Protectconfiguration file dsm.opt . . . . . . . . 154Base IBM Spectrum Protect client configurationfiles for IBM Spectrum Scale usage . . . . . 155

Restoring a subset of files or directories from alocal file system snapshot . . . . . . . . . 156Restoring a subset of files or directories from alocal fileset snapshot . . . . . . . . . . . 157Restoring a subset of files or directories from localsnapshots using the sample script . . . . . . 159Creating and managing file systems using GUI . . 159

iv IBM Spectrum Scale 5.0.2: Administration Guide

|||

|||

||

Chapter 14. File system formatchanges between versions of IBMSpectrum Scale . . . . . . . . . . 165

Chapter 15. Managing disks . . . . . 169Displaying disks in a GPFS cluster . . . . . . 169Adding disks to a file system . . . . . . . . 170Deleting disks from a file system . . . . . . . 170Replacing disks in a GPFS file system . . . . . 172Additional considerations for managing disks . . 174Displaying GPFS disk states . . . . . . . . 174

Disk availability . . . . . . . . . . . 174Disk status . . . . . . . . . . . . . 174

Changing GPFS disk states and parameters . . . 175Changing your NSD configuration . . . . . . 177Changing NSD server usage and failback . . . . 178Enabling and disabling Persistent Reserve . . . . 178

Chapter 16. Managing protocolservices. . . . . . . . . . . . . . 181Configuring and enabling SMB and NFS protocolservices . . . . . . . . . . . . . . . 181Configuring and enabling the Object protocolservice. . . . . . . . . . . . . . . . 182

Performance tuning for object services . . . . 183Configuring and enabling the BLOCK service . . 183Disabling protocol services . . . . . . . . . 185

Chapter 17. Managing protocol userauthentication . . . . . . . . . . . 187Setting up authentication servers to configureprotocol user access . . . . . . . . . . . 187

Integrating with AD server . . . . . . . . 187Integrating with LDAP server . . . . . . . 188Integrating with Keystone Identity Service . . 193

Configuring authentication and ID mapping for fileaccess . . . . . . . . . . . . . . . . 194

Prerequisite for configuring Kerberos-basedSMB access . . . . . . . . . . . . . 195Configuring AD-based authentication for fileaccess . . . . . . . . . . . . . . . 196Configuring LDAP-based authentication for fileaccess . . . . . . . . . . . . . . . 203Configuring NIS-based authentication . . . . 208Authentication considerations for NFSv4 basedaccess . . . . . . . . . . . . . . . 209Prerequisites for configuring Kerberos basedNFS access . . . . . . . . . . . . . 209

Managing user-defined authentication . . . . . 210Configuring authentication for object access . . . 214

Configuring local authentication for objectaccess . . . . . . . . . . . . . . . 215Configuring an AD-based authentication forobject access . . . . . . . . . . . . . 216Configuring an LDAP-based authentication forobject access . . . . . . . . . . . . . 219Configuring object authentication with anexternal keystone server . . . . . . . . . 221Creating object accounts . . . . . . . . . 222

Managing object users, roles, and projects . . . 223Deleting expired tokens . . . . . . . . . 226

Deleting the authentication and the ID mappingconfiguration . . . . . . . . . . . . . 227Listing the authentication configuration . . . . 228Verifying the authentication services configured inthe system . . . . . . . . . . . . . . 229Modifying the authentication method . . . . . 230Authentication limitations . . . . . . . . . 230

Chapter 18. Managing protocol dataexports . . . . . . . . . . . . . . 235Managing SMB shares . . . . . . . . . . 235

Creating SMB share . . . . . . . . . . 235Changing SMB share configuration . . . . . 236Creating SMB share ACLs . . . . . . . . 236Removing SMB shares . . . . . . . . . 237Listing SMB shares . . . . . . . . . . 237Managing SMB shares using MMC . . . . . 237

Managing NFS exports . . . . . . . . . . 241Creating NFS exports. . . . . . . . . . 241Changing NFS export configuration . . . . . 242Removing NFS exports . . . . . . . . . 242Listing NFS exports . . . . . . . . . . 243GUI navigation for NFS exports . . . . . . 243Making bulk changes to NFS exports . . . . 243

Multiprotocol exports . . . . . . . . . . 246Multiprotocol export considerations . . . . . . 246

Chapter 19. Managing object storage 249Understanding and managing Object services . . 249Understanding the mapping of OpenStackcommands to IBM Spectrum Scale administratorcommands . . . . . . . . . . . . . . 251Changing Object configuration values . . . . . 252Changing the object base configuration to enableS3 API. . . . . . . . . . . . . . . . 252Configuring OpenStack EC2 credentials . . . . 253Managing OpenStack access control lists using S3API. . . . . . . . . . . . . . . . . 253Managing object capabilities . . . . . . . . 254Managing object versioning . . . . . . . . 255

Enabling object versioning . . . . . . . . 255Disabling object versioning . . . . . . . . 256Creating a version of an object: Example . . . 256

Mapping of storage policies to filesets . . . . . 257Administering storage policies for object storage 257

Creating storage policy for object compression 258Creating storage policy for object encryption 259

Adding a region in a multi-region objectdeployment . . . . . . . . . . . . . . 260Administering a multi-region object deploymentenvironment. . . . . . . . . . . . . . 261Unified file and object access in IBM SpectrumScale . . . . . . . . . . . . . . . . 262

Enabling object access to existing filesets . . . 262Identity management modes for unified file andobject access . . . . . . . . . . . . . 264Authentication in unified file and object access 269Validating shared authentication ID mapping 269

Contents v

The objectizer process . . . . . . . . . 271File path in unified file and object access . . . 272Administering unified file and object access . . 273In-place analytics using unified file and objectaccess . . . . . . . . . . . . . . . 280Limitations of unified file and object access . . 281Constraints applicable to unified file and objectaccess . . . . . . . . . . . . . . . 282Data ingestion examples. . . . . . . . . 283curl commands for unified file and object accessrelated user tasks . . . . . . . . . . . 284

Configuration files for IBM Spectrum Scale forobject storage . . . . . . . . . . . . . 284Backing up and restoring object storage . . . . 288

Backing up the object storage . . . . . . . 288Restoring the object storage . . . . . . . 290

Configuration of object for isolated node andnetwork groups . . . . . . . . . . . . 293Enabling the object heatmap policy . . . . . . 294

Chapter 20. Managing GPFS quotas 297Enabling and disabling GPFS quota management 297Default quotas . . . . . . . . . . . . . 298Implications of quotas for different protocols . . . 300Explicitly establishing and changing quotas . . . 301Setting quotas for users on a per-project basis . . 302Checking quotas . . . . . . . . . . . . 304Listing quotas . . . . . . . . . . . . . 305Activating quota limit checking . . . . . . . 306Deactivating quota limit checking . . . . . . 307Changing the scope of quota limit checking . . . 307Creating file system quota reports . . . . . . 307Restoring quota files . . . . . . . . . . . 308

Chapter 21. Managing GUI users . . . 311

Chapter 22. Managing GPFS accesscontrol lists . . . . . . . . . . . . 315Traditional GPFS ACL administration . . . . . 315

Setting traditional GPFS access control lists . . 316Displaying traditional GPFS access control lists 317Applying an existing traditional GPFS accesscontrol list . . . . . . . . . . . . . 317Changing traditional GPFS access control lists 318Deleting traditional GPFS access control lists 318

NFS V4 ACL administration . . . . . . . . 319NFS V4 ACL Syntax . . . . . . . . . . 319NFS V4 ACL translation . . . . . . . . . 321Setting NFS V4 access control lists . . . . . 322Displaying NFS V4 access control lists . . . . 322Applying an existing NFS V4 access control list 322Changing NFS V4 access control lists . . . . 322Deleting NFS V4 access control lists . . . . . 323Considerations when using GPFS with NFS V4ACLs . . . . . . . . . . . . . . . 323

Authorizing protocol users . . . . . . . . . 323Authorizing file protocol users . . . . . . 323Authorizing object users. . . . . . . . . 333Authorization limitations . . . . . . . . 339

Chapter 23. Native NFS and GPFS . . 341Exporting a GPFS file system using NFS . . . . 341

Export considerations . . . . . . . . . 342NFS usage of GPFS cache . . . . . . . . . 344Synchronous writing using NFS . . . . . . . 344Unmounting a file system after NFS export . . . 344NFS automount considerations . . . . . . . 345Clustered NFS and GPFS on Linux . . . . . . 345

Chapter 24. Considerations for GPFSapplications . . . . . . . . . . . . 347Exceptions to Open Group technical standards . . 347Determining if a file system is controlled by GPFS 347Exceptions and limitations to NFS V4 ACLssupport . . . . . . . . . . . . . . . 348

Linux ACLs and extended attributes. . . . . 348General CES NFS Linux limitations . . . . . . 349Considerations for the use of direct I/O(O_DIRECT). . . . . . . . . . . . . . 349

Chapter 25. Accessing a remote GPFSfile system . . . . . . . . . . . . 351Remote user access to a GPFS file system . . . . 353Using NFS/SMB protocol over remote clustermounts . . . . . . . . . . . . . . . 354

Configuring protocols on a separate cluster . . 355Managing multi-cluster protocol environments 356Upgrading multi-cluster environments . . . . 357Limitations of protocols on remotely mountedfile systems . . . . . . . . . . . . . 357

Mounting a remote GPFS file system . . . . . 358Managing remote access to a GPFS file system . . 360Using remote access with multiple networkdefinitions . . . . . . . . . . . . . . 360Using multiple security levels for remote access 362Changing security keys with remote access . . . 363NIST compliance . . . . . . . . . . . . 364Important information about remote access . . . 365

Chapter 26. Information lifecyclemanagement for IBM Spectrum Scale . 367Storage pools . . . . . . . . . . . . . 367

Internal storage pools . . . . . . . . . 368External storage pools . . . . . . . . . 373

Policies for automating file management . . . . 374Overview of policies . . . . . . . . . . 374Policy rules . . . . . . . . . . . . . 375The mmapplypolicy command and policy rules 395Policy rules: Examples and tips . . . . . . 398Managing policies . . . . . . . . . . . 403Working with external storage pools. . . . . 407Backup and restore with storage pools . . . . 412ILM for snapshots . . . . . . . . . . . 413

Filesets . . . . . . . . . . . . . . . 414Fileset namespace . . . . . . . . . . . 415Filesets and quotas . . . . . . . . . . 416Filesets and storage pools . . . . . . . . 416Filesets and global snapshots . . . . . . . 416Fileset-level snapshots . . . . . . . . . 417

vi IBM Spectrum Scale 5.0.2: Administration Guide

Filesets and backup . . . . . . . . . . 417Managing filesets . . . . . . . . . . . 418

Immutability and appendOnly features . . . . . 421

Chapter 27. Creating and maintainingsnapshots of file systems . . . . . . 425Creating a snapshot . . . . . . . . . . . 425Listing snapshots . . . . . . . . . . . . 426Restoring a file system from a snapshot . . . . 427Reading a snapshot with the policy engine . . . 428Linking to a snapshot . . . . . . . . . . 428Deleting a snapshot . . . . . . . . . . . 429Managing snapshots using IBM Spectrum ScaleGUI . . . . . . . . . . . . . . . . 430

Chapter 28. Creating and managingfile clones . . . . . . . . . . . . . 433Creating file clones . . . . . . . . . . . 433Listing file clones . . . . . . . . . . . . 434Deleting file clones . . . . . . . . . . . 435Splitting file clones from clone parents . . . . . 435File clones and disk space management . . . . 435File clones and snapshots . . . . . . . . . 435File clones and policy files . . . . . . . . . 436

Chapter 29. Scale Out Backup andRestore (SOBAR). . . . . . . . . . 437Backup procedure with SOBAR . . . . . . . 437Restore procedure with SOBAR . . . . . . . 439

Chapter 30. Data Mirroring andReplication . . . . . . . . . . . . 443General considerations for using storage replicationwith GPFS . . . . . . . . . . . . . . 444Data integrity and the use of consistency groups 444Handling multiple versions of IBM Spectrum Scaledata . . . . . . . . . . . . . . . . 444Continuous Replication of IBM Spectrum Scaledata . . . . . . . . . . . . . . . . 445

Synchronous mirroring with GPFS replication 445Synchronous mirroring utilizing storage basedreplication . . . . . . . . . . . . . 455Point In Time Copy of IBM Spectrum Scale data 463

Chapter 31. Implementing a clusteredNFS environment on Linux . . . . . 467NFS monitoring . . . . . . . . . . . . 467NFS failover . . . . . . . . . . . . . . 467NFS locking and load balancing . . . . . . . 467CNFS network setup . . . . . . . . . . . 468CNFS setup . . . . . . . . . . . . . . 468CNFS administration . . . . . . . . . . . 469

Chapter 32. Implementing ClusterExport Services . . . . . . . . . . 471CES features. . . . . . . . . . . . . . 471

CES cluster setup . . . . . . . . . . . 471CES network configuration . . . . . . . . 472

CES address failover and distribution policies 473CES protocol management . . . . . . . . 474CES management and administration . . . . 475

CES NFS support . . . . . . . . . . . . 475CES SMB support . . . . . . . . . . . . 477CES OBJ support . . . . . . . . . . . . 478Migration of CNFS clusters to CES clusters . . . 481

Chapter 33. Identity management onWindows . . . . . . . . . . . . . 485Auto-generated ID mappings . . . . . . . . 485Installing Windows IDMU . . . . . . . . . 485Configuring ID mappings in IDMU . . . . . . 486

Chapter 34. Protocols cluster disasterrecovery . . . . . . . . . . . . . 489Protocols cluster disaster recovery limitations andprerequisites. . . . . . . . . . . . . . 489Example setup for protocols disaster recovery . . 490Setting up gateway nodes to ensure clustercommunication during failover . . . . . . . 491Creating the inband disaster recovery setup . . . 491Creating the outband disaster recovery setup. . . 493Performing failover for protocols cluster whenprimary cluster fails . . . . . . . . . . . 495

Re-create file export configuration . . . . . 495Restore file export configuration . . . . . . 495

Performing failback to old primary for protocolscluster . . . . . . . . . . . . . . . . 496

Re-create file protocol configuration for oldprimary . . . . . . . . . . . . . . 496Restore file protocol configuration for oldprimary . . . . . . . . . . . . . . 497

Performing failback to new primary for protocolscluster . . . . . . . . . . . . . . . . 499

Re-create file protocol configuration for newprimary . . . . . . . . . . . . . . 499Restore file protocol configuration for newprimary . . . . . . . . . . . . . . 502

Backing up and restoring protocols and CESconfiguration information . . . . . . . . . 505Updating protocols and CES configurationinformation . . . . . . . . . . . . . . 506Protocols and cluster configuration data requiredfor disaster recovery . . . . . . . . . . . 506

Object data required for protocols cluster DR 506SMB data required for protocols cluster DR . . 512NFS data required for protocols cluster DR . . 514Authentication related data required forprotocols cluster DR . . . . . . . . . . 515CES data required for protocols cluster DR . . 516

Chapter 35. File Placement Optimizer 519Distributing data across a cluster . . . . . . . 523FPO pool file placement and AFM . . . . . . 524Configuring FPO . . . . . . . . . . . . 524

Configuring IBM Spectrum Scale Clusters . . . 524Basic Configuration Recommendations . . . . 529Configuration and tuning of Hadoop workloads 540

Contents vii

Configuration and tuning of databaseworkloads . . . . . . . . . . . . . 541Configuring and tuning SparkWorkloads . . . 541

Ingesting data into IBM Spectrum Scale clusters 542Exporting data out of IBM Spectrum Scale clusters 542Upgrading FPO . . . . . . . . . . . . 542Monitoring and administering IBM Spectrum ScaleFPO clusters. . . . . . . . . . . . . . 545

Rolling upgrades . . . . . . . . . . . 546The IBM Spectrum Scale FPO cluster . . . . 548Failure detection . . . . . . . . . . . 550Disk Failures . . . . . . . . . . . . 550Node failure. . . . . . . . . . . . . 553Handling multiple nodes failure . . . . . . 554Network switch failure . . . . . . . . . 555Data locality. . . . . . . . . . . . . 555Disk Replacement . . . . . . . . . . . 564

Auto recovery . . . . . . . . . . . . . 566Failure and recovery . . . . . . . . . . 566QoS support for autorecovery . . . . . . . 568

Restrictions . . . . . . . . . . . . . . 568

Chapter 36. Encryption . . . . . . . 569Encryption keys . . . . . . . . . . . . 569Encryption policies . . . . . . . . . . . 570Encryption policy rules . . . . . . . . . . 570Preparation for encryption . . . . . . . . . 575Establishing an encryption-enabled environment 581

Simplified setup: Using SKLM with aself-signed certificate . . . . . . . . . . 581Simplified setup: Using SKLM with a certificatechain . . . . . . . . . . . . . . . 588Simplified setup: Valid and invalidconfigurations . . . . . . . . . . . . 597Simplified setup: Accessing a remote file system 600Simplified setup: Doing other tasks . . . . . 604Regular setup: Using SKLM with a self-signedcertificate . . . . . . . . . . . . . . 610Regular setup: Using SKLM with a certificatechain . . . . . . . . . . . . . . . 618Configuring encryption with SKLM v2.7 or later 627Configuring encryption with the Vormetric DSMkey server . . . . . . . . . . . . . 630

Certificate expiration warnings . . . . . . . 637Renewing client and server certificates . . . . . 640

Certificate expiration errors. . . . . . . . 640Renewing expired server certificates . . . . . 641Renewing expired client certificates . . . . . 647

Encryption hints . . . . . . . . . . . . 651Secure deletion . . . . . . . . . . . . . 652Encryption and standards compliance . . . . . 653

Encryption and FIPS-140-2 certification . . . . 654Encryption and NIST SP800-131A compliance 654

Encryption in a multicluster environment . . . . 654Encryption in a Disaster Recovery environment 654Encryption and backup/restore . . . . . . . 655Encryption and snapshots . . . . . . . . . 655Encryption and a local read-only cache (LROC)device . . . . . . . . . . . . . . . . 655Encryption and external pools . . . . . . . . 656Encryption requirements and limitations . . . . 656

Chapter 37. Managing certificates tosecure communications between GUIweb server and web browsers . . . . 659

Chapter 38. Securing protocol data 661Planning for protocol data security . . . . . . 663Configuring protocol data security . . . . . . 663

Enabling secured connection between the IBMSpectrum Scale system and authenticationserver . . . . . . . . . . . . . . . 663Securing data transfer . . . . . . . . . 666Securing NFS data transfer . . . . . . . . 666Securing SMB data transfer. . . . . . . . 669Secured object data transfer . . . . . . . 669

Data security limitations. . . . . . . . . . 669

Chapter 39. Cloud services:Transparent cloud tiering and Clouddata sharing . . . . . . . . . . . . 671Administering files for Transparent cloud tiering 671

Applying a policy on a Transparent cloudtiering node . . . . . . . . . . . . . 671Migrating files to the cloud storage tier. . . . 674Pre-migrating files to the cloud storage tier . . 674Recalling files from the cloud storage tier . . . 676Reconciling files between IBM Spectrum Scalefile system and cloud storage tier. . . . . . 677Cleaning up files transferred to the cloudstorage tier . . . . . . . . . . . . . 678Deleting cloud objects . . . . . . . . . 678Managing reversioned files . . . . . . . . 679Listing files migrated to the cloud storage tier 680Restoring files . . . . . . . . . . . . 680Restoring Cloud services configuration . . . . 682Checking the Cloud services database integrity 682Manual recovery of Transparent cloud tieringdatabase . . . . . . . . . . . . . . 683Scale out backup and restore (SOBAR) forCloud services . . . . . . . . . . . . 683

Cloud data sharing . . . . . . . . . . . 696Listing files exported to the cloud . . . . . 697Importing cloud objects exported through anold version of Cloud data sharing . . . . . 700

Administering Transparent cloud tiering and Clouddata sharing services . . . . . . . . . . . 700

Stopping Cloud services software . . . . . 700Monitoring the health of Cloud servicessoftware . . . . . . . . . . . . . . 701Checking the Cloud services version . . . . 702

Known limitations of Cloud services . . . . . 703

Chapter 40. Managing file auditlogging . . . . . . . . . . . . . . 705Starting consumers in file audit logging . . . . 705Stopping consumers in file audit logging . . . . 705Displaying topics that are registered in the messagequeue for file audit logging . . . . . . . . 705Enabling file audit logging on a new spectrumscalecluster node . . . . . . . . . . . . . . 706

viii IBM Spectrum Scale 5.0.2: Administration Guide

||

||

||

Managing the list of monitored events . . . . . 706Designating additional broker nodes for increasedperformance . . . . . . . . . . . . . . 707

Chapter 41. Performing a watch withwatch folder . . . . . . . . . . . . 709

Chapter 42. Administering AFM. . . . 711Creating an AFM relationship by using the NFSprotocol . . . . . . . . . . . . . . . 711

Setting up the home cluster. . . . . . . . 711Setting up the cache cluster. . . . . . . . 712Example of creating an AFM relationship byusing the NFS protocol . . . . . . . . . 713

Creating an AFM relationship by using GPFSprotocol . . . . . . . . . . . . . . . 715

Setting up the home cluster . . . . . . . 715Setting up the cache cluster. . . . . . . . 715Example of creating an AFM relationship byusing the GPFS protocol . . . . . . . . . 715

Chapter 43. Administering AFM DR 717Creating an AFM-based DR relationship . . . . 717Converting GPFS filesets to AFM DR . . . . . 718Converting AFM relationship to AFM DR . . . . 719

Chapter 44. Highly available writecache (HAWC) . . . . . . . . . . . 721Applications that can benefit from HAWC. . . . 722Restrictions and tuning recommendations forHAWC . . . . . . . . . . . . . . . 722Using HAWC . . . . . . . . . . . . . 723

Chapter 45. Local read-only cache 725

Chapter 46. Miscellaneous advancedadministration topics . . . . . . . . 727Changing IP addresses and host names. . . . . 727Enabling a cluster for IPv6 . . . . . . . . . 728Using multiple token servers . . . . . . . . 729Exporting file system definitions between clusters 729IBM Spectrum Scale port usage . . . . . . . 730Securing the IBM Spectrum Scale system usingfirewall . . . . . . . . . . . . . . . 732

Firewall recommendations for the IBMSpectrum Scale installation . . . . . . . . 733Firewall recommendations for internalcommunication among nodes . . . . . . . 734Firewall recommendations for protocol access 735Firewall recommendations for IBM SpectrumScale GUI . . . . . . . . . . . . . 739Firewall recommendations for IBM SKLM . . . 740Firewall recommendations for Vormetric DSM 740Firewall recommendations for the REST API . . 741Firewall recommendations for PerformanceMonitoring tool . . . . . . . . . . . 741Firewall considerations for Active FileManagement (AFM) . . . . . . . . . . 742Firewall considerations for remote mounting offile systems . . . . . . . . . . . . . 742Firewall recommendations for using IBMSpectrum Protect with IBM Spectrum Scale . . 743Firewall considerations for using IBM SpectrumArchive with IBM Spectrum Scale . . . . . 743Firewall recommendations for file audit logging 743Firewall recommendations for call home . . . 744Examples of how to open firewall ports . . . 744

Logging file system activities . . . . . . . . 746Supported web browser versions and web browsersettings for GUI . . . . . . . . . . . . 746

Chapter 47. GUI limitations . . . . . 749

Accessibility features for IBMSpectrum Scale . . . . . . . . . . 751Accessibility features . . . . . . . . . . . 751Keyboard navigation . . . . . . . . . . . 751IBM and accessibility . . . . . . . . . . . 751

Notices . . . . . . . . . . . . . . 753Trademarks . . . . . . . . . . . . . . 754Terms and conditions for product documentation 755IBM Online Privacy Statement. . . . . . . . 755

Glossary . . . . . . . . . . . . . 757

Index . . . . . . . . . . . . . . . 763

Contents ix

|||||

|||

x IBM Spectrum Scale 5.0.2: Administration Guide

Tables

1. IBM Spectrum Scale library information units xiv2. Conventions . . . . . . . . . . . . xxv3. List of changes in documentation xxxiv4. Configuration attributes on the mmchconfig

command . . . . . . . . . . . . . 65. Attributes and default values . . . . . . 706. Supported Components . . . . . . . . 717. Configuration parameters at cache and their

default values at the cache cluster . . . . . 938. Configuration parameters at cache and their

default values at the cache cluster - Validvalues . . . . . . . . . . . . . . 96

9. Configuration parameters at cache for paralleldata transfer . . . . . . . . . . . . 97

10. Configuration parameters at cache for paralleldata transfer - valid values . . . . . . . 98

11. Configuration parameters at primary andtheir default values . . . . . . . . . 101

12. Configuration parameters at primary andtheir default values - Valid values. . . . . 102

13. Configuration parameters at cache for paralleldata transfer . . . . . . . . . . . . 103

14. Configuration parameters at cache for paralleldata transfer - valid values . . . . . . . 104

15. NFS server parameters . . . . . . . . 10816. COMPRESSION and illCompressed flags 13517. Set QoS classes to unlimited . . . . . . 13918. Allocate the available IOPS . . . . . . . 13919. Authentication requirements for each file

access protocol. . . . . . . . . . . . 21220. Object services and protocol nodes . . . . 25121. Object input behavior in unified_mode 26722. Configuration options for [swift-constraints]

in swift.conf . . . . . . . . . . . . 28323. Configurable options for [DEFAULT] in

object-server-sof.conf . . . . . . . . . 28524. Configurable options for [capabilities] in

spectrum-scale-object.conf . . . . . . . 28625. Configuration options for [DEFAULT] in

spectrum-scale-objectizer.conf . . . . . . 28626. Configuration options for

[IBMOBJECTIZER-LOGGER] inspectrum-scale-objectizer.conf . . . . . . 287

27. Configuration options for object-server.conf 28728. Configuration options for

/etc/sysconfig/memcached. . . . . . . 28729. Configuration options for proxy-server.conf 28730. mkldap command parameters . . . . . . . 31331. Removal of a file with ACL entries DELETE

and DELETE_CHILD . . . . . . . . . 32132. Mapping from SMB Security Descriptor to

NFSv4 ACL entry . . . . . . . . . . 32533. Mapping from NFSv4 ACL entry to SMB

Security Descriptor. . . . . . . . . . 325

34. ACL permissions required to work on filesand directories, while using SMB protocol(table 1 of 2) . . . . . . . . . . . . 328

35. ACL permissions required to work on filesand directories, while using SMB protocol(table 2 of 2) . . . . . . . . . . . . 328

36. ACL permissions required to work on filesand directories, while using NFS protocol(table 1 of 2) . . . . . . . . . . . . 329

37. ACL permissions required to work on filesand directories, while using NFS protocol(table 2 of 2) . . . . . . . . . . . . 330

38. Commands and reference to manage ACLtasks . . . . . . . . . . . . . . 332

39. ACL options that are available to manipulateobject read ACLs . . . . . . . . . . 337

40. Summary of commands to set upcross-cluster file system access.. . . . . . 360

41. The effects of file operations on an immutablefile or an appendOnly file . . . . . . . 422

42. IAM modes and their effects on fileoperations on immutable files . . . . . . 423

43. Example for retention period . . . . . . 43044. Example - Time stamp of snapshots that are

retained based on the retention policy . . . 43045. Valid EncParamString values . . . . . . 57146. Valid combine parameter string values 57147. Valid wrapping parameter string values 57148. Required version of IBM Spectrum Scale 57649. Remote Key Management servers . . . . . 57650. The RKM.conf file . . . . . . . . . . 57851. The client keystore directory . . . . . . 58052. Configuring a node for encryption in the

simplified setup. . . . . . . . . . . 58453. Configuring a node for encryption in the

simplified setup. . . . . . . . . . . 59254. Setup of Cluster1 and Cluster2. . . . . . 60055. Managing another key server . . . . . . 60556. Frequency of warnings . . . . . . . . 63957. Comparing default lifetimes of key server

and key client certificates . . . . . . . 64058. Security features that are used to secure

authentication server . . . . . . . . . 66159. Sample policy list . . . . . . . . . . 67360. Parameter description. . . . . . . . . 69461. Parameter description. . . . . . . . . 69462. Parameter description. . . . . . . . . 69563. Parameter description. . . . . . . . . 69564. Parameter description. . . . . . . . . 69665. IBM Spectrum Scale port usage . . . . . 73066. Firewall related information. . . . . . . 73267. Recommended port numbers that can be used

for installation . . . . . . . . . . . 73368. Recommended port numbers that can be used

for internal communication . . . . . . . 73469. Recommended port numbers for NFS access 735

© Copyright IBM Corp. 2014, 2019 xi

||

|||

||

70. Recommended port numbers for SMB access 73671. Recommended port numbers for iSCSI access 73672. Port numbers for object access . . . . . . 73773. Port numbers for object authentication 73774. Port numbers for Postgres database for object

protocol . . . . . . . . . . . . . 73875. Consolidated list of recommended ports for

different functions . . . . . . . . . . 73876. Firewall recommendations for GUI . . . . 73977. Firewall recommendations for SKLM 74078. Firewall recommendations for SKLM 741

79. Firewall recommendations for REST API 74180. Recommended port numbers that can be used

for Performance Monitoring tool . . . . . 74181. Required port number for mmbackup and

HSM connectivity to IBM Spectrum Protectserver . . . . . . . . . . . . . . 743

82. Recommended port numbers that can be usedfor file audit logging . . . . . . . . . 744

83. Recommended port numbers that can be usedfor call home. . . . . . . . . . . . 744

xii IBM Spectrum Scale 5.0.2: Administration Guide

About this information

This edition applies to IBM Spectrum Scale™ version 5.0.2 for AIX®, Linux, and Windows.

IBM Spectrum Scale is a file management infrastructure, based on IBM® General Parallel File System(GPFS™) technology, which provides unmatched performance and reliability with scalable access tocritical file data.

To find out which version of IBM Spectrum Scale is running on a particular AIX node, enter:lslpp -l gpfs\*

To find out which version of IBM Spectrum Scale is running on a particular Linux node, enter:rpm -qa | grep gpfs (for SLES and Red Hat Enterprise Linux)

dpkg -l | grep gpfs (for Ubuntu Linux)

To find out which version of IBM Spectrum Scale is running on a particular Windows node, openPrograms and Features in the control panel. The IBM Spectrum Scale installed program name includesthe version number.

Which IBM Spectrum Scale information unit provides the information you need?

The IBM Spectrum Scale library consists of the information units listed in Table 1 on page xiv.

To use these information units effectively, you must be familiar with IBM Spectrum Scale and the AIX,Linux, or Windows operating system, or all of them, depending on which operating systems are in use atyour installation. Where necessary, these information units provide some background information relatingto AIX, Linux, or Windows. However, more commonly they refer to the appropriate operating systemdocumentation.

Note: Throughout this documentation, the term “Linux” refers to all supported distributions of Linux,unless otherwise specified.

© Copyright IBM Corp. 2014, 2019 xiii

Table 1. IBM Spectrum Scale library information units

Information unit Type of information Intended users

IBM Spectrum Scale: Concepts,Planning, and Installation Guide

This guide provides the followinginformation:

Product overview

v Overview of IBM Spectrum Scalev GPFS architecturev Protocols support overview:

Integration of protocol accessmethods with GPFS

v Active File Managementv AFM-based Asynchronous Disaster

Recovery (AFM DR)

v Data protection and disaster recoveryin IBM Spectrum Scale

v Introduction to IBM Spectrum ScaleGUI

v IBM Spectrum Scale management APIv Introduction to Cloud servicesv Introduction to file audit loggingv Introduction to watch folderv IBM Spectrum Scale in an OpenStack

cloud deployment

v IBM Spectrum Scale product editionsv IBM Spectrum Scale license

designation

v Capacity based licensingv IBM Spectrum Storage™ Suite

Planning

v Planning for GPFSv Planning for protocolsv Planning for Cloud servicesv Planning for AFMv Planning for AFM DRv Firewall recommendationsv Considerations for GPFS applications

System administrators, analysts,installers, planners, and programmers ofIBM Spectrum Scale clusters who arevery experienced with the operatingsystems on which each IBM SpectrumScale cluster is based

xiv IBM Spectrum Scale 5.0.2: Administration Guide

|

|

|

Table 1. IBM Spectrum Scale library information units (continued)



Installing

v Steps for establishing and startingyour IBM Spectrum Scale cluster

v Installing IBM Spectrum Scale onLinux nodes and deploying protocols

v Installing IBM Spectrum Scale on AIXnodes

v Installing IBM Spectrum Scale onWindows nodes

v Installing Cloud services on IBMSpectrum Scale nodes

v Installing and configuring IBMSpectrum Scale management API

v Installing Active File Managementv Installing and upgrading AFM-based

Disaster Recovery

v Installing call homev Installing file audit loggingv Installing watch folderv Steps to permanently uninstall GPFS

and/or Protocols


About this information xv

|




Upgrading

v IBM Spectrum Scale supportedupgrade paths

v Upgrading to IBM Spectrum Scale5.0.x from IBM Spectrum Scale 4.2.y

v Upgrading to IBM Spectrum Scale4.2.y from IBM Spectrum Scale 4.1.x

v Upgrading to IBM Spectrum Scale4.1.1.x from GPFS V4.1.0.x

v Upgrading from GPFS 3.5v Online upgrade support for protocols

and performance monitoring

v Upgrading AFM and AFM DRv Upgrading object packagesv Upgrading SMB packagesv Upgrading NFS packagesv Upgrading call homev Manually upgrading the performance

monitoring tool

v Manually upgrading pmswiftv Manually upgrading the IBM

Spectrum Scale management GUI

v Upgrading Cloud servicesv Upgrading to IBM Cloud Object

Storage software level 3.7.2 and above

v Upgrading file audit loggingauthentication

v Upgrading watch folder callbacksv Upgrading IBM Spectrum Scale

components with the installationtoolkit

v Changing the IBM Spectrum Scaleproduct edition

v Completing the upgrade to a newlevel of IBM Spectrum Scale

v Reverting to the previous level ofIBM Spectrum Scale

v Coexistence considerationsv Compatibility considerationsv Considerations for IBM Spectrum

Protect™ for Space Management

v GUI user role considerationsv Applying maintenance to your GPFS

system

v Guidance for upgrading the operatingsystem on IBM Spectrum Scale nodes


xvi IBM Spectrum Scale 5.0.2: Administration Guide

|

||

||



IBM Spectrum Scale:Administration Guide


Configuring

v Configuring the GPFS clusterv Configuring the CES and protocol

configuration

v Configuring and tuning your systemfor GPFS

v Parameters for performance tuningand optimization

v Ensuring high availability of the GUIservice

v Configuring and tuning your systemfor Cloud services

v Configuring file audit loggingv Configuring Active File Managementv Configuring AFM-based DRv Tuning for Kernel NFS backend on

AFM and AFM DR

Administering

v Performing GPFS administration tasksv Verifying network operation with the

mmnetverify command

v Managing file systemsv File system format changes between

versions of IBM Spectrum Scale

v Managing disksv Managing protocol servicesv Managing protocol user

authentication

v Managing protocol data exportsv Managing object storagev Managing GPFS quotasv Managing GUI usersv Managing GPFS access control listsv Considerations for GPFS applicationsv Accessing a remote GPFS file system

System administrators or programmersof IBM Spectrum Scale systems

About this information xvii



IBM Spectrum Scale:Administration Guide

v Information lifecycle management forIBM Spectrum Scale

v Creating and maintaining snapshotsof file systems

v Creating and managing file clonesv Scale Out Backup and Restore

(SOBAR)

v Data Mirroring and Replicationv Implementing a clustered NFS

environment on Linux

v Implementing Cluster Export Servicesv Identity management on Windowsv Protocols cluster disaster recoveryv File Placement Optimizerv Encryptionv Managing certificates to secure

communications between GUI webserver and web browsers

v Securing protocol datav Cloud services: Transparent cloud

tiering and Cloud data sharing

v Managing file audit loggingv Performing a watch with watch folderv Administering AFMv Administering AFM DRv Highly-available write cache (HAWC)v Local read-only cachev Miscellaneous advanced

administration

v GUI limitations

System administrators or programmersof IBM Spectrum Scale systems

xviii IBM Spectrum Scale 5.0.2: Administration Guide

|

|

|



IBM Spectrum Scale: ProblemDetermination Guide


Monitoring

v Performance monitoringv Monitoring system health through the

IBM Spectrum Scale GUI

v Monitoring system health by usingthe mmhealth command

v Monitoring events through callbacksv Monitoring capacity through GUIv Monitoring AFM and AFM DRv GPFS SNMP supportv Monitoring the IBM Spectrum Scale

system by using call home

v Monitoring remote cluster throughGUI

v Monitoring file audit logging

Troubleshooting

v Best practices for troubleshootingv Understanding the system limitationsv Collecting details of the issuesv Managing deadlocksv Installation and configuration issuesv Upgrade issuesv Network issuesv File system issuesv Disk issuesv Security issuesv Protocol issuesv Disaster recovery issuesv Performance issuesv GUI issuesv AFM issuesv AFM DR issuesv Transparent cloud tiering issuesv File audit logging issuesv Troubleshooting watch folderv Maintenance proceduresv Recovery proceduresv Support for troubleshootingv References

System administrators of GPFS systemswho are experienced with thesubsystems used to manage disks andwho are familiar with the conceptspresented in the IBM Spectrum Scale:Concepts, Planning, and Installation Guide

About this information xix

|



IBM Spectrum Scale: Commandand Programming Reference


Command reference

v gpfs.snap commandv mmaddcallback commandv mmadddisk commandv mmaddnode commandv mmadquery commandv mmafmconfig commandv mmafmctl commandv mmafmlocal commandv mmapplypolicy commandv mmaudit commandv mmauth commandv mmbackup commandv mmbackupconfig commandv mmblock commandv mmbuildgpl commandv mmcachectl commandv mmcallhome commandv mmces commandv mmcesdr commandv mmchattr commandv mmchcluster commandv mmchconfig commandv mmchdisk commandv mmcheckquota commandv mmchfileset commandv mmchfs commandv mmchlicense commandv mmchmgr commandv mmchnode commandv mmchnodeclass commandv mmchnsd commandv mmchpolicy commandv mmchpool commandv mmchqos commandv mmclidecode commandv mmclone commandv mmcloudgateway commandv mmcrcluster commandv mmcrfileset commandv mmcrfs commandv mmcrnodeclass commandv mmcrnsd commandv mmcrsnapshot command

v System administrators of IBMSpectrum Scale systems

v Application programmers who areexperienced with IBM Spectrum Scalesystems and familiar with theterminology and concepts in theXDSM standard

xx IBM Spectrum Scale 5.0.2: Administration Guide




v mmdefedquota commandv mmdefquotaoff commandv mmdefquotaon commandv mmdefragfs commandv mmdelacl commandv mmdelcallback commandv mmdeldisk commandv mmdelfileset commandv mmdelfs commandv mmdelnode commandv mmdelnodeclass commandv mmdelnsd commandv mmdelsnapshot commandv mmdf commandv mmdiag commandv mmdsh commandv mmeditacl commandv mmedquota commandv mmexportfs commandv mmfsck commandv mmfsctl commandv mmgetacl commandv mmgetstate commandv mmhadoopctl commandv mmhealth commandv mmimgbackup commandv mmimgrestore commandv mmimportfs commandv mmkeyserv commandv mmlinkfileset commandv mmlsattr commandv mmlscallback commandv mmlscluster commandv mmlsconfig commandv mmlsdisk commandv mmlsfileset commandv mmlsfs commandv mmlslicense commandv mmlsmgr command



About this information xxi




v mmlsmount commandv mmlsnodeclass commandv mmlsnsd commandv mmlspolicy commandv mmlspool commandv mmlsqos commandv mmlsquota commandv mmlssnapshot commandv mmmigratefs commandv mmmount commandv mmmsgqueue commandv mmnetverify commandv mmnfs commandv mmnsddiscover commandv mmobj commandv mmperfmon commandv mmpmon commandv mmprotocoltrace commandv mmpsnap commandv mmputacl commandv mmquotaoff commandv mmquotaon commandv mmremotecluster commandv mmremotefs commandv mmrepquota commandv mmrestoreconfig commandv mmrestorefs commandv mmrestripefile commandv mmrestripefs commandv mmrpldisk commandv mmsdrrestore commandv mmsetquota commandv mmshutdown commandv mmsmb commandv mmsnapdir commandv mmstartup commandv mmtracectl commandv mmumount commandv mmunlinkfileset commandv mmuserauth commandv mmwatch commandv mmwinservctl commandv spectrumscale command



xxii IBM Spectrum Scale 5.0.2: Administration Guide

|




Programming reference

v IBM Spectrum Scale DataManagement API for GPFSinformation

v GPFS programming interfacesv GPFS user exitsv IBM Spectrum Scale management API

commands

v Watch folder API



IBM Spectrum Scale: Big Data andAnalytics Guide


Hadoop Scale Storage Architecture

v Elastic Storage Server (ESS)v Share Storage (SAN-based storage)v File Placement Optimizer (FPO)v Deployment modelv Additional supported features about

storage

IBM Spectrum Scale support forHadoop

v HDFS transparencyv Supported IBM Spectrum Scale

storage modes

v Hadoop cluster planningv Installation and configuration of

HDFS transparency

v Application interaction with HDFStransparency

v Upgrading the HDFS Transparencycluster

v Rolling upgrade for HDFSTransparency

v Securityv Advanced featuresv Hadoop distribution supportv Limitations and differences from

native HDFS

v Problem determination

IBM Spectrum Scale Hadoopperformance tuning guide

v Overviewv Performance overviewv Hadoop Performance Planning over

IBM Spectrum Scale

v Performance guide



About this information xxiii

|



IBM Spectrum Scale: Big Data andAnalytics Guide

Hortonworks Data Platform 3.X

v Planningv Installationv Upgrading and uninstallationv Configurationv Administrationv Limitationsv Problem determination

Open Source Apache Hadoop

v Apache Hadoop 3.0.x Support

BigInsights 4.2.5 and Hortonworks DataPlatform 2.6

v Planningv Installationv Upgrading software stackv Configurationv Administrationv Troubleshootingv Limitationsv FAQ



IBM Spectrum Scale on AWSGuide

IBM Spectrum Scale on AWS

v Introduction to IBM Spectrum Scaleon AWS

v Setting up the IBM Spectrum Scaleenvironment in the AWS Cloud

v Deploying IBM Spectrum Scale onAWS

v Cleaning up the cluster and the stackv Data security and AWS Identity and

Access Management

v Cluster lifecycle management anddebug data collection

v Upgrading IBM Spectrum Scalev Troubleshootingv Frequently Asked Questions



Prerequisite and related informationFor updates to this information, see IBM Spectrum Scale in IBM Knowledge Center (www.ibm.com/support/knowledgecenter/STXKQY/ibmspectrumscale_welcome.html).

For the latest support information, see the IBM Spectrum Scale FAQ in IBM Knowledge Center(www.ibm.com/support/knowledgecenter/STXKQY/gpfsclustersfaq.html).

xxiv IBM Spectrum Scale 5.0.2: Administration Guide

http://www.ibm.com/support/knowledgecenter/STXKQY/ibmspectrumscale_welcome.htmlhttp://www.ibm.com/support/knowledgecenter/STXKQY/ibmspectrumscale_welcome.htmlhttp://www.ibm.com/support/knowledgecenter/STXKQY/gpfsclustersfaq.htmlhttp://www.ibm.com/support/knowledgecenter/STXKQY/gpfsclustersfaq.html

Conventions used in this informationTable 2 describes the typographic conventions used in this information. UNIX file name conventions areused throughout this information.

Note: Users of IBM Spectrum Scale for Windows must be aware that on Windows, UNIX-style filenames need to be converted appropriately. For example, the GPFS cluster configuration data is stored inthe /var/mmfs/gen/mmsdrfs file. On Windows, the UNIX namespace starts under the %SystemDrive%\cygwin64 directory, so the GPFS cluster configuration data is stored in the C:\cygwin64\var\mmfs\gen\mmsdrfs file.

Table 2. Conventions

Convention Usage

bold Bold words or characters represent system elements that you must use literally, such ascommands, flags, values, and selected menu options.

Depending on the context, bold typeface sometimes represents path names, directories, or filenames.

bold underlined bold underlined keywords are defaults. These take effect if you do not specify a differentkeyword.

constant width Examples and information that the system displays appear in constant-width typeface.

Depending on the context, constant-width typeface sometimes represents path names,directories, or file names.

italic Italic words or characters represent variable values that you must supply.

Italics are also used for information unit titles, for the first use of a glossary term, and forgeneral emphasis in text.

Angle brackets (less-than and greater-than) enclose the name of a key on the keyboard. Forexample, refers to the key on your terminal or workstation that is labeled with theword Enter.

\ In command examples, a backslash indicates that the command or coding example continueson the next line. For example:

mkcondition -r IBM.FileSystem -e "PercentTotUsed > 90" \-E "PercentTotUsed < 85" -m p "FileSystem space used"

{item} Braces enclose a list from which you must choose an item in format and syntax descriptions.

[item] Brackets enclose optional items in format and syntax descriptions.

The notation indicates a control character sequence. For example, meansthat you hold down the control key while pressing .

item... Ellipses indicate that you can repeat the preceding item one or more times.

| In synopsis statements, vertical lines separate a list of choices. In other words, a vertical linemeans Or.

In the left margin of the document, vertical lines indicate technical changes to theinformation.

Note: CLI options that accept a list of option values delimit with a comma and no space between values.As an example, to display the state on three nodes use mmgetstate -N NodeA,NodeB,NodeC. Exceptions tothis syntax are listed specifically within the command.

About this information xxv

How to send your commentsYour feedback is important in helping us to produce accurate, high-quality information. If you have anycomments about this information or any other IBM Spectrum Scale documentation, send your commentsto the following e-mail address:

[email protected]

Include the publication title and order number, and, if applicable, the specific location of the informationabout which you have comments (for example, a page number or a table number).

To contact the IBM Spectrum Scale development organization, send your comments to the followinge-mail address:

[email protected]

xxvi IBM Spectrum Scale 5.0.2: Administration Guide

Summary of changes

This topic summarizes changes to the IBM Spectrum Scale licensed program and the IBM Spectrum Scalelibrary. Within each information unit in the library, a vertical line (|) to the left of text and illustrationsindicates technical changes or additions that are made to the previous edition of the information.

Summary of changesfor IBM Spectrum Scale version 5.0.2as updated, February 2019

This release of the IBM Spectrum Scale licensed program and the IBM Spectrum Scale library includes thefollowing improvements. All improvements are available after an upgrade, unless otherwise specified.

AFM and AFM DR-related changes

v Enabled user-defined gateway node assignment to AFM and AFM DR filesets by modifyingafmHashVersion value to 5 and adding the gateway node as afmGateway. For more information,see the topics mmchfileset command and mmcrfileset command in the IBM Spectrum Scale: Commandand Programming Reference.

v Added new options to mmafmctl prefetch. For more information, see the topic mmafmctlcommand in the IBM Spectrum Scale: Command and Programming Reference.

v Read-Only NFS export is supported for AFM RO mode filesets. For more information, see thetopic Introduction to Active File Management (AFM) in the IBM Spectrum Scale: Concepts, Planning,and Installation Guide.

Authentication-related changesThe --password, --ks-admin-pwd, and --ks-swift-pwd parameters are removed from mmuserauthCLI command. For more information, see the topic mmuserauth command in the IBM SpectrumScale: Command and Programming Reference.

IBM Spectrum Scale on AWSIBM Spectrum Scale can now be deployed on AWS. For more information, see IBM SpectrumScale on AWS.

Big data and analytics changesFor information on changes in IBM Spectrum Scale Big Data and Analytics support, see Big Dataand Analytics - summary of changes.

Cloud services changesCloud services has the following updates:v Support for RHEL 7.4 and 7.5 on both Power® and x86 machines.v Support for Openstack Swift 2.13, IBM Cloud Object Storage 3.13.4.40, and Swift3 2.13

Data Access EditionIBM Spectrum Scale Data Access Edition is a new edition based on capacity-based licensing thatprovides identical functionality as IBM Spectrum Scale Standard Edition. For more information,see IBM Spectrum Scale product editions in IBM Spectrum Scale: Concepts, Planning, and InstallationGuide.

File audit logging updatesFile audit logging has the following updates:v Listing or viewing the contents of directories within file audit logging enabled file systems will

produce OPEN and CLOSE events in the audit logs. For more information, see JSON reportingissues in file audit logging in the IBM Spectrum Scale: Problem Determination Guide.

v Added option to enable and disable file audit logging from the IBM Spectrum Scalemanagement GUI. You can enable file audit logging at the file system level while creating or

© Copyright IBM Corp. 2014, 2019 xxvii

|||

||

|

||||

||

|||

||||

|||

|||

||

|

|

|||||

||

|||

||

https://www.ibm.com/support/knowledgecenter/STXKQY_AWS_SHR/com.ibm.spectrum.scale.aws.v5r01.doc/bl1cld_aws_kclanding.htmhttps://www.ibm.com/support/knowledgecenter/STXKQY_AWS_SHR/com.ibm.spectrum.scale.aws.v5r01.doc/bl1cld_aws_kclanding.htmhttps://www.ibm.com/support/knowledgecenter/STXKQY_BDA_SHR/bl1bda_soc.htmhttps://www.ibm.com/support/knowledgecenter/STXKQY_BDA_SHR/bl1bda_soc.htm

modifying a file system from the Files > File Systems page. For more information, see Enablingand disabling file audit logging using the GUI in the IBM Spectrum Scale: Administration Guide.

v Support for Linux on Z (RHEL 7.x, Ubuntu 16.04 and Ubuntu 18.04 on s390x).v Multi-cluster/remote mount is supported. For more information, see Remotely mounted file

systems in file audit logging in the IBM Spectrum Scale: Concepts, Planning, and Installation Guide.v Improved monitoring of Kafka producers.v Subset of events is supported.v Ability to add broker nodes to the message queue dynamically. For more information, see

Designating additional broker nodes for increased performance in the IBM Spectrum Scale:Administration Guide.

v Call home is supported.

File system core improvements

Certificate expiration warnings are logged for key clients and RKM serversThe GPFS daemon writes warning messages into the error log as digital certificates forkey clients and Remote Key Management (RKM) servers approach their expiration dates.The frequency of warnings increases as the expiration date nears. Only certificates thatare regularly being used for authentication between a key client and an RKM server aretracked. For more information, see the topic Certificate expiration warnings in the IBMSpectrum Scale: Administration Guide.

Combined gpfs.base and gpfs.ext into a single package on LinuxOn Linux, the gpfs.base and gpfs.ext packages are combined into a single package. As aresult, the gpfs.ext package is no longer available on Linux.

File system maintenance mode provides a safe access window for file system maintenanceFile system maintenance mode provides a way to enable a file system maintenancewindow. Use file system maintenance mode whenever you perform maintenance oneither NSD disks or NSD servers that might result in NSDs becoming unavailable.

You must use the --maintenance-mode parameter with the mmchfs and mmlsfs commandsto use file system maintenance mode. For more information, see the topic File systemmaintenance mode in the IBM Spectrum Scale: Administration Guide.

The GPFS portability layer (GPL) can be rebuilt automaticallyFor more information, see the entry Installation improvements later in this topic.

The maxActiveIallocSegs attribute improves the performance of deletes and unlinksThe maxActiveIallocSegs attribute of the mmchconfig command can significantly improveperformance in the following scenario:1. A single node has created a large number of files in multiple directories.2. Processes and threads on multiple nodes are now concurrently attempting to delete or

unlink files in those directories

For more information, see the topic mmchconfig command in the IBM Spectrum Scale:Command and Programming Reference.

The mmnetverify command checks the connectivity of remote clustersThe mmnetverify command checks remote clusters for host-name resolution, networkconnectivity by ping, and GPFS daemon connectivity. It checks known remote clustersfrom the mmsdrfs file and can also check remote clusters that are specified on thecommand line. For more information, see the topic mmnetverify command in the IBMSpectrum Scale: Command and Programming Reference.

The stat cache is effective on Linux in all configurationsThe stat cache is effective on the Linux operating system in all supported hardwareconfigurations, not just when a Local Read-Only Cache (LROC) device is attached. A statcache improves the performance of system calls that return file attributes, such as stat().

xxviii IBM Spectrum Scale 5.0.2: Administration Guide

||

|

||

|

|

|||

|

|

|||||||

|||

||||

|||

||

|||

|

||

||

||||||

||||

The default values of maxStatCache and the factors for estimating a nondefault value formaxStatCache are now the same for Linux as they are for other supported operatingsystems. For more information, see the topic mmchconfig command in the IBM SpectrumScale: Command and Programming Reference.

Two features cope with long I/O waits on directly attached disksThis feature applies only to disks that a node is directly attached to.v The diskIOHang callback event allows you to add notification and data collection

scripts to analyze the cause of a local I/O request that has been pending in the nodekernel for more than 5 minutes. For more information, see the topic mmaddcallbackcommand in the IBM Spectrum Scale: Command and Programming Reference.

v The panicOnIOHang attribute controls whether the GPFS daemon panics the nodekernel when a local I/O request has been pending in the kernel for more than fiveminutes. For more information, see the topic mmchconfig command in the IBM SpectrumScale: Command and Programming Reference.

IBM Spectrum Scale management API changesAdded the following API commands:v PUT /filesystems/{filesystemName}/auditv GET: /smb/shares/{shareName}/aclv GET: /smb/shares/{shareName}/acl/{name}v DELETE: /smb/shares/{shareName}/aclv DELETE: /smb/shares/{shareName}/acl/{name}v PUT: /smb/shares/{shareName}/acl/{name}

For more information on the API commands, see IBM Spectrum Scale management API commands inIBM Spectrum Scale: Command and Programming Reference. You can also access the documentationcorresponding to each API command from the GUI itself. The API documentation is available inthe GUI at: https://:/ibm/api/explorer/. For example:https://scalegui.ibm.com:443/ibm/api/explorer.

IBM Spectrum Scale GUI changes

The following changes are made to the GUI:v Added options to create and manage node classes. The two types of node classes that can be

defined in the IBM Spectrum Scale system are system node classes and user-defined nodeclasses. You can create user-defined node classes by using the Nodes > Node Classes > CreateNode Class option. The system node classes are pre-defined and you cannot create or modifythem using the management GUI.

v Added option to enable and disable file audit logging. File audit logging captures fileoperations on a file system and logs them to a retention enabled fileset. You can enable fileaudit logging at the file system level while creating or modifying a file system from the Files >File Systems page.

v Added option to configure automatic assignment of certain sensors to a single node, in theServices > Performance Monitoring > Sensors page. A single node is automatically selected bythe system to run the GPFSFilesetQuota, GPFSFileset, GPFSPool, and GPFSDiskCap sensors.

v Improved remote cluster monitoring options capabilities.v Introduced an integrated view of GUI services and GUI user management in the Services >

GUI page.Removed the GUI Users and GUI Access pages from the Access menu and integrated the GUIuser management features in the Services > GUI page.

v The GUI automatically logs out the users if the administrator changes the user role or expirethe password of the currently logged in user.

Summary of changes xxix

||||

||

||||

||||

||

|

|

|

|

|

|

|||||

|

|

|||||

||||

|||

|

||

||

||

v Improved filtering and listing of events. Added graphical view of events reported against eachcomponent in the Monitoring > Events page. Clicking on the graph displays only the relevantevents in the grid view. Removed the Unread Events filter option and introduced Current Stateand Notices filter options.

v Improved health status monitoring of CES IPs. Renamed Services > CES Network to Services> CES Network and CES IPs. The Addresses section shows the health status of the CES IPcomponent. It also shows the preferred nodes and non-hostable nodes if Node Affinity isselected as the IP address distribution method.

v Introduced a What's new window to highlight the changes made to the IBM Spectrum ScaleGUI in a release. When the user launches the GUI for the first time after installing orupgrading to the latest version, the GUI users can see the list of GUI changes made in thatrelease. The user can also launch the What's New window by using the What's New? optionthat is available in the Help menu of the management GUI. The Help menu is available at theupper right corner of the GUI.

Installation improvements

The GPFS portability layer (GPL) can be rebuilt automatically.You can now configure a cluster to automatically rebuild the GPL whenever a new levelof the Linux kernel is installed or whenever a new level of IBM Spectrum Scale isinstalled. For more information, see the description of the autoBuildGPL attribute in thetopic mmchconfig command in the IBM Spectrum Scale: Command and Programming Reference.

Installation toolkit changes

v Support for IBM Z (RHEL 7.x, SLES12.x, Ubuntu 16.04 and Ubuntu 18.04 on s390x)v Support for Red Hat Enterprise Linux 7.6 on x86_64, PPC64, PPC64LE, and s390x.v Support for offline upgrade of nodes while they are down or unhealthy. For more information,

see Performing offline upgrade or excluding nodes from upgrade using installation toolkit in IBMSpectrum Scale: Concepts, Planning, and Installation Guide.

v Support for excluding nodes from an upgrade run. For more information, see Performing offlineupgrade or excluding nodes from upgrade using installation toolkit in IBM Spectrum Scale: Concepts,Planning, and Installation Guide.

v Support for rerunning an upgrade procedure after a failure. For more information, see Upgradererun after an upgrade failure in IBM Spectrum Scale: Concepts, Planning, and Installation Guide.

v Support for watch folder. For more information, see Enabling and configuring watch folder usinginstallation toolkit in IBM Spectrum Scale: Concepts, Planning, and Installation Guide.

v Enhancements in CES shared root creation and detection in config populatev Upgraded bundled Chef package. For more information, see Preparing to use installation toolkit

in IBM Spectrum Scale: Concepts, Planning, and Installation Guide.

mmces command enhancements

New --extended-list and --full-list options that show additional data for node affinity. Formore information, see the mmces command section in the IBM Spectrum Scale: Command andProgramming Reference.

mmperfmon command enhancements

The query --list=expiredKeys option that displays the group keys for the entities that have notreturned any metrics values within the default retention period of 14 days. The delete {--expiredKeys | --key Key[,Key...] } allows users to review and delete historical perfmon keysfor renamed and deleted components in the cluster. For more information, see the mmperfmoncommand section in the IBM Spectrum Scale: Command and Programming Reference.

mmcallhome command enhancements

xxx IBM Spectrum Scale 5.0.2: Administration Guide

||||

||||

||||||

|

|||||

|

|

|

|||

|||

||

||

|

||

|

|||

|

|||||

||

New option that executes one-time gather or send tasks for a Salesforce case descriptor. For moreinformation, see the mmcallhome command section in the IBM Spectrum Scale: Command andProgramming Reference.

mmhealth command enhancements

New --show-state-changes option that displays the additional information about the state of anode has been added to the mmhealth command. For more information, see the mmhealth commandsection in the IBM Spectrum Scale: Command and Programming Reference.

NFS changesCES packages for NFS-Ganesha name changed to avoid conflict with distribution NFS-Ganeshapackages.

Object changesTwo parameters were added to the mmobj command. The --pwd-file parameter specifies a filethat contains administrative user passwords for Object access protocol authenticationconfiguration. The -Y parameter displays headers and output in a machine-readable,colon-delimited format. For more information, see the topic mmobj command in the IBM SpectrumScale: Command and Programming Reference.

IBM Spectrum Scale SMB changesThe following enhancements are available:v Configurations

– IBM Spectrum Scale does not change back the supported SMB protocol versions when theyare changed by a user.- This does not change the supported protocol levels; it just prevents that changes to

unsupported versions are automatically reverted.- No longer force SMB2 for 'server min protocol' in registry template.- No longer force SMB3 for 'server max protocol' in registry template.

v Security– Local user enumeration without credentials has been disabled by setting 'restrict anonymous

= 2' by default. This can be changed back if needed using mmsmb config change --option"restrict anonymous"=0 "

v Stability enhancements– The load on the idmap cache has been reduced by

- Removing calls to gencache_stabilize() in net utility and on smbd server exit- adding an additional cache layer in memory so that the performance of workloads

involving many idmap lookups (with hide unreadable, for example) should improve andtimeouts during idmap cache access are avoided.

– More graceful behavior of CTDB in out-of-memory situations (avoid crash, going unhealthy,log memory usage data).

– Speed up wbinfo -p (the command for local winbind monitoring) to minimize the likelihoodof winbindd monitor timeouts and resulting fail-overs.

v Ubuntu enhancements– As a consequence of the system library dependencies, GPFS SMB on Ubuntu now uses

heimdal libraries for Kerberos to avoid linking against two Kerberos libraries.– Added support for OpenSSL 1.1.0 to allow GPFS SMB build on Ubuntu 18.04.

v Usability– Removed the wrong "malloc fail" error messages and fixed error messages during charset

conversion.– Adjusted debug level when get_winattrs returns EBADF to remove warning Getting

winattrs failed for ..: Bad file descriptor

Summary of changes xxxi

|||

|

|||

|||

||||||

||

|

||

||

|

|

|

|||

|

|

|

|||

||

||

|

||

|

|

||

||

v GPFS SMB version enhanced from 4.6.14_gpfs_36 to gpfs_50-1 4.6.15_gpfs_49-1v Miscellaneous

– Cache dfree information based on query path.– Fix Windows Quota report issue: Fix keep_old_entries logic for older kerberos libraries.– Update SELinux policy module for postfix to avoid SELinux warnings /usr/libexec/

postfix/local from getattr.– Add TasksMax option to smbd systemd service file for SLES12 and Ubuntu to avoid the OS

to prevent forks on new SMB connections as the default is too small.– Restored pre-4.6.9 behavior of GPFS SMB to implicitly set SMB_ACE4_SYNCHRONIZE on

NFSv4 ACLs - more strict ACLs checks have broken downwards compatibility– Fix reporting wrong session times for stale connections in the MMC.

Single IBM Spectrum Scale package for all Linux distributions (per architecture)There is no longer the need to differentiate between protocols and non-protocols packages. Thereis now only one package for all Linux distributions.

System Health changes

v Users can now create, raise, and find custom events. For more information, see the Creating,raising, and finding custom defined events section in the IBM Spectrum Scale: Problem DeterminationGuide.

v Users can now raise events for file audit logging and watch folder. For more information, seethe File audit logging events and Watch folder events sections in the IBM Spectrum Scale: ProblemDetermination Guide.

Upgrades to call home configuration

Updated the list of collected data that is called home.

Watch folderWatch folder is a flexible API that allows programmatic actions to be taken based on file systemevents. For more information, see Introduction to watch folder in the IBM Spectrum Scale: Concepts,Planning, and Installation Guide. It has the following features:v Watch folder can be run against folders, filesets, and inode spaces.v Watch folder is modeled after Linux inotify, but works with clustered file systems and supports

recursive watches for filesets and inode spaces.v Watch folder has two primary components:

– The GPFS programming interfaces, which are included within . For moreinformation, see Watch folder API in the IBM Spectrum Scale: Command and ProgrammingReference.

– The mmwatch command, which provides information for all of the watches running within acluster. For more information, see mmwatch command in the IBM Spectrum Scale: Command andProgramming Reference.

v A watch folder application uses the API to run on a node within an IBM Spectrum Scalecluster.– It utilizes the message queue to receive events from multiple nodes and consume from the

node that is running the application.– Lightweight events come in from all eligible nodes within a cluster and from accessing

clusters.v Watch folder is integrated into call home, IBM Spectrum Scale snap log collection, and IBM

Spectrum Scale trace.

xxxii IBM Spectrum Scale 5.0.2: Administration Guide

|

|

|

|

||

||

||

|

|||

|

|||

|||

|

|

||||

|

||

|

|||

|||

||

||

||

||

Windows 10 supportIBM Spectrum Scale now supports Windows 10 (Pro and Enterprise editions), in bothheterogeneous and homogeneous clusters. Secure Boot must be disabled on Windows 10 nodesfor IBM Spectrum Scale to install and function.

Documented commands, structures, and subroutinesThe following section lists the modifications to the documented commands, structures, andsubroutines:

New commandsThe following command is new in this release:v mmwatch

New structuresThere are no new structures.

New subroutinesThe following subroutines are new:v gpfs_add_watchv gpfs_add_fset_watchv gpfs_add_inodespace_watchv gpfs_close_watchv gpfs_init_watchv gpfs_read_watchv gpfs_rm_watchv gpfs_watch_strerror

Changed commandsThe following commands were changed:v mmaddcallbackv mmafmctlv mmauditv mmbuildgplv mmcallhomev mmcesv mmchconfigv mmhealthv mmmsgqueuev mmnfsv mmobjv mmperfmonv mmrestripefilev mmuserauthv spectrumscale

Changed structuresThere are no changed structures.

Changed subroutinesThere are no changed subroutines.

Deleted commandsThere are no deleted commands.

Summary of changes xxxiii

||||

|||

||

|

||

||

|

|

|

|

|

|

|

|

||

|

|

|

|

|

|

|

|

|

|

|

|

|

|

|

||

||

||

Deleted structuresThere are no deleted structures.

Deleted subroutinesThere are no deleted subroutines.

MessagesThe following are the new, changed, and deleted messages:

New messages6027-1758, 6027-1759, 6027-1760, 6027-1828, 6027-2409, 6027-2410, 6027-2411, 6027-3731,6027-3732, 6027-3733, 6027-3734, 6027-3934, 6027-3935, 6027-3936, 6027-3937, and 6027-3409

Changed messages6027-1265, 6027-1303, 6027-1307, 6027-1309, and 6027-2798

Deleted messagesNone.

Changes in documentation

List of documentation changes in product guides and respective Knowledge Center sections

The following is a list of documentation changes including changes in topic titles,changes in placement of topics, and deleted topics:

Table 3. List of changes in documentation

GuideKnowledge centersection List of changes

Concepts, Planning,and Installation Guide

Product overview v Changed the titles of the following topics:– Primary gateway and afmHashVersion in the IBM Spectrum Scale:

Concepts, Planning, and Installation Guide.

Planning Moved the following topics from the Installing section:

v Requirements for UID and GID on the cache and home clustersv Recommended worker1threads on cache clusterv Inode limits to set at cache and homev Requirements for UID/GID on cache and home clustersv Recommended worker1threads on primary clusterv NFS setup on the secondary cluster

Upgrading v Changed the titles of the following topics:– Manually upgrading file audit logging authentication in IBM

Spectrum Scale: Concepts, Planning, and Installation Guide.

– Changing Express Edition to Standard Edition in IBM SpectrumScale: Concepts, Planning, and Installation Guide.

Administration Guide Configuring Moved the following topics from the Installing section of the IBMSpectrum Scale: Concepts, Planning, and Installation Guide.

v Configuration changes in an existing AFM relationshipv Changing configuration in an existing AFM DR relationship

Administering Moved the following topics from the Installing section of the IBMSpectrum Scale: Concepts, Planning, and Installation Guide.

v Creating an AFM relationship by using the NFS protocolv Creating an AFM relationship by using GPFS protocolv Creating an AFM-based DR relationship using NFS protocolv Converting GPFS filesets to AFM DRv Converting AFM relationship to AFM DR

xxxiv IBM Spectrum Scale 5.0.2: Administration Guide

||

||

||

|||

||

||

|

|

||

||

||||

||||

||

||

|

|

|

|

|

|

||

||

||

||||

|

|

|||

|

|

|

|

||

Chapter 1. Configuring the GPFS cluster

There are several tasks involved in managing your GPFS cluster. This topic points you to the informationyou need to get started.

GPFS cluster

IBM Spectrum Scale 5.0.2: Administration Guide€¦ · IBM Spectrum Scale V ersion 5.0.2 Administra...

Documents

Transcript of IBM Spectrum Scale 5.0.2: Administration Guide€¦ · IBM Spectrum Scale V ersion 5.0.2 Administra...