UNH–IOL NVMe Testing Service · 2020-11-02 · University of New Hampshire InterOperability...

UNH–IOL NVMe Testing Service

Test Plan for NVMe Cloud SSD Conformance Version 1.0

Target Specification: OCP NVMe Cloud SSD 1.0 Technical Document

Last Updated November 2, 2020

UNH–IOL NVMe Testing Service 21 Madbury Rd Suite 100 Durham, NH 03824

Tel: +1 603–862–0090 Fax: +1 603–862–4181 Email: [email protected]

University of New Hampshire InterOperability Laboratory – NVMe Cloud SSD Conformance Test Suite

UNH–IOL NVMe Testing Services 2 NVMe Cloud SSD Conformance Test Suite v1.0 ©2020 UNH–IOL

TABLE OF CONTENTS

TABLE OF CONTENTS ..................................................................................................................... 2

MODIFICATION RECORD ................................................................................................................ 6

ACKNOWLEDGMENTS ..................................................................................................................... 7 INTRODUCTION ............................................................................................................................... 8

REFERENCES ................................................................................................................................. 10

Group 1: NVM Express Requirements ..................................................................................... 11 Test 1.1 – NVM Express Requirements ............................................................................................... 12

Case 1: NVMe-1 .................................................................................................................................. 12 Case 2: NVMe-2 .................................................................................................................................. 12 Case 3: NVMe-3 .................................................................................................................................. 12

Test 1.2 – NVMe Reset Supported ....................................................................................................... 13 Case 1: NVMeR-1 ................................................................................................................................ 13 Case 2: NVMeR-2 ................................................................................................................................ 13

Test 1.3 – NVMe Controller Configuration and Behavior ................................................................ 15 Case 1: NVMe-CFG-1 ......................................................................................................................... 15 Case 2: NVMe-CFG-2 ......................................................................................................................... 15 Case 3: NVMe-CFG-3 ......................................................................................................................... 15 Case 4: NVMe-CFG-4 ......................................................................................................................... 16 Case 5: NVMe-CFG-5 ......................................................................................................................... 16 Case 6: NVMe-CFG-6 ......................................................................................................................... 16 Case 7: NVMe-CFG-7 ......................................................................................................................... 16 Case 8: NVMe-CFG-8 ......................................................................................................................... 16

Test 1.4 – NVMe Admin Command Set .............................................................................................. 18 Case 1: NVMe-AD-1 ........................................................................................................................... 18 Case 2: NVMe-AD-2 ........................................................................................................................... 18 Case 3: NVMe-AD-3 ........................................................................................................................... 18 Case 4: NVMe-AD-4 ........................................................................................................................... 19 Case 5: NVMe-AD-5 ........................................................................................................................... 19 Case 6: NVMe-AD-6 ........................................................................................................................... 19

Test 1.5 – Namespace Management / Attachment Commands ......................................................... 20 Case 1: NSM-1 ..................................................................................................................................... 20 Case 2: NSM-2 ..................................................................................................................................... 20 Case 3: NSM-3 ..................................................................................................................................... 20

Test 1.6 – Namespace Utilization .......................................................................................................... 22 Case 1: NUSE-1 ................................................................................................................................... 22

Test 1.7 – NVMe I/O Command Set .................................................................................................... 23 Case 1: NVMe-IO-1 ............................................................................................................................. 23 Case 2: NVMe-IO-2 ............................................................................................................................. 23

Test 1.8 – Optional NVMe Feature Support ....................................................................................... 24 Case 1: NVMe-OPT-1 ......................................................................................................................... 24 Case 2: NVMe-OPT-2 ......................................................................................................................... 24

Test 1.9 – Command Timeout .............................................................................................................. 25 Case 1: CTO-1, CTO-2 ........................................................................................................................ 25 Case 2: CTO-3 ..................................................................................................................................... 25



Test 1.10 – Standard Log Page Requirements .................................................................................... 27 Case 1: STD-LOG-1 ............................................................................................................................ 27 Case 2: STD-LOG-2, STD-LOG-3, STD-LOG-4 ............................................................................... 27 Case 3: STD-LOG-5, STD-LOG-6, STD-LOG-7, STD-LOG-8 ......................................................... 27

Test 1.11 – Telemetry Logging and Interface for Failure Analysis .................................................. 29 Case 1: TEL-1,2,3 ................................................................................................................................ 29 Case 2: TEL-4 ...................................................................................................................................... 29 Case 3: TEL-5 ...................................................................................................................................... 29 Case 4: TEL-6 ...................................................................................................................................... 30

Test 1.12 – SMART Cloud Health Log (0xC0) ................................................................................... 31 Case 1: SLOG-1 ................................................................................................................................... 31 Case 2: SLOG-2 ................................................................................................................................... 31 Case 3: SLOG-3 ................................................................................................................................... 32 Case 4: SLOG-4 ................................................................................................................................... 32 Case 5: SLOG-5 ................................................................................................................................... 32 Case 6: SLOG-6 ................................................................................................................................... 33 Case 7: SLOG-7 ................................................................................................................................... 33 Case 8: SLOG-8 ................................................................................................................................... 33 Case 9: SLOG-9 ................................................................................................................................... 34 Case 10: SLOG-10, SLOG-11 ............................................................................................................. 34

Test 1.13 – SMART Cloud Attributes Log Page ................................................................................. 35 Case 1: SMART-1 ................................................................................................................................ 35 Case 2: SMART-2 ................................................................................................................................ 35 Case 3: SMART-3 ................................................................................................................................ 35 Case 4: SMART-4 ................................................................................................................................ 36 Case 5: SMART-5 ................................................................................................................................ 36 Case 6: SMART-6 ................................................................................................................................ 36 Case 7: SMART-7 ................................................................................................................................ 36 Case 8: SMART-8 ................................................................................................................................ 36 Case 9: SMART-9 ................................................................................................................................ 37 Case 10: SMART-10 ............................................................................................................................ 37 Case 11: SMART-11 ............................................................................................................................ 37 Case 12: SMART-12 ............................................................................................................................ 37 Case 13: SMART-13 ............................................................................................................................ 38 Case 14: SMART-14 ............................................................................................................................ 38 Case 15: SMART-15 ............................................................................................................................ 38 Case 16: SMART-16 ............................................................................................................................ 38 Case 17: SMART-17 ............................................................................................................................ 38 Case 18: SMART-18 ............................................................................................................................ 38 Case 19: SMART-19 ............................................................................................................................ 39 Case 20: SMART-20 ............................................................................................................................ 39 Case 21: SMART-21 ............................................................................................................................ 39 Case 22: SMART-22 ............................................................................................................................ 39 Case 23: SMART-23 ............................................................................................................................ 39 Case 24: SMART-24 ............................................................................................................................ 40 Case 25: SMART-25 ............................................................................................................................ 40 Case 26: SMART-26 ............................................................................................................................ 40 Case 27: SMART-27 ............................................................................................................................ 40 Case 28: SMART-28 ............................................................................................................................ 40

Test 1.14 – Error Recovery Log Page .................................................................................................. 42



Case 1: EREC-1 ................................................................................................................................... 42 Case 2: EREC-2 ................................................................................................................................... 42 Case 3: EREC-3 ................................................................................................................................... 42 Case 4: EREC-4 ................................................................................................................................... 42 Case 5: EREC-5 ................................................................................................................................... 42 Case 6: EREC-6 ................................................................................................................................... 43 Case 7: EREC-7 ................................................................................................................................... 43 Case 8: EREC-8 ................................................................................................................................... 43 Case 9: EREC-9 ................................................................................................................................... 43 Case 10: EREC-10 ............................................................................................................................... 43 Case 11: EREC-11 ............................................................................................................................... 43 Case 12: EREC-12 ............................................................................................................................... 44

Test 1.15 – Firmware Activation History ............................................................................................ 45 Case 1: FWHST-LOG-1, FWHT-LOG-2 ............................................................................................ 45 Case 2: FWHST-LOG-3 ...................................................................................................................... 45 Case 3: FWHST-LOG-4 ...................................................................................................................... 45 Case 4: FWHST-LOG-5 ...................................................................................................................... 46

Test 1.16 – Firmware Activation History Log Page Format ............................................................. 47 Case 1: FAHL-1 ................................................................................................................................... 47 Case 2: FAHL-2 ................................................................................................................................... 47 Case 3: FAHL-3 ................................................................................................................................... 47 Case 4: FAHL-4 ................................................................................................................................... 47 Case 5: FAHL -5 .................................................................................................................................. 48 Case 6: FAHL-6 ................................................................................................................................... 48 Case 7: FAHL-7 ................................................................................................................................... 48

Test 1.17 – Firmware Activation History Entry Format ................................................................... 49 Case 1: FAHE-1 ................................................................................................................................... 49 Case 2: FAHE-2 ................................................................................................................................... 49 Case 3: FAHE-3 ................................................................................................................................... 49 Case 4: FAHE-4 ................................................................................................................................... 50 Case 5: FAHE-5 ................................................................................................................................... 50 Case 6: FAHE-6 ................................................................................................................................... 50 Case 7: FAHE-7 ................................................................................................................................... 50 Case 8: FAHE-8 ................................................................................................................................... 50 Case 9: FAHE-9 ................................................................................................................................... 51 Case 10: FAHE-10 ............................................................................................................................... 51 Case 11: FAHE-11 ............................................................................................................................... 51 Case 12: FAHE-12 ............................................................................................................................... 51 Case 13: FAHE-12 ............................................................................................................................... 51

Test 1.18 – Firmware Update Requirements ...................................................................................... 53 Case 1: FWUP-1 .................................................................................................................................. 53 Case 2: FWUP-2 .................................................................................................................................. 53 Case 3: FWUP-3 .................................................................................................................................. 53 Case 4: FWUP-4 .................................................................................................................................. 53 Case 5: FWUP-5 .................................................................................................................................. 54 Case 6: FWUP-6 .................................................................................................................................. 54 Case 7: FWUP-7 .................................................................................................................................. 54 Case 8: FWUP-8 .................................................................................................................................. 54 Case 9: FWUP-9 .................................................................................................................................. 54

Test 1.19 – De-Allocation Requirements ............................................................................................. 55



Case 1: TRIM-1 ................................................................................................................................... 55 Case 2: TRIM-2 ................................................................................................................................... 55 Case 3: TRIM-3 ................................................................................................................................... 55

Test 1.20 – Sector Size and Namespace Support ................................................................................ 57 Case 1: SECTOR-1 .............................................................................................................................. 57 Case 2: SECTOR-2 .............................................................................................................................. 57 Case 3: SECTOR-3 .............................................................................................................................. 57

Test 1.21 – Set/Get Features Requirements ........................................................................................ 58 Case 1: GETF-1 ................................................................................................................................... 58

Test 1.22 – Error Injection Set Feature Identifier (0xC0) ................................................................. 59 Case 1: SERRI-1-15, ERRI-1,3-4, ERRIE-1-4, ERRIEDP-1-4, GERRI-1-2 ...................................... 59 Case 2: ERRI-2 .................................................................................................................................... 60

Test 1.23 – Clear Firmware Update History Set Feature Identifier (0xC1) .................................... 62 Case 1: CFUH-1-15 ............................................................................................................................. 62

Test 1.24 – Read Only/Write Through Mode Set Feature Identifier (0xC2) ................................... 63 Case 1: ROWTM-1 .............................................................................................................................. 63 Case 2: SROWTM-1-15 ...................................................................................................................... 63

Test 1.25 – Read Only/Write Through Mode Get Feature Identifier (0xC2) .................................. 65 Case 1: GROWTM-1-2 ........................................................................................................................ 65

Test 1.26 – Clear PCIe Correctable Error Counters Set Feature Identifier (0xC3) ....................... 66 Case 1: CPCIE-1-15 ............................................................................................................................. 66

Test 1.27 – Enable IEEE1667 Silo Set Feature Identifier (0xC4) ..................................................... 67 Case 1: S1667-1-15 .............................................................................................................................. 67

Test 1.28 – Enable IEEE1667 Silo Get Feature Identifier (0xC4) ..................................................... 68 Case 1: G1667-1-2 ............................................................................................................................... 68

Group 2: PCIe Requirements .................................................................................................... 70

Group 3: Reliability .................................................................................................................... 71 Group 4: Endurance ................................................................................................................... 72

Group 5: Thermal ....................................................................................................................... 75

Group 6: Form Factor Requirements ....................................................................................... 76

Group 7: SMBus Support .......................................................................................................... 77 Group 8: Security ........................................................................................................................ 78

Group 9: Labeling ....................................................................................................................... 79

Group 10: Compliance ............................................................................................................... 80

Group 11: Shock and Vibration ................................................................................................ 81 Group 12: NVMe Linux CLI Plug-In Requirements .............................................................. 82

Appendix A: TEST SETUP ........................................................................................................... 83

Appendix B: NOTES ON TEST PROCEDURES .............................................................................. 84

Appendix C: TEST TOOLS ................................................................ Error! Bookmark not defined. Appendix D: NVME-OF INTEGRATORS LIST REQUIREMENTS ...... Error! Bookmark not defined.



MODIFICATION RECORD 2020 July 31 (Version 1.0) Initial Release

David Woolf:

2020 November 2 (Version 1.0) Interim Release David Woolf:

1. Typos and editorial fixes.



ACKNOWLEDGMENTS

The UNH–IOL would like to acknowledge the efforts of the following individuals in the development of this test plan: David Woolf UNH InterOperability Laboratory



INTRODUCTION The University of New Hampshire’s InterOperability Laboratory (IOL) is an institution designed to improve the interoperability of standards–based products by providing a neutral environment where a product can be tested against other implementations of a common standard, both in terms of interoperability and conformance. This particular suite of tests has been developed to help implementers evaluate the functionality of NVMe products. These tests are designed to determine if a product conforms to requirements defined in the Open Compute Project NVMe Cloud SSD specification, hereafter referred to as the “CloudSSD Specification”). The tests contained in this document are organized in order to simplify the identification of information related to a test, and to facilitate in the actual testing process. Tests are separated into groups, primarily in order to reduce setup time in the lab environment, however the different groups typically also tend to focus on specific aspects of device functionality. A two–number, dot–notated naming system is used to catalog the tests. This format allows for the addition of future tests in the appropriate groups without requiring the renumbering of the subsequent tests. The test definitions themselves are intended to provide a high–level description of the motivation, resources, procedures, and methodologies specific to each test. Formally, each test description contains the following sections: Purpose The purpose is a brief statement outlining what the test attempts to achieve. The test is written at the functional level. References This section specifies all reference material external to the test suite, including the specific references for the test in question, and any other references that might be helpful in understanding the test methodology and/or test results. External sources are always referenced by a bracketed number (e.g., [1]) when mentioned in the test description. Any other references in the test description that are not indicated in this manner refer to elements within the test suite document itself (e.g., “Appendix 5.A”, or “Table 5.1.1–1”). Resource Requirements The requirements section specifies the test hardware and/or software needed to perform the test. This is generally expressed in terms of minimum requirements, however in some cases specific equipment manufacturer/model information may be provided. Last Modification This specifies the date of the last modification to this test. Discussion The discussion covers the assumptions made in the design or implementation of the test, as well as known limitations. Other items specific to the test are covered here as well. Test Setup The setup section describes the initial configuration of the test environment. Small changes in the configuration should not be included here, and are generally covered in the test procedure section (next). Procedure The procedure section of the test description contains the systematic instructions for carrying out the test. It provides a cookbook approach to testing, and may be interspersed with observable results. These procedures should be the ideal test methodology, independent of specific tool limitations or restrictions. Observable Results This section lists the specific observable items that can be examined by the tester in order to verify that the DUT is operating properly. When multiple values for an observable are possible, this section provides a short discussion on



how to interpret them. The determination of a pass or fail outcome for a particular test is generally based on the successful (or unsuccessful) detection of a specific observable. Possible Problems This section contains a description of known issues with the test procedure, which may affect test results in certain situations. It may also refer the reader to test suite appendices and/or other external sources that may provide more detail regarding these issues.



REFERENCES The following documents are referenced in this text:

1. Open Compute Project NVMe Cloud SSD Specification Version 1.0 (03102020)



Group 1: NVM Express Requirements Overview:

This section describes a method for performing conformance verification for NVMe products implementing OCP Cloud SSD requirements described in Chapter 3 of the NVMe Cloud SSD Specification. Notes:

The preliminary draft descriptions for the tests defined in this group are considered complete, and the tests are pending implementation (during which time additional revisions/modifications are likely to occur).



Test 1.1 – NVM Express Requirements Purpose: To verify that an NVMe SSD properly supports NVMe requirements. References: [1] Cloud SSD Specification 3.1 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: June 20, 2020 Discussion: The Cloud SSD Specification requires NVMe v1.4 compliance and documentation of optional feature support. Test Setup: See Appendix A. Case 1: NVMe-1 Test Procedure:

1. Execute all applicable tests defined in the most recent NVMe Test Plan published by UNH-IOL at https://www.iol.unh.edu/testing/storage/nvme/test-plans.

Observable Results:

1. Verify that all applicable tests pass. Case 2: NVMe-2 Test Procedure:

1. Review product documentation for optional feature list. Observable Results:

1. Verify that product documentation includes optional feature support that is not required by Cloud SSD spec-ification.

Case 3: NVMe-3 Test Procedure:

1. Review product documentation for vendor unique feature list. Observable Results:

1. Verify that product documentation includes vendor unique feature list. Possible Problems: None.



Test 1.2 – NVMe Reset Supported Purpose: To verify that an NVMe SSD properly supports NVMe reset. References: [1] Cloud SSD Specification 3.2 [2] UNH-IOL NVMe Conformance Test Suite Tests 6.1, 6.4 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: June 20, 2020 Discussion: The Cloud SSD Specification requires support for NVMe subsystem reset and controller reset. Test Setup: See Appendix A. Case 1: NVMeR-1 Test Procedure:

1. Configure the NVMe Host to read the CAP.NSSRS field to determine if the DUT supports writing to the NSSR.NSSRC field. If the NSSRS field is cleared to ‘0’, the test is not applicable. If the CAP.NSSRS field is set to ‘1’, continue to step 2.

2. Configure the NVMe host to read the CSTS.NSSRO register value, and record the value. If the value is not zero, perform a power cycle of the DUT and restart the test.

3. Configure the NVMe host to write a value of 4E564D65h (“NVMe”) to the NSSR.NSSRC field. 4. When the reset is complete, the PCIe link is reestablished, the NVMe controller is enabled and an Identify is

performed. Observable Results:

1. Verify that when the reset is performed, the drive is temporarily no longer visible from the host system. Depending on the reset and bring up speed of the device, this may not be observable.

2. Verify that the NVMe Controller is able to properly execute NVMe Identify command after the reset is complete.

3. Verify that the NVMe Controller performs the following actions when the NVM Subsystem Reset is initiated: a. The controller stops processing any outstanding Admin or I/O commands. b. All I/O Submission Queues are deleted. c. All I/O Completion Queues are deleted. d. The controller is brought to an Idle state. When this is complete, CSTS.RDY is cleared to ‘0’. e. All controller registers defined in section 3 of the NVMe Specification and internal controller state

are reset. Case 2: NVMeR-2 Test Procedure:

1. Configure the NVMe Host to write a value of ‘0’ to the CC.EN controller register field. 2. When the reset is complete, configure the NVMe Host to issue a Write and then a Read command to the

NVMe Controller. Observable Results:

1. Verify that the NVMe Controller is able to properly execute NVMe Write and Read commands after the reset is complete.

2. Verify that the NVMe Controller performs the following actions when the Controller Reset is initiated: a. The controller stops processing any outstanding Admin or I/O commands.



b. All I/O Submission Queues are deleted. c. All I/O Completion Queues are deleted. d. The controller is brought to an Idle state. When this is complete, CSTS.RDY is cleared to ‘0’. e. The Admin Queue registers (AQA, ASQ, or ACQ) are not reset as part of a controller reset. f. All other controller registers defined in section 3 of the NVMe Specification and internal con-

troller state are reset. Possible Problems: For case NVMeR-1, when the subsystem reset is performed, the drive is temporarily no longer visible from the host system. Depending on the reset and bring up speed of the device, this may not be observable.



Test 1.3 – NVMe Controller Configuration and Behavior Purpose: To verify that an NVMe SSD controller is properly configured. References: [1] Cloud SSD Specification 3.3 [2] UNH-IOL NVMe Conformance Test Suite Test 4.13, 1.1.2, 4.17 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: November 2, 2020 Discussion: The Cloud SSD Specification requires support for certain NVMe capabilities. Test Setup: See Appendix A. Case 1: NVMe-CFG-1 Test Procedure:

1. Configure the NVMe Host to set CC.EN to 0. 2. Configure the NVMe Host to read the CC.AMS register field of the NVMe Controller. 3. Configure the NVMe Host to read the CAP.AMS register field of the NVMe Controller. 4. Configure the NVMe Host to set the CC.AMS register field to 001b to enable Weighted Round Robin with

Urgent Priority Class 5. Configure the NVMe Host to read the CC.AMS register field of the NVMe Controller.

Observable Results:

1. Verify that in Step 2, the value of the CC.AMS register field is set to its default value of 000b when CC.EN is set to 0, to indicate a default arbitration method of Round Robin.

2. Verify that in Step 3, the CAP.AMS bit 17 is set to 1 to indicate support for Weighted Round Robin with Urgent Class Priority.

3. Verify that in Step 5, the value of the CC.AMS register field is set to of 001b, as set by the host in the previous step, to indicate that Weighted Round Robin with Urgent Class Priority arbitration is enabled.

Case 2: NVMe-CFG-2 Test Procedure:

1. Configure the NVMe Host to read the CAP.MPSMIN register field of the NVMe Controller 2. Configure the NVMe Host to issue an Identify command specifying CNS value 01h to the controller in order

to receive back an Identify Controller data structure. Observable Results:

1. Verify that the value for Maximum Data Transfer Size (MDTS) is 256 KB or larger. A value of 0h indicates no maximum data transfer size.


1. Configure the NVMe Host to read the CSTS.CFS field (bit 01) of the NVMe Controller. 2. Report the value of the CSTS.CFS field.

Observable Results:

1. Verify that the controller returns a value of 0h.




1. Collect the “Model Part Number” (MPN) from the product label and datasheet. 2. Configure the NVMe Host to issue an Identify command specifying CNS value 01h to the controller in order

to receive back an Identify Controller data structure and read the ASCII string in the Model Number (MN) field.

Observable Results:

1. Verify “Model Part Number” from the product label and datasheet is identical to the “Model Number” field reported in the Identify Controller Data Structure.


1. Configure the NVMe Host to read the Capabilities Register CAP.MQES field to obtain the Maximum Queue Size supported by the DUT. Record the value.

2. Configure the NVMe Host to perform a Create I/O Completion Queue and Create I/O Submission Queue with a Queue Size of 1024.

Observable Results:

1. Verify that the CAP.MQES field indicates a Queue Size of 1024 or higher. 2. Verify that the Create I/O Completion Queue and Create I/O Submission Queue commands complete

successfully. Case 6: NVMe-CFG-6 Test Procedure:

1. Configure the NVMe Host to perform a Create I/O Completion Queue and Create I/O Submission Queue with a Queue Size of 1024. Repeat until 64 Queue Pairs have been created.

2. Perform a READ operation for each queue pair. Observable Results:

1. Verify that the CAP.MQES field indicates a Queue Size of 1024 or higher. 2. Verify that the Create I/O Completion Queue and Create I/O Submission Queue commands complete

successfully for all 64 Queue pairs created. 3. Verify that the READ operations completed successfully.


1. Configure the NVMe Host to issue an Identify command specifying CNS value 02h to the controller in order to receive back an Active Namespace ID list.

2. Configure the NVMe Host to issue an Identify command specifying CNS value 00h to the controller in order to receive back an Identify Controller Namespace data structure and read the EUI64 field. Repeat for all Namespaces.

Observable Results:

1. Verify that the EUI64 value is unique for all Namespaces. Case 8: NVMe-CFG-8 Test Procedure:



1. Configure the NVMe Host to issue an Identify command specifying CNS value 02h to the controller in order to receive back an Active Namespace ID list.

2. Configure the NVMe Host to issue an Identify command specifying CNS value 00h to the controller in order to receive back an Identify Controller Namespace data structure and read the NGUID field. Repeat for all Namespaces.

Observable Results:

1. Verify that the NGUID value is unique for all Namespaces. Possible Problems: None.



Test 1.4 – NVMe Admin Command Set Purpose: To verify that an NVMe SSD properly supports the NVMe Admin Command Set. References: [1] Cloud SSD Specification 3.4 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: June 22, 2020 Discussion: The Cloud SSD Specification requires support for NVMe Admin Command Set. Test Setup: See Appendix A. Case 1: NVMe-AD-1 Test Procedure:

1. Configure the NVMe Host to perform the following NVMe Admin Commands, nominal supported values should be used.

1. Create I/O Completion Queue 2. Create I/O Submission Queue 3. Get Log Page 4. Identify (CNS=00h) 5. Set Features 6. Get Features 7. Abort 8. Asynchronous Event Request 9. Delete I/O Submission Queue 10. Delete I/O Completion Queue

Observable Results:

1. Verify that each admin command completes successfully. Case 2: NVMe-AD-2 Test Procedure:

1. Configure the NVMe Host to perform an Identify Command with CNS=00h 2. Configure the NVMe Host to perform an Identify Command with CNS=01h 3. Configure the NVMe Host to perform an Identify Command with CNS=02h 4. Configure the NVMe Host to perform an Identify Command with CNS=03h

Observable Results:

1. Verify that each Identify command completes successfully and all required fields are supported. 2. Verify that Bit 7 of the FPI field in the Namespace Data Structure returned in response to the Identify with

CNS=00h is set to 1, to indicate support for the Format Progress Indicator. 3. Verify that Bit 4 of the NSFEAT field in the Namespace Data Structure returned in response to the Identify

with CNS=00h is set to 1, to indicate support for Performance and Endurance Optimizations. Case 3: NVMe-AD-3 Test Procedure:

1. Configure the NVMe Host to perform an Identify Command with CNS=01h. Record the OACS field.



2. Configure the NVMe Host to perform an Identify Command with CNS=02h. 3. Configure the NVMe Host to perform a Namespace Management command with Create action. 4. Configure the NVMe Host to perform an Identify Command with CNS=02h. 5. Configure the NVMe Host to perform a Namespace Management command with Delete action. 6. Configure the NVMe Host to perform an Identify Command with CNS=02h.

Observable Results:

1. Verify that the controller indicates support for Namespace Management by setting Bit 3 of the OACS field in the Identify Controller Data Structure to 1.

2. Verify that each Namespace Management command completes successfully. 3. Verify that the Namespaces created and deleted are reflected in the Active Namespace List returned by the

controller in response to the Identify Command with CNS=02h. Case 4: NVMe-AD-4 Test Procedure:

1. Configure the NVMe Host to perform a Namespace Management command with Create action. 2. Configure the NVMe Host to perform a Namespace Attachment command to attach the created namespace. 3. Configure the NVMe Host to perform a Namespace Attachment command to detach the created namespace. 4. Configure the NVMe Host to perform a Namespace Management command with Delete action.

Observable Results:

1. Verify that each Namespace Attachment command completes successfully. Case 5: NVMe-AD-5 Test Procedure:

1. Configure the NVMe Host to perform an Identify Command with CNS=01h. Record the OACS field. 2. Configure the NVMe Host to perform a Format NVM command with SES=000b. 3. Configure the NVMe Host to perform a Format NVM command with SES=001b. 4. Configure the NVMe Host to perform a Format NVM command with SES=010b.

Observable Results:

1. Verify that the controller indicates support for the Format NVM command by setting Bit 1 of the OACS field in the Identify Controller Data Structure to 1.

2. Verify that each Format NVM command completes successfully. Case 6: NVMe-AD-6 Test Procedure:

1. Configure the NVMe Host to perform an Identify Command with CNS=01h. Record the OACS field. If Bit 6 of the OACS field is set to 0, this test is not applicable.

2. Configure the NVMe Host to perform a NVMe-MI Send command. 3. Configure the NVMe Host to perform a NVMe-MI Receive command.

Observable Results:

1. If NVMe-MI Send/Receive is supported, as indicated by Bit 6 of the OACS field in the Identify Controller Data Structure to 1, verify that each NVMe-MI Send/Receive command completes successfully.

Possible Problems: None.



Test 1.5 – Namespace Management / Attachment Commands Purpose: To verify that an NVMe SSD properly supports Namespace Management and Attachment Commands. References: [1] Cloud SSD Specification 3.4.1 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 2, 2020 Discussion: The namespace management command along with the attach/detach commands is used to increase device over-provisioning beyond the default minimum over-provisioning. Test Setup: See Appendix A. Case 1: NSM-1 Test Procedure:

1. Configure the NVMe Host to perform an Identify Command with CNS=01h. Record the OACS field. 2. Configure the NVMe Host to perform an Identify Command with CNS=02h. 3. Configure the NVMe Host to perform a Namespace Management command with Create action. 4. Configure the NVMe Host to perform an Identify Command with CNS=02h. 5. Configure the NVMe Host to perform a Namespace Management command with Delete action. 6. Configure the NVMe Host to perform an Identify Command with CNS=02h.

Observable Results:

1. Verify that the controller indicates support for Namespace Management by setting Bit 3 of the OACS field in the Identify Controller Data Structure to 1.

2. Verify that each Namespace Management command completes successfully. 3. Verify that the Namespaces created and deleted are reflected in the Active Namespace List returned by the

controller in response to the Identify Command with CNS=02h. Case 2: NSM-2 Test Procedure:

1. Configure the NVMe Host to perform an Identify Command with CNS=02h. 2. Configure the NVMe Host to perform a Namespace Management command with Create action and FLBAS

parameter set to 0, to indicate the controller should use the default value. 3. Configure the NVMe Host to perform an Identify Command with CNS=02h.

Observable Results:

1. Verify that the Namespaces created and deleted are reflected in the Active Namespace List returned by the controller in response to the Identify Command with CNS=02h and that all commands complete successfully.

2. Verify that the FLBAS value returned for each Identify Command is identical to indicate that the controller used the default FLBAS value.

Case 3: NSM-3 Test Procedure:

1. Configure the NVMe Host to perform an Identify Command with CNS=02h. 2. Configure the NVMe Host to perform a Namespace Management command with Create action and FLBAS

parameter set to 0.



3. Configure the NVMe Host to perform an Identify Command with CNS=02h. 4. Perform a Format NVM command to the created namespace specifying LBAF=0. 5. Configure the NVMe Host to perform an Identify Command with CNS=02h.

Observable Results:

1. Verify that the Format NVM command completes successfully. 2. Verify that the FLBAS value returned for each Identify Command is identical to indicate that the controller

used the default FLBAS value when performing the Format NVM operation where LBAF=0. Possible Problems: None.



Test 1.6 – Namespace Utilization Purpose: To verify that an NVMe SSD properly supports the Namespace Utilization field. References: [1] Cloud SSD Specification 3.4.2 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 2, 2020 Discussion: The NUSE shall be equal to the number of logical blocks currently allocated in the namespace. NUSE cannot be hardcoded to be equal to NCAP. Test Setup: See Appendix A. Case 1: NUSE-1 Test Procedure:

1. Perform a Format NVM command an active namespace specifying SES=001b. 2. Perform an Identify Command with CNS=00h, record values of NUSE and NCAP. 3. Write 1 GB of data using NVMe Write operation. 4. Perform an Identify Command with CNS=00h, record values of NUSE and NCAP. 5. Perform a Dataset Management Command with a Deallocate attribute for one of the allocated blocks. 6. Perform an Identify Command with CNS=00h, record values of NUSE and NCAP.

Observable Results:

1. Verify that after the Format NVM command completes successfully, the NUSE value is reported as 0 and NUSE and NCAP are not equal.

2. Verify that after the Write operation, the NUSE value indicates the correct number of Logical Blocks Allo-cated relative to the formatted LBA size, should be greater than 0, and NUSE and NCAP are not equal.

3. Verify that after the Dataset Management Deallocate operation, the NUSE value indicates the correct number of Logical Blocks Allocated relative to the formatted LBA size, should be less than was observed after the Write operation, and NUSE and NCAP are not equal.




Test 1.7 – NVMe I/O Command Set Purpose: To verify that an NVMe SSD properly supports the NVMe I/O Command set. References: [1] Cloud SSD Specification 3.5 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 2, 2020 Discussion: The device shall support all mandatory NVMe I/O commands. The device shall support Dataset Management and at a minimum De-Allocate. Test Setup: See Appendix A. Case 1: NVMe-IO-1 Test Procedure:

1. Perform a Write Operation to an active namespace. 2. Perform a Read operation to an active namespace. 3. Perform a Flush operation to an active namespace.

Observable Results:

1. Verify that each command completes successfully. Case 2: NVMe-IO-2 Test Procedure:

1. Perform a Format NVM command an active namespace specifying SES=001b. 2. Perform an Identify Command with CNS=00h, record values of NUSE and NCAP. 3. Write 1 GB of data using NVMe Write operation. 4. Perform an Identify Command with CNS=00h, record values of NUSE and NCAP. 5. Perform a Dataset Management Command with a Deallocate attribute for one of the allocated blocks. 6. Perform an Identify Command with CNS=00h, record values of NUSE and NCAP.

Observable Results:

1. Verify that after the Format NVM command completes successfully, the NUSE value is reported as 0 and NUSE and NCAP are not equal.

2. Verify that after the Write operation, the NUSE value indicates the correct number of Logical Blocks Allo-cated relative to the formatted LBA size, should be greater than 0, and NUSE and NCAP are not equal.

3. Verify that after the Dataset Management Deallocate operation, the NUSE value indicates the correct number of Logical Blocks Allocated relative to the formatted LBA size, should be less than was observed after the Write operation, and NUSE and NCAP are not equal.




Test 1.8 – Optional NVMe Feature Support Purpose: To verify that an NVMe SSD properly supports the required NVMe Optional features. References: [1] Cloud SSD Specification 3.6 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 6, 2020 Discussion: Telemetry shall be supported. Both Host Initiated Telemetry and Controller Initiated Telemetry shall be supported. Timestamp shall be supported to align the devices internal logs. Test Setup: See Appendix A. Case 1: NVMe-OPT-1 Test Procedure:

1. The Testing Station acting as host should perform an Identify Command to CNS 01h for the Controller Data Structure and record the value of the Log Page Attributes (LPA) field.

2. The Testing Station as host should perform a Get Log Page command for LID 07h, Telemetry Host-Initiated 3. The Testing Station as host should perform a Get Log Page command for LID 08h, Telemetry Controller-

Initiated. Observable Results:

1. Verify that Bit 3 of the LPA field of the Identify Controller Data Structure is set to 1 to indicate support of Telemetry Host-Initiated and Controller Initiated Log Pages.

2. Verify that the Get Log Page Commands for LID 07h and 08h complete successfully and the Log Page info is returned.

Case 2: NVMe-OPT-2 Test Procedure:

Check the Identify Controller Data Structure ONCS field Bit 6 to determine if the DUT supports the Timestamp feature. If the DUT does not support the Timestamp feature, then this test is not applicable.

Perform a Set Feature with FID 0Eh to set a timestamp value in the DUT, start a timer on the testing station. Perform a Get Feature operation with FID=0Eh to request to current timestamp value. Perform a Controller Level Reset. Perform a Get Feature operation with FID=0Eh to request to current timestamp value.

Observable Results:

Verify in all cases that the Set/Get Feature operations for FID=0Eh completed successfully. If the Synch bit (returned in response to the Get Feature command) is set to 0, indicating that the controller

does not stop counting, verify that the timestamp returned after the controller level reset matches the timer running on the testing station. If the Timestamp Origin field is set to 000b verify that the timestamp shows the time since the Controller

Level Reset. If the Timestamp Origin field is set to 001b, verify that the timestamp reflects the value that was initialized

in the Set Features command. Possible Problems: None.



Test 1.9 – Command Timeout Purpose: To verify that an NVMe SSD properly supports the required NVMe Optional features. References: [1] Cloud SSD Specification 3.7 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 6, 2020 Discussion: ADMIN Commands shall take no more than 10 seconds from submission to completion. The only exceptions to this shall be Format, Self-Test and Sanitize and the TCG commands Revert, Revert SP and Change Key. When CSTS.RDY is set to 1b, I/O Commands shall take no more than 8 seconds from submission to completion. The device shall not have more than 7 IOs take more than 2 seconds in one hour. I/O command processing time shall not be a function of device capacity. Test Setup: See Appendix A. Case 1: CTO-1, CTO-2 Test Procedure:

1. The Testing Station acting as host should perform Create I/O Completion Queue Command. 2. The Testing Station acting as host should perform Create I/O Submission Queue Command. 3. The Testing Station should begin a continuous stream of Write and Read operations, 64B in length to the

DUT and continue this throughout this test procedure. While that I/O stream is being performed, the Testing Station should perform each following step 1000 times:

a. The Testing Station should perform a Read operation, immediately followed by an Abort command to abort that Read operation.

b. The Testing Station should perform an Asynchronous Event Request command. c. The Testing Station should perform a Doorbell Buffer Config command. d. The Testing Station should perform a Directive Send and Directive Receive command. e. The Testing Station should perform a Firmware Image Download Command. f. The Testing Station should perform a Firmware Commit Command. g. The Testing Station should perform a Get Feature command for all supported features. h. The Testing Station should perform a Set Feature command for all supported features. i. The Testing Station should perform a Get Log Page command for all supported LIDs. j. The Testing Station should perform an Identify Command to all supported CNS values. k. The Testing Station should perform a Namespace Management Create Action command. l. The Testing Station should perform a Namespace Attachment Attach Action command. m. The Testing Station should perform a Namespace Attachment Detach Action command. n. The Testing Station should perform a Namespace Management Delete Action command. o. The Testing Station should perform a Get LBA status command.

Observable Results:

1. Verify that none of the commands took more than 10 seconds from submission to completion. 2. If any commands take more than 10 seconds to complete, issue a warning in the test results log.

Case 2: CTO-3 Test Procedure:

Configure the Testing Station to perform a Controller Level Reset and wait for the Controller to set CSTS.RDY to 1.



The Testing Station should begin a continuous stream of Write and Read operations for 1 hour. Operations should be 64B in length. Measure the completion time of each I/O operation.

Observable Results: Verify that all I/O operations finished with 8 seconds from submission to completion and no more than 7

I/Os took more than 2 seconds during the test. If any commands take more than 10 seconds to complete, issue a warning in the test results log.




Test 1.10 – Standard Log Page Requirements Purpose: To verify that an NVMe SSD properly supports the Standard Log Page requirements. References: [1] Cloud SSD Specification 3.8.1 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 6, 2020 Discussion: DUT shall support the following Log Pages: Error Information (Log Identifier 01h), SMART/Health Information (Log Identifier 02h), Firmware Slot Information (Log Identifier 03h), Commands Supported and Effects (Log Page 0x05), Telemetry Host-Initiated (Log Page 0x07), Telemetry Controller-Initiated (Log Page 0x08).

Further, under no conditions shall the Percentage Used field in the SMART/Health Information (Log Identifier 02h) be reset. The Percentage Used field in the SMART/Health Information (Log Identifier 02h) shall be based on the average P/E cycle of the device. In addition, this field shall be based on the actual P/E cycle count of the media and not on the Power On Hours (POH) of the device. Test Setup: See Appendix A. Case 1: STD-LOG-1 Test Procedure:

1. The Testing Station acting as host should perform a Get Log Page for LID 01h (Error Information Log) Observable Results:

1. Verify that the expected Log Page is returned and the command completes with status Success. Case 2: STD-LOG-2, STD-LOG-3, STD-LOG-4 Test Procedure:

1. The Testing Station acting as host should perform a Get Log Page for LID 02h (SMART/Health Information Log)

2. Perform a Controller Level Reset 3. The Testing Station acting as host should perform a Get Log Page for LID 02h (SMART/Health Information

Log) Observable Results:

1. Verify that the expected Log Page is returned each time and the command completes with status Success. 2. Verify that the Percentage Used field in the SMART/Health Information Log is not reset between the two

Log Page requests. 3. Verify that the Percentage Used field is based on the average P/E cycle of the device. In addition, this field

shall be based on the actual P/E cycle count of the media and not on the Power On Hours (POH) of the device. Case 3: STD-LOG-5, STD-LOG-6, STD-LOG-7, STD-LOG-8 Test Procedure:

1. The Testing Station acting as host should perform a Get Log Page for LID 03h (Firmware Slot Information Log).

2. The Testing Station acting as host should perform a Get Log Page for LID 05h (Commands Supported and Effects Log).

3. The Testing Station acting as host should perform a Get Log Page for LID 07h (Telemetry Host-Initiated Log)



4. The Testing Station acting as host should perform a Get Log Page for LID 08h (Telemetry Controller-Initiated Log)

Observable Results:

1. Verify that the expected Log Page is returned each time and the command completes with status Success. Possible Problems: None.



Test 1.11 – Telemetry Logging and Interface for Failure Analysis Purpose: To verify that an NVMe SSD properly supports the Telemetry Logging field. References: [1] Cloud SSD Specification 3.8.2 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 23, 2020 Discussion: The device shall track the operational/event history and any critical parameters that can be used to debug issues. The supplier shall provide a table that categorizes the reason identifiers. Test Setup: See Appendix A. Case 1: TEL-1,2,3 Test Procedure:

1. Perform a Get Log Page for LID 07h, Telemetry Host Initiated. 2. Perform a Get Log Page for LID 08h, Telemetry Controller Initiated. 3. Perform a Get Log Page for LID 07h, Telemetry Host Initiated. 4. Perform a Get Log Page for LID 08h, Telemetry Controller Initiated. 5. Remove power to perform an ungraceful shutdown. 6. Perform a Get Log Page for LID 07h, Telemetry Host Initiated. 7. Perform a Get Log Page for LID 08h, Telemetry Controller Initiated.

Observable Results:

1. Verify that the Log Pages returned in Step 3 and 4 matched the Log Pages returned in Step 6 and 7 with the exception of counts for power cycles.

Case 2: TEL-4 Test Procedure:

8. Perform a Get Log Page for LID 07h, Telemetry Host Initiated. 9. Perform a Get Log Page for LID 08h, Telemetry Controller Initiated. 10. Perform a Controller Level Reset. 11. Perform a Get Log Page for LID 07h, Telemetry Host Initiated. 12. Perform a Get Log Page for LID 08h, Telemetry Controller Initiated.

Observable Results:

2. Verify that the Reason Identifier field in each Log Page returned for LID 07h identical. 3. Verify that the Reason Identifier field in each Log Page returned for LID 08h identical.


1. The Testing Station should Perform 1000 alternating Write and 1000 operations, 64B in length to the DUT. Determine the maximum observed IO latency with no Telemetry Operations occurring.

2. The Testing Station should Perform 100 alternating Write and 1000 operations, 64B in length to the DUT. Determine the maximum observed IO latency.

3. Perform a Get Log Page for LID 07h, Telemetry Host Initiated. 4. Perform a Get Log Page for LID 08h, Telemetry Controller Initiated.




Observable Results:

1. Verify that the Data Area 1 is populated in each Log Page returned for LID 07h and LID 08h. 2. Verify that no single IO operation incurred a latency that exceeded the maximum observed latency in Step 1,

plus 10 ms.


1. Perform a Get Feature command for FID 0Bh, Asynchronous Event Configuration. Observable Results:

1. Verify that Telemetry Log Notices (Bit 10) is set to 0. Possible Problems: None.



Test 1.12 – SMART Cloud Health Log (0xC0) Purpose: To verify that an NVMe SSD properly supports the SMART Cloud Health Log. References: [1] Cloud SSD Specification 3.8.3 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 6, 2020 Discussion: Regarding the SMART Cloud Health Log:

• All values in the Vendor Log pages shall be persistent across power cycles unless otherwise specified. • All counters shall be saturating counters (i.e. if the counter reaches the maximum allowable size it stops

incrementing and does NOT roll back to 0 unless otherwise specified). • All values in logs shall be little endian format. • A normalized counter, unless otherwise specified, shall be reported as the following: 100% shall represent

the number at factory exit. 1% shall represent the minimum amount to be reliable. A value of 0% means the device shall no longer be considered reliable. 100% shall be represented as 0x64.

• Devices shall support the attributes listed in section 5.14.1.2 of the NVMe specification version 1.4. • A Read of any of the SMART logs (0x02 or 0xC0) shall not require an update of the SMART values. It shall

be a simple read of the current data and shall not block IO. • Unless otherwise specified, the device shall update these values in the background at least once every ten

minutes. • The composite and raw temperature sensor values shall be updated when the log page is accessed. • All assert events and controller-initiated log captures will require an associated vendor-specific “Reason

Identifier” that uniquely identifies the assert /controller condition • The device shall not lose any of the SMART (Health 0x02 or Cloud Health 0xC0) data logs which are more

than 10 minutes old including across power cycles/resets. • The device shall not lose any back up energy source failure information or SMART (Health 0x02 or Cloud

Health 0xC0) critical warnings including across power cycles/resets. Test Setup: See Appendix A. Case 1: SLOG-1 Test Procedure:

1. The Testing Station acting as host should perform a Get Log Page for LID 0xC0 (SMART Cloud Health Log).

2. Power Cycle the Device. 3. The Testing Station acting as host should perform a Get Log Page for LID 0xC0 (SMART Cloud Health

Log). Observable Results:

1. Verify that the SMART Cloud Health Log contents were persistent across power cycle. With the following exceptions:

a. Unaligned I/O (Bytes 143:136) Case 2: SLOG-2 Test Procedure:



1. It may be advisable to perform this test after other tests included in this test suite. Due to time constraints it may not be possible to perform this test. Alternatively vendors could provide a DUT with artificially inflated counter values.


3. Perform Write I/O and Read I/O to the DUT at the maximum throughput the testing station is capable of for 24 hours.


5. Check the Physical Media Units Written and Physical Media Units Read values. 6. Repeat previous 2 steps until the values returned for Physical Media Units Written and Physical Media Units

Read is FFFFFFFFFFFFFFFFh. Observable Results:

1. Verify that the SMART Cloud Health Log counters for Physical Media Units Written and Physical Media Units Read did not wrap to 0.

Case 3: SLOG-3 Test Procedure:


2. Perform 1000 Write I/O operations and 1000 Read I/O operations of 4096 Bytes each to the DUT. 3. The Testing Station acting as host should perform a Get Log Page for LID 0xC0 (SMART Cloud Health


1. Verify that the SMART Cloud Health Log counters for Physical Media Units Written and Physical Media Units Read increased by the expected amount based on the number of I/Os performed and their size and that the counter was in little endian format.



2. Perform Write I/O operations and Read I/O operations of 4096 Bytes each to the DUT for 10 minutes. 3. The Testing Station acting as host should perform a Get Log Page for LID 0xC0 (SMART Cloud Health


1. Verify that the SMART Cloud Health Log counters for the following parameters include normalized counters which are accurate according to the amount of I/O performed between requests for the SMART Cloud Health Log.

a. System Data % Used b. % Free Blocks


1. The Testing Station acting as host should perform a Get Log Page for LID 0x02 (SMART / Health Information Log).

Observable Results:



1. Verify that the Log Page returned contains all fields specified in the SMART / Health Information Log from the NVMe v1.4 specification.


1. The Testing Station should Perform 1000 alternating Write and 1000 operations, 64B in length to the DUT. Determine the maximum observed IO latency with no Telemetry Operations occurring.


3. Perform a Get Log Page for LID 02h, SMART / Health Information Log. 4. Perform a Get Log Page for LID C0h, SMART Cloud Health Log. 5. The Testing Station should Perform 1000 alternating Write and 1000 operations, 64B in length to the DUT.

Determine the maximum observed IO latency. Observable Results:

1. Verify that no single IO operation incurred a latency that exceeded the maximum observed latency in Step 1, plus 10 ms.



2. Perform Write I/O operations and Read I/O operations of 4096 Bytes each to the DUT for 10 minutes. 3. The Testing Station acting as host should perform a Get Log Page for LID 0xC0 (SMART Cloud Health


1. Verify that the SMART Cloud Health Log counters for the following parameters include normalized counters which are accurate according to the amount of I/O performed between requests for the SMART Cloud Health Log.

a. Physical Media Units Written (Bytes 15:0) b. Physical Media Units Read (Bytes 31:16) c. System Data % Used (Byte 80) d. % Free Blocks (Byte 120)


1. The Testing Station acting as host should perform a Get Log Page for LID 0x02 (SMART / Health Information Log).

2. Perform Write I/O operations and Read I/O operations of 4096 Bytes each to the DUT for 3 minutes. 3. The Testing Station acting as host should perform a Get Log Page for LID 0x02 (SMART / Health

Information Log). 4. Wait 3 minutes with no I/O. 5. The Testing Station acting as host should perform a Get Log Page for LID 0x02 (SMART / Health

Information Log). Observable Results:

1. Verify that the SMART / Health Information Log value for Composite Temperature was updated each time the Log Page was returned.




1. The Testing Station acting as host should perform a Get Log Page for LID 0xC0 (Cloud Health Log). Observable Results:

1. Verify that all assert events and controller-initiated log captures have an associated vendor-specific “Reason Identifier” that uniquely identifies the assert /controller condition. This may require vendor provided docu-mentation.

Case 10: SLOG-10, SLOG-11 Test Procedure:

1. The Testing Station acting as host should perform a Get Log Page for LID 0x02 (SMART / Health Information Log) and LID 0xC0.

2. Wait 9 minutes 3. Perform a Controller Level Reset 4. The Testing Station acting as host should perform a Get Log Page for LID 0x02 (SMART / Health

Information Log) and LID 0xC0. 5. The Testing Station acting as host should perform a Get Log Page for LID 0x02 (SMART / Health

Information Log) and LID 0xC0. 6. Wait 10 minutes 7. Perform a Controller Level Reset 8. The Testing Station acting as host should perform a Get Log Page for LID 0x02 (SMART / Health

Information Log) and LID 0xC0. 9. The Testing Station acting as host should perform a Get Log Page for LID 0x02 (SMART / Health

Information Log) and LID 0xC0. 10. Wait 11 minutes 11. Perform a Controller Level Reset 12. The Testing Station acting as host should perform a Get Log Page for LID 0x02 (SMART / Health

Information Log) and LID 0xC0. 13. Repeat the entire procedure using Conventional Reset instead of Controller Level Reset, then repeat again

and remove power rather than performing a reset. Observable Results:

1. Verify that in each case the contents of the SMART Health Log and the SMART Cloud Health Log are identical.




Test 1.13 – SMART Cloud Attributes Log Page Purpose: To verify that an NVMe SSD properly supports the SMART Cloud Attributes Log Page. References: [1] Cloud SSD Specification 3.8.4 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 6, 2020 Discussion: There are requirements on the values in the SMART Cloud Attributes Log Page. Test Setup: See Appendix A. Case 1: SMART-1 Test Procedure:


2. Record bytes 15:0 3. Perform a Write Operation of 64 Bytes. 4. The Testing Station acting as host should perform a Get Log Page for LID 0xC0 (SMART Cloud Health

Log). 5. Record bytes 15:0

Observable Results:

1. Verify that the SMART Cloud Attributes Log Page is 512 bytes in size. 2. Verify that the Physical Media Units Written field (Bytes 15:0) in the SMART Cloud Attributes Log Page

were updated properly due to the Write operation. Case 2: SMART-2 Test Procedure:


2. Record bytes 31:16 3. Perform a Read Operation of 64 Bytes. 4. The Testing Station acting as host should perform a Get Log Page for LID 0xC0 (SMART Cloud Health

Log). 5. Record bytes 31:16

Observable Results:

1. Verify that the Physical Media Units Read field (Bytes 31:16) in the SMART Cloud Attributes Log Page were updated properly due to the Read operation.

Case 3: SMART-3 Test Procedure:


2. Record bytes 39:32



Observable Results: 1. Verify that the Bad User NAND Blocks field (Bytes 39:32) in the SMART Cloud Attributes Log Page are

set to 0x64 for Bytes 39:38 (Normalized Value) and 0x00 for 37:32 (Raw Count), assuming the device is in the same state as factory exit.



2. Record bytes 47:40 Observable Results:

1. Verify that the Bad System NAND Blocks field (Bytes 47:40) in the SMART Cloud Attributes Log Page are set to 0x64 for Bytes 47:46 (Normalized Value) and 0x00 for 45:40 (Raw Count), assuming the device is in the same state as factory exit.




1. Verify that the XOR Recovery Count (Bytes 55:48) in the SMART Cloud Attributes Log Page is set to 0x00, assuming the device is in the same state as factory exit.




1. Verify that the Uncorrectable Read Error Count (Bytes 63:56) in the SMART Cloud Attributes Log Page is set correctly.




1. Verify that the Soft ECC Error Count (Bytes 71:64) in the SMART Cloud Attributes Log Page is set correctly. Case 8: SMART-8 Test Procedure:





1. Verify that the End to End Correction Counts (Bytes 79:72) in the SMART Cloud Attributes Log Page is set correctly.



2. Record byte 80. 3. The testing station should perform Write operations to fill the device, then perform a Format NVM command

to erase all data. 4. The Testing Station acting as host should perform a Get Log Page for LID 0xC0 (SMART Cloud Health

Log). 5. Record byte 80.

Observable Results:

1. Verify that the System Data % Used (Byte 80) in the SMART Cloud Attributes Log Page is set correctly. Case 10: SMART-10 Test Procedure:



1. Verify that the Refresh Counts (Bytes 87:81) in the SMART Cloud Attributes Log Page is set correctly. Case 11: SMART-11 Test Procedure:



1. Verify that the User Data Erase Counts (Bytes 95:88) in the SMART Cloud Attributes Log Page is set cor-rectly.




1. Verify that the Thermal Throttling Status and Counts (Bytes 97:96) in the SMART Cloud Attributes Log Page is set to 0, assuming the device is in factory exit condition.






1. Verify that the Bytes 103:98 in the SMART Cloud Attributes Log Page are set to 0. Case 14: SMART-14 Test Procedure:



1. Verify that the PCIe Correctable Error Count (Bytes 111:104) in the SMART Cloud Attributes Log Page is set to 0, assuming the device is in factory exit condition.




1. Verify that the Incomplete Shutdowns (Bytes 115:112) in the SMART Cloud Attributes Log Page is set to 0, assuming the device is in factory exit condition.




1. Verify that Bytes 119:116 in the SMART Cloud Attributes Log Page are set to 0. Case 17: SMART-17 Test Procedure:


Observable Results:

1. Verify that the % Free Blocks (Byte 120) in the SMART Cloud Attributes Log Page is set correctly. Case 18: SMART-18



Test Procedure: 1. The Testing Station acting as host should perform a Get Log Page for LID 0xC0 (SMART Cloud Health




Observable Results:

1. Verify that the Capacitor Health (Bytes 129:128) in the SMART Cloud Attributes Log Page is a. Set to 0xFFFF if no capacitor is present b. Set to a value greater than 100 if the device is in factory exit condition c. Not set to a negative value.



Observable Results:



Observable Results:

1. Verify that the Unaligned I/O (Bytes 143:136) in the SMART Cloud Attributes Log Page are set to 0, assum-ing the device is in factory exit condition.



Observable Results:

1. Verify that the Security Version Number (Bytes 151:144) in the SMART Cloud Attributes Log Page are set to less than 3.





2. The Testing Station acting as host should perform an Identify Namespace command (CNS=00h) to retrieve the Namespace Data Structure.

Observable Results:

1. Verify that the NUSE value reported in the SMART Cloud Health Log (Bytes 159:152) is identical to the NUSE value reported in the Namespace Data Structure.



2. Reduce the device power supply and allow the device to restart. 3. The Testing Station acting as host should perform a Get Log Page for LID 0xC0 (SMART Cloud Health


1. Verify that the PLP Start Count value (Bytes 175:160) reported in the SMART Cloud Health Log increments by 1 due to the power loss condition during the test procedure.



2. The Testing Station acting as host should perform a Get Log Page for LID 09h Endurance Group Log). Observable Results:

1. Verify that the Endurance Estimate value reported in the SMART Cloud Health Log (Bytes 191:176) is identical to the Endurance Estimate value reported in the Endurance Group Log.



Observable Results:



Observable Results:

1. Verify that the Log Page Version (Bytes 495:494) in the SMART Cloud Attributes Log Page is set to 0x0002. Case 28: SMART-28 Test Procedure:




Observable Results: 1. Verify that the Log Page GUID (Bytes 511:496) in the SMART Cloud Attributes Log Page is set to 0x

AFD514C97C6F4F9CA4f2BFEA2810AFC5. Possible Problems: None.



Test 1.14 – Error Recovery Log Page Purpose: To verify that an NVMe SSD properly supports the Error Recovery Log Page. References: [1] Cloud SSD Specification 3.8.5 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 7, 2020 Discussion: The vendor-specific log page, 0xC1 shall be 512-bytes and has defined attributes. Test Setup: See Appendix A. Case 1: EREC-1 Test Procedure:

1. The Testing Station acting as host should perform a Get Log Page for LID 0xC1 (Error Recovery Log Page). Observable Results:

1. Verify that the Error Recovery Log Page is 512 bytes in size. 2. Verify that the Panic Reset Wait Time (Bytes 1:0) is set correctly.

Case 2: EREC-2 Test Procedure:


1. Verify that the Panic Reset Action (Byte 2) is set correctly. 2. Verify that Bit 7:6 of the Panic Reset Action is set to 0.



1. Verify that the Device Recovery Action (Byte 3) is set correctly. 2. Verify that Device Recovery Action is not set to 0x06-0xFF.



1. Verify that the Panic ID (Bytes 11:4) is set correctly. Case 5: EREC-5



Test Procedure: 1. The Testing Station acting as host should perform a Get Log Page for LID 0xC1 (Error Recovery Log Page).

Observable Results:

1. Verify that the Device Capabilities (Bytes 15:12) is set correctly. Case 6: EREC-6 Test Procedure:


1. Verify that the if Device Recovery Action (Byte 3) is not set to 0x2, that Vendor Specific Recovery Option (Byte 16) is set to 0x0.



1. Verify that Bytes 19:17 are set to 0x0. Case 8: EREC-8 Test Procedure:


1. Verify that the if Device Recovery Action (Byte 3) is not set to 0x2, that Vendor Specific Command CDW12 (Bytes 23:20) are set to 0x0.



1. Verify that the if Device Recovery Action (Byte 3) is not set to 0x2, that Vendor Specific Command CDW13 (Bytes 27:24) are set to 0x0.



1. Verify that Bytes 493:28 are set to 0x0. Case 11: EREC-11 Test Procedure:

1. The Testing Station acting as host should perform a Get Log Page for LID 0xC1 (Error Recovery Log Page).



Observable Results:

1. Verify that Log Page Version (Bytes 495:494) are set to 0x0001. Case 12: EREC-12 Test Procedure:


1. Verify that Log Page GUID (Bytes 511:496) are set to 0x5A1983BA3DFD4DABAE3430FE2131D944 Possible Problems: None.



Test 1.15 – Firmware Activation History Purpose: To verify that an NVMe SSD properly supports Firmware Activation History. References: [1] Cloud SSD Specification 3.8.6 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 13, 2020 Discussion: The vendor-specific log page, 0xC2 shall be 4096-bytes and has a defined field format. Test Setup: See Appendix A. Case 1: FWHST-LOG-1, FWHT-LOG-2 Test Procedure:

1. The Testing Station acting as host should perform a Get Log Page for LID 0xC2. 2. Provide a warning to user that next steps in the test procedure are irreversible and ask user whether to proceed

or not. 3. The Testing Station should download 2 firmware images to the DUT. 4. The Testing Station should alternate activating the 2 firmware images 21 times using the Firmware Commit

command. After each firmware activation the Testing Station acting as host should perform a Get Log Page for LID 0xC2.

Observable Results:

1. Verify that in the first returned Log Page there are no entries in Bytes 1287:8. 2. Verify that in the second returned Log Page, Valid Firmware Activation History Entries 0-20 (Bytes 1287:8)

contain the last 20 firmware activation entries, and that the latest activation entry appears in the Entry 0. Case 2: FWHST-LOG-3 Test Procedure:

1. The Testing Station acting as host should perform a Get Log Page for LID 0xC2. 2. The Testing Station should download 2 firmware images to the DUT. 3. The Testing Station should perform a Firmware Commit command with Commit Actions (CA) of 000b,

001b, 010b, 011b. 4. The Testing Station acting as host should perform a Get Log Page for LID 0xC2.

Observable Results:

1. Verify that only 4 additional entries are recorded in the Valid Firmware Activation History Entries field (Bytes 7:4).

2. Verify that only 4 additional entries are recorded in the Firmware Activation History Entry N (Bytes 1287:8) Case 3: FWHST-LOG-4 Test Procedure:

1. The Testing Station acting as host should perform a Get Log Page for LID 0xC2. 2. The Testing Station should download 2 identical firmware images to the DUT. 3. The Testing Station should perform a Firmware Commit command with following criteria:

a. Power on Hours is within 1 minute from the last RECORDED entry b. Power cycle count is the same



c. Current firmware is the same d. New firmware activated is the same e. Slot number is the same f. Commit Action Type is the same g. The Result field has not changed

4. The Testing Station acting as host should perform a Get Log Page for LID 0xC2. Observable Results:

1. Verify that no additional entries are recorded in the Valid Firmware Activation History Entries field (Bytes 7:4) or in the Firmware Activation History Entry N (Bytes 1287:8)

Case 4: FWHST-LOG-5 Test Procedure:

1. Perform all test cases in Test 1.16. The Testing Station acting as host should perform a Get Log Page for LID 0xC1 (Error Recovery Log Page).

Observable Results:

1. Verify that the DUT passes all test cases in Test 1.16. Possible Problems: None.



Test 1.16 – Firmware Activation History Log Page Format Purpose: To verify that an NVMe SSD properly implements the Firmware Activation History Log Page. References: [1] Cloud SSD Specification 3.8.6.1 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 13, 2020 Discussion: The vendor-specific log page, Firmware Activation History Log Page, has LID 0xC2 and shall be 4096-bytes and has a defined field format. Test Setup: See Appendix A. Case 1: FAHL-1 Test Procedure:


3. Verify that the Firmware Activation History Log Page is 4096 bytes in size. 4. Verify that the Log Identifier (Byte 0) is set to 0xC2.

Case 2: FAHL-2 Test Procedure:


1. Verify that the Firmware Activation History Log Page is 4096 bytes in size. 2. Verify that Bytes 3:1 are set to 0x0.

Case 3: FAHL-3 Test Procedure:

1. The Testing Station acting as host should perform a Get Log Page for LID 0xC2. 2. Perform a Firmware Commit command. 3. The Testing Station acting as host should perform a Get Log Page for LID 0xC2. 4. Perform a Set Feature Clear Firmware Update Activation History command. 5. The Testing Station acting as host should perform a Get Log Page for LID 0xC2.

Observable Results:

1. If the DUT is directly from the Factory verify that Valid Firmware Activation History Entries (Bytes 7:4) is set to 0x0 in the first returned Log Page.

2. Verify that Valid Firmware Activation History Entries (Bytes 7:4) is set to 0x1 in the second returned Log Page.

3. Verify that Valid Firmware Activation History Entries (Bytes 7:4) is set to 0x0 in the third returned Log Page.

Case 4: FAHL-4



Test Procedure: 1. The Testing Station acting as host should perform a Get Log Page for LID 0xC2. 2. Provide a warning to user that next steps in the test procedure are irreversible and ask user whether to proceed

or not. 3. The Testing Station should download 2 firmware images to the DUT. 4. The Testing Station should alternate activating the 2 firmware images 21 times using the Firmware Commit


Observable Results:

1. Verify that in the first returned Log Page there are no entries in Bytes 1287:8. 2. Verify that in the second returned Log Page, Valid Firmware Activation History Entries 0-20 (Bytes 1287:8)

contain the last 20 firmware activation entries, and that the latest activation entry appears in the Entry 0. Case 5: FAHL -5 Test Procedure:


1. Verify that Bytes 4077:1288 are set to 0x0. Case 6: FAHL-6 Test Procedure:


1. Verify that Log Page Version (Bytes 4079:4078) are set to 0x0001. Case 7: FAHL-7 Test Procedure:


1. Verify that Log Page GUID (Bytes 4095:4080) are set to 0xD11Cf3AC8AB24DE2A3F6DAB4769A796D. Possible Problems: None.



Test 1.17 – Firmware Activation History Entry Format Purpose: To verify that an NVMe SSD properly implements the Firmware Activation History Entry format. References: [1] Cloud SSD Specification 3.8.6.2 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 13, 2020 Discussion: The specification defines the History Entry format for recording Firmware Activation History events. Test Setup: See Appendix A. Case 1: FAHE-1 Test Procedure:

1. The Testing Station acting as host should perform a Get Log Page for LID 0xC2. 2. The Testing Station should download 2 firmware images to the DUT. 3. The Testing Station should alternate activating the 2 firmware images 21 times using the Firmware Commit


Observable Results:

1. Verify that each Valid Firmware Activation History Entries 0-20 (Bytes 1287:8) has the Entry Version Num-ber (Byte 0) set 0x01.

Case 2: FAHE-2 Test Procedure:



Observable Results:

1. Verify that each Valid Firmware Activation History Entries 0-20 (Bytes 1287:8) has the Entry Length (Byte 1) set to 0x40 (64).




Observable Results:

1. Verify that each Valid Firmware Activation History Entries 0-20 (Bytes 1287:8) has Byte 3:2 set to 0x0.




1. The Testing Station acting as host should perform a Get Log Page for LID 0xC2. 2. The Testing Station should perform firmware activation. 3. The Testing Station acting as host should perform a Get Log Page for LID 0xC2.

Observable Results:

1. Verify that each Valid Firmware Activation History Entries (Bytes 5:4) increments in each Firmware Acti-vation History Entry.


1. The Testing Station acting as host should perform a Get Log Page for LID 0xC2. 2. The Testing Station should perform firmware activation. 3. The Testing Station acting as host should perform a Get Log Page for LID 0xC2.

Observable Results:

1. Verify that the Timestamp (Bytes 13:6) is correct in each Firmware Activation History Entry. Case 6: FAHE-6 Test Procedure:


1. Verify that Bytes 21:14 are set to 0x0 in each Firmware Activation History Entry. Case 7: FAHE-7 Test Procedure:

1. The Testing Station should perform firmware activation. 2. The Testing Station acting as host should perform a Get Log Page for LID 0xC2. 3. Power Cycle the DUT. 4. The Testing Station should perform firmware activation for a different firmware image. 5. The Testing Station acting as host should perform a Get Log Page for LID 0xC2.

Observable Results:

1. Verify that Bytes 29:22 are set to correct Power Cycle Counts in each Firmware Activation History Entry. Case 8: FAHE-8 Test Procedure:

1. The Testing Station should perform firmware activation. 2. The Testing Station acting as host should perform a Get Log Page for LID 0xC2. 3. The Testing Station should perform firmware activation for a different firmware image. 4. The Testing Station acting as host should perform a Get Log Page for LID 0xC2.

Observable Results:

1. Verify that Bytes 37:30 in the Firmware Activation History Entries in each returned Log Page are set to correct Previous Firmware values and follow the correct format as defined for the FR field in the NVMe v1.4 specification.





Observable Results:

1. Verify that Bytes 45:38 in the Firmware Activation History Entries in each returned Log Page are set to correct Current Firmware values and follow the correct format as defined for the FR field in the NVMe v1.4 specification.



Observable Results:

1. Verify that Byte 46 in the Firmware Activation History Entries in each returned Log Page are set to correct Slot Number values.



Observable Results:

1. Verify that Byte 47 in the Firmware Activation History Entries in each returned Log Page are set to correct Commit Action Types.



Observable Results:

1. Verify that Bytes 49:48 in the Firmware Activation History Entries in each returned Log Page are set to correct Results.


1. The Testing Station should perform firmware activation. 2. The Testing Station acting as host should perform a Get Log Page for LID 0xC2.



3. The Testing Station should perform firmware activation for a different firmware image. 4. The Testing Station acting as host should perform a Get Log Page for LID 0xC2.

Observable Results:

1. Verify that Bytes 63:50 in the Firmware Activation History Entries in each returned Log Page are set to 0x0. Possible Problems: None.



Test 1.18 – Firmware Update Requirements Purpose: To verify that an NVMe SSD properly follows the Firmware Update Requirements. References: [1] Cloud SSD Specification 3.8.7 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 13, 2020 Discussion: The specification defines the Firmware Update Requirements. Test Setup: See Appendix A. Case 1: FWUP-1 Test Procedure:

1. Perform all tests in 1.15 and 1.16. Observable Results:

1. Verify that the DUT passed all test cases in Test 1.15 and 1.16. Case 2: FWUP-2 Test Procedure:

1. The Testing Station should download a firmware image to the DUT. 2. Repeat 1000 times.

Observable Results:

1. Verify that each Firmware Download Command completed successfully. Case 3: FWUP-3 Test Procedure:

1. The Testing Station acting as host should perform a Get Log Page for LID 0xC2. 2. The Testing Station should download 2 firmware images to the DUT. 3. The Testing Station should perform a Firmware Commit command with Commit Actions (CA) of 000b,

001b, 010b, 011b. 4. The Testing Station acting as host should perform a Get Log Page for LID 0xC2.

Observable Results:

1. Verify that each Firmware Commit command using each Commit Action completes successfully. Case 4: FWUP-4 Test Procedure:

1. The Testing Station should download a firmware image to the DUT. Observable Results:

1. Verify that each Firmware Download Command completed successfully.



Case 5: FWUP-5 Test Procedure:

1. The Testing Station acting as host should perform a Identify Controller Data Structure, CNS=0x01. Observable Results:

1. Verify that the FWUG field in the Identify Controller Data Structure is set to 0x01. Case 6: FWUP-6 Test Procedure:

1. The Testing Station acting as host should perform a Identify Controller Data Structure, CNS=0x01. Observable Results:

1. Verify that the FRMW field Bits 3:1 in the Identify Controller Data Structure indicate that the device supports between 2 and 7 firmware slots.


1. Perform a Identify Controller Data Structure, CNS=0x01. 2. Perform a Firmware Commit with action 011b (firmware activation without reset) 3. Wait 1 second. 4. Perform Read Command 5. Perform a Identify Controller Data Structure, CNS=0x01.

Observable Results:

1. Verify that the Read operation completed successfully. 2. Verify that the MTFA field in the Identify Controller Data Structure is not greater than 0xA.


1. If possible, obtain a firmware image that is incompatible with the current version of firmware, this may vary between product vendors.

2. The Testing Station should perform Firmware Download of the incompatible image. 3. The Testing Station should perform Firmware Commit with activation of the incompatible image.

Observable Results:

1. Verify that the activation or download cannot be completed due to firmware rollback protection. Case 9: FWUP-9 Test Procedure:

1. If possible, obtain a corrupted firmware image, this may vary between product vendors. 2. The Testing Station should perform Firmware Download of the corrupted image. 3. The Testing Station should perform Firmware Commit with activation of the corrupted image. 4. Perform a Controller Level reset.

Observable Results:

1. Verify that the device is able to be enabled and CSTS.RDY is set to 1. Possible Problems: None.



Test 1.19 – De-Allocation Requirements Purpose: To verify that an NVMe SSD properly follows the De-Allocation Requirements. References: [1] Cloud SSD Specification 3.9 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 29, 2020 Discussion: The specification defines the De-allocation requirements. Test Setup: See Appendix A. Case 1: TRIM-1 Test Procedure:

1. The Testing Station should perform an Identify command to retrieve the Controller Data Structure and save the ONCS field.

Observable Results:

1. Verify that ONCS Bit 2 is set to 1 to indicate support for Dataset Management. Case 2: TRIM-2 Test Procedure:

1. The Testing Station should perform a Identify command to the DUT to retrieve the Controller Data Structure and save the TNVMCAP value.

2. The Testing Station should perform a Format NVM to the DUT using a 4096 Byte sector size. 3. Perform a Write operation to 2 different sectors using a known data pattern (i.e. AAh). 4. Perform a Read operation to the same 2 sectors to verify that the pattern was written properly. 5. Perform a Dataset Management command to Deallocate the 2 sectors read in the previous step. 6. Perform a Read operation to the same 2 sectors. 7. Repeat this procedure using a 512 Byte sector size if the DUT capacity is less than 8 TB (as indicated by

TNVMCAP). Observable Results:

1. Verify that the data read by the Read Operation following the Dataset Management command was either all 0, all 1, or unchanged from the previous Write operation for each sector.

Case 3: TRIM-3 Test Procedure:


2. The Testing Station should perform a Format NVM to the DUT using a 4096 Byte sector size. 3. Perform a Write operation to 2 different sectors using a known data pattern (i.e. AAh). 4. Perform a Read operation to the same 2 sectors to verify that the pattern was written properly. 5. Perform a Dataset Management command to Deallocate the 2 sectors read in the previous step. 6. Remove power to the DUT. Once the DUT is shutdown, restore power and allow the DUT to boot again. 7. When the DUT is ready, perform a Read operation to the same 2 sectors previously deallocated.



8. Repeat this procedure using a 512 Byte sector size if the DUT capacity is less than 8 TB (as indicated by TNVMCAP).

Observable Results:

1. Verify that the data read by the Read Operation following the Dataset Management command was either all 0, all 1, or unchanged from the previous Write operation for each sector.




Test 1.20 – Sector Size and Namespace Support Purpose: To verify that an NVMe SSD properly follows the Sector Size and Namespace Support requirements. References: [1] Cloud SSD Specification 3.10 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 29, 2020 Discussion: The specification defines the De-allocation requirements. Test Setup: See Appendix A. Case 1: SECTOR-1 Test Procedure:


2. The Testing Station should perform a Format NVM to the DUT using a 4096 Byte sector size. 3. Repeat this procedure using a 512 Byte sector size if the DUT capacity is less than 8 TB (as indicated by

TNVMCAP). Observable Results:

1. Verify that the Format NVM operations complete successfully. Case 2: SECTOR-2 Test Procedure:


2. The Testing Station should perform a Format NVM to the DUT using a 512 Byte sector size if the DUT capacity is less than 8 TB (as indicated by TNVMCAP), otherwise this test case is not applicable.

Observable Results:

1. Verify that the Format NVM operations complete successfully. Case 3: SECTOR-3 Test Procedure:

1. The Testing Station should perform a Identify command to to CNS 02h to retrieve the list of Active Namespaces.

Observable Results:

1. Verify that the DUT has only 1 Namespace. Possible Problems: None.



Test 1.21 – Set/Get Features Requirements Purpose: To verify that an NVMe SSD properly follows the Set/Get Features requirements. References: [1] Cloud SSD Specification 3.11 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 29, 2020 Discussion: The specification requires support for several vendor unique Features IDs. Test Setup: See Appendix A. Case 1: GETF-1 Test Procedure:

1. The Testing Station should perform a Get Feature Command with SEL=00b for the follow Feature IDs: a. 0xC0 – Error Injection Set b. 0xC1 – Clear Firmware Update History c. 0xC2 – Read Only/Write Through Mode d. 0xC3 – Clear PCIe Correctable Error Counters e. 0xC4 – Enable IEEE1667 Silo

2. Repeat the previous step with SEL = 01b, 10b and 11b. Observable Results:

1. Verify that all attempted Get Feature operations complete successfully. Possible Problems: None.



Test 1.22 – Error Injection Set Feature Identifier (0xC0) Purpose: To verify that an NVMe SSD properly follows the requirements for the Error Injection Set Feature Identifier. References: [1] Cloud SSD Specification 3.11.1, 3.11.1.1, 3.11.1.2, 3.11.2 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 29, 2020 Discussion: The Error Injection Set Feature is a Feature to inject one or more error conditions to be reported by the device. If multiple Set Features commands for this feature are processed, then only information from the most recent successful command is retained (i.e., subsequent commands replace information provided by previous commands). Test Setup: See Appendix A. Case 1: SERRI-1-15, ERRI-1,3-4, ERRIE-1-4, ERRIEDP-1-4, GERRI-1-2 Test Procedure:

1. The Testing Station should perform a Set Feature Command for FID=0Bh to enable Asynchronous Event Notifications.

2. The Testing Station should perform a Get Feature Command for FID=0xC0 and SEL=000b. 3. The Testing Station should perform a Set Feature Command with the following parameters:

a. CID - Dword 0 Bits 31:16 set to a valid value b. PSDT – Dword 0 Bits 15:14 set to 00b c. Reserved – Dword 0 Bits 13:10 set to 0 d. FUSE – Dword 0 Bits 9:8 set to 00b e. OPC – Dword 0 Bits 7:0 set to 09h f. NSID – Dword 1 Bits 31:0 set to 0 g. Reserved - Dword 2 and 3 Bits 31:0 set to 0 h. MPTR - Dword 4 and 5 Bits 31:0 set to 0 i. DPTR – Dword 6:9 Bits 31:0 point to a physically contiguous 4096-byte address range

containing 1 Error Injections Data Structure Entry. i. The Error Injection Data Structure Entry should have the following settings

1. Byte 0 Error Entry Flags Bit 7:2 set to 0, Bit 1 set to 1 to indicate single instance error injections, Bit 0 set to 1 to indicate Error Injection is enabled.

2. Byte 1 Reserved, set to 0 3. Byte 3:2 Error Injection Type, set to a valid value between 1h to 9h 4. Byte 31:4 Error Injection Type Specific Definition Byte 31:6 set to 0, Byte

5:4 set to a decimal value of 10 (0Ah). j. SV – Dword 10 Bit 31 set to 0b k. Reserved – Dword 10 Bits 30:8 set to 0 l. FID – Dword 10 Bits 7:0 set to C0h m. Reserved Dword 11 Bits 31:7 set to 0 n. Number of Error Injections – Dword 11 Bits 6:0 set to 1. o. Reserved – Dword 12:15 Bits 31:0 set to 0.

4. Perform 5 Read commands. 5. The Testing Station should perform a Get Feature Command for FID=0xC0 and SEL=000b. 6. Perform 5 Read commands. 7. Testing Station should wllow the DUT to transmit Asynchronous Event Notification. 8. Testing Station should read the CSTS.CFS register. 9. Repeat this procedure for all values of Error Injection Type, 1h through 9h.



Observable Results:

1. Verify that all attempted Set and Get Feature operations complete successfully. 2. Verify that the first Get Feature command returns a data buffer of all zeros. 3. Verify that the second Get Feature command returns a data buffer matching the Error Injection Data Structure

Entry transmitted by the Testing Station in the previous Set Features command. 4. Verify that the Error condition specified by the Error Injection Type field (1h through 9h) was indicated by

the DUT via the CSTS.CFS register or by Asynchronous Event Notification. Case 2: ERRI-2 Test Procedure:

1. The Testing Station should perform a Get Feature Command for FID=0xC0 and SEL=000b. 2. The Testing Station should perform a Set Feature Command with the following parameters:




2. Byte 1 Reserved, set to 0 3. Byte 3:2 Error Injection Type, set to a valid value between 1h to 9h 4. Byte 31:4 Error Injection Type Specific Definition Byte 31:6 set to 0, Byte

5:4 set to a decimal value of 10 (0Ah). j. SV – Dword 10 Bit 31 set to 0b k. Reserved – Dword 10 Bits 30:8 set to 0 l. FID – Dword 10 Bits 7:0 set to C0h m. Reserved Dword 11 Bits 31:7 set to 0 n. Number of Error Injections – Dword 11 Bits 6:0 set to 1. o. Reserved – Dword 12:15 Bits 31:0 set to 0.

3. Perform 5 Read commands. 4. The Testing Station should perform a Get Feature Command for FID=0xC0 and SEL=000b. 5. The Testing Station should perform a Set Feature Command with the following parameters:




2. Byte 1 Reserved, set to 0 3. Byte 3:2 Error Injection Type, set to a valid value between 1h to 9h



4. Byte 31:4 Error Injection Type Specific Definition Byte 31:6 set to 0, Byte 5:4 set to a decimal value of 10 (0Ah).

j. SV – Dword 10 Bit 31 set to 0b k. Reserved – Dword 10 Bits 30:8 set to 0 l. FID – Dword 10 Bits 7:0 set to C0h m. Reserved Dword 11 Bits 31:7 set to 0 n. Number of Error Injections – Dword 11 Bits 6:0 set to 0000000b. o. Reserved – Dword 12:15 Bits 31:0 set to 0.

6. The Testing Station should perform a Get Feature Command for FID=0xC0 and SEL=000b. 7. Perform 5 Read commands. 8. Testing Station should read the CSTS.CFS register.

Observable Results:

1. Verify that all attempted Set and Get Feature operations complete successfully. 2. Verify that the first Get Feature command returns a data buffer of all zeros. 3. Verify that the second Get Feature command returns a data buffer matching the Error Injection Data Structure

Entry transmitted by the Testing Station in the previous Set Features command. 4. Verify that the third Get Feature command returns a data buffer of all zeros. 5. Verify that the Error condition specified by the Error Injection Type field (1h through 9h) was not indicated

by the DUT via the CSTS.CFS register or by Asynchronous Event Notification. Possible Problems: None.



Test 1.23 – Clear Firmware Update History Set Feature Identifier (0xC1) Purpose: To verify that an NVMe SSD properly follows the requirements for FID 0xC1. References: [1] Cloud SSD Specification 3.11.3 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 29, 2020 Discussion: The specification requires support for Feature ID 0xC1. Test Setup: See Appendix A. Case 1: CFUH-1-15 Test Procedure:

1. The Testing Station acting as host should perform a Get Log Page for LID 0xC2. 2. The Testing Station should download 2 firmware images to the DUT. 3. The Testing Station should perform a Firmware Commit command with Commit Actions (CA) of 011b. 4. The Testing Station acting as host should perform a Get Log Page for LID 0xC2. 5. The Testing Station should perform a Set Feature Command with the following parameters:

a. CID - Dword 0 Bits 31:16 set to a valid value b. PSDT – Dword 0 Bits 15:14 set to 00b c. Reserved – Dword 0 Bits 13:10 set to 0 d. FUSE – Dword 0 Bits 9:8 set to 00b e. OPC – Dword 0 Bits 7:0 set to 09h f. NSID – Dword 1 Bits 31:0 set to 0 g. Reserved - Dword 2 and 3 Bits 31:0 set to 0 h. MPTR - Dword 4 and 5 Bits 31:0 set to 0 i. DPTR – Dword 6:9 Bits 31:0 set to 0 j. SV – Dword 10 Bit 31 set to 0b k. Reserved – Dword 10 Bits 30:8 set to 0 l. FID – Dword 10 Bits 7:0 set to C1h m. Clear Firmware Update History Log Dword 11 Bit 31 set to 1b n. Reserved – Dword 11 Bits 30:0 set to 0 o. Reserved – Dword 12:15 Bits 31:0 set to 0


1. Verify that the Set Feature command completed successfully. 2. Verify that the second Get Log Page Operation returned a Firmware activation History Log Page that in-

cluded the Firmware Activations in Step 3. 3. Verify that the third Get Log Page Operation returned a cleared Firmware activation History Log Page.




Test 1.24 – Read Only/Write Through Mode Set Feature Identifier (0xC2) Purpose: To verify that an NVMe SSD properly follows the requirements for FID 0xC2. References: [1] Cloud SSD Specification 3.11.4 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 29, 2020 Discussion: The specification requires support for Feature ID 0xC2. Test Setup: See Appendix A. Case 1: ROWTM-1 Test Procedure:

1. The Testing Station acting as host should perform a Get Feature for FID 0xC2 with SEL=01b (Default state) Observable Results:

1. Verify that the Get Feature command completed successfully and the DUT returned completion queue entry with Dword 0 Bits 2:0 indicated a value of 001b, that the Read Only Mode is the factory default.

Case 2: SROWTM-1-15 Test Procedure:

1. The Testing Station acting as host should perform a Get Feature for FID 0xC2. 2. The Testing Station should perform a Set Feature Command with the following parameters:

a. CID - Dword 0 Bits 31:16 set to a valid value b. PSDT – Dword 0 Bits 15:14 set to 00b c. Reserved – Dword 0 Bits 13:10 set to 0 d. FUSE – Dword 0 Bits 9:8 set to 00b e. OPC – Dword 0 Bits 7:0 set to 09h f. NSID – Dword 1 Bits 31:0 set to 0 g. Reserved - Dword 2 and 3 Bits 31:0 set to 0 h. MPTR - Dword 4 and 5 Bits 31:0 set to 0 i. DPTR – Dword 6:9 Bits 31:0 set to 0 j. SV – Dword 10 Bit 31 set to 1b k. Reserved – Dword 10 Bits 30:8 set to 0 l. FID – Dword 10 Bits 7:0 set to C2h m. End of Life Behavior Dword 11 Bits 31:30 set to 10b (Write Through Mode) n. Reserved – Dword 11 Bits 29:0 set to 0 o. Reserved – Dword 12:15 Bits 31:0 set to 0

3. The Testing Station acting as host should perform a Get Feature for FID 0xC2 with SEL=00b (Current State). 4. The Testing Station should perform a Set Feature Command with the same parameters as the previously sent

Set Feature command except End of Life Behavior, Dword 11 Bits 31:30, should be set to 01b (Read Only Mode)

5. The Testing Station acting as host should perform a Get Feature for FID 0xC2 with SEL=00b (Current State). Observable Results:

1. Verify that the Set Feature command completed successfully.



2. Verify that the second Get Feature operation for FID 0xC2 indicates that the Current State of End of Life Behavior is Write Through Mode, by setting Completion Queue Entry Dword 0 Bits 2:0 = 010b.

3. Verify that the second Get Feature operation for FID 0xC2 indicates that the Current State of End of Life Behavior is Read Only Mode, by setting Completion Queue Entry Dword 0 Bits 2:0 = 001b.




Test 1.25 – Read Only/Write Through Mode Get Feature Identifier (0xC2) Purpose: To verify that an NVMe SSD properly follows the requirements for FID 0xC2. References: [1] Cloud SSD Specification 3.11.5 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 29, 2020 Discussion: The specification requires support for Feature ID 0xC2. Test Setup: See Appendix A. Case 1: GROWTM-1-2 Test Procedure:

1. The Testing Station acting as host should perform a Get Feature for FID 0xC2 with SEL=00b (Current State). 2. The Testing Station acting as host should perform a Get Feature for FID 0xC2 with SEL=01b (Default State). 3. The Testing Station acting as host should perform a Get Feature for FID 0xC2 with SEL=10b (Saved State). 4. The Testing Station acting as host should perform a Get Feature for FID 0xC2 with SEL=11b (Capabilities).

Observable Results:

1. Verify that each Get Feature command completed successfully. 2. Verify that for the first, second, and third Get Feature operations, the DUT sets Completion Queue Entry

Dword 0 Bits 2:0 = 001b to indicate the Current State, Default State, and Saved State respectively. 3. Verify that for the fourth Get Feature operation, the DUT sets Completion Queue Entry Dword 0 Bits 2:0 =

101b to indicate the feature is saveable, changeable, and not namespace specific.




Test 1.26 – Clear PCIe Correctable Error Counters Set Feature Identifier (0xC3) Purpose: To verify that an NVMe SSD properly follows the requirements for FID 0xC3. References: [1] Cloud SSD Specification 3.11.6 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 29, 2020 Discussion: The specification requires support for Feature ID 0xC3. Test Setup: See Appendix A. Case 1: CPCIE-1-15 Test Procedure:

1. The Testing Station acting as host should perform a Get Log Page operation for LID 0xCA. 2. The Testing Station should perform a Set Feature Command with the following parameters:

a. CID - Dword 0 Bits 31:16 set to a valid value b. PSDT – Dword 0 Bits 15:14 set to 00b c. Reserved – Dword 0 Bits 13:10 set to 0 d. FUSE – Dword 0 Bits 9:8 set to 00b e. OPC – Dword 0 Bits 7:0 set to 09h f. NSID – Dword 1 Bits 31:0 set to 0 g. Reserved - Dword 2 and 3 Bits 31:0 set to 0 h. MPTR - Dword 4 and 5 Bits 31:0 set to 0 i. DPTR – Dword 6:9 Bits 31:0 set to 0 j. SV – Dword 10 Bit 31 set to 0b k. Reserved – Dword 10 Bits 30:8 set to 0 l. FID – Dword 10 Bits 7:0 set to C3h m. Clear PCIe Error Counters Dword 11 Bit 31 set to 1b n. Reserved – Dword 11 Bits 30:0 set to 0 o. Reserved – Dword 12:15 Bits 31:0 set to 0

3. The Testing Station acting as host should perform a Get Log Page operation for LID 0xCA. Observable Results:

1. Verify that the Set Feature command completed successfully. 2. Verify that the second Get Log Page operation for LID 0xCA showed that all PCIe Correctable Error Coun-

ters had been cleared to 0. If no errors where indicated in the Log Page returned in response to the first Get Log Page operation it cannot be determined if the counters were cleared or not.




Test 1.27 – Enable IEEE1667 Silo Set Feature Identifier (0xC4) Purpose: To verify that an NVMe SSD properly follows the requirements for FID 0xC4. References: [1] Cloud SSD Specification 3.11.7 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 29, 2020 Discussion: The specification requires support for Feature ID 0xC4. Test Setup: See Appendix A. Case 1: S1667-1-15 Test Procedure:

1. The Testing Station should perform a Set Feature Command with the following parameters: a. CID - Dword 0 Bits 31:16 set to a valid value b. PSDT – Dword 0 Bits 15:14 set to 00b c. Reserved – Dword 0 Bits 13:10 set to 0 d. FUSE – Dword 0 Bits 9:8 set to 00b e. OPC – Dword 0 Bits 7:0 set to 09h f. NSID – Dword 1 Bits 31:0 set to 0 g. Reserved - Dword 2 and 3 Bits 31:0 set to 0 h. MPTR - Dword 4 and 5 Bits 31:0 set to 0 i. DPTR – Dword 6:9 Bits 31:0 set to 0 j. SV – Dword 10 Bit 31 set to 1b k. Reserved – Dword 10 Bits 30:8 set to 0 l. FID – Dword 10 Bits 7:0 set to C4h m. Enable IEEE1667 Silo Dword 11 Bit 31 set to 0b (Disabled) n. Reserved – Dword 11 Bits 30:0 set to 0 o. Reserved – Dword 12:15 Bits 31:0 set to 0

2. The Testing Station acting as host should perform a Get Feature for FID 0xC4 with SEL=00b (Current State). 3. The Testing Station should perform a Set Feature Command with the same parameters as the previously sent

Set Feature command except Enable IEEE1667 Silo, Dword 11 Bit 31, should be set to 1b (Enabled) 4. The Testing Station acting as host should perform a Get Feature for FID 0xC4 with SEL=00b (Current State). 5. The Testing Station should perform a Set Feature Command with the same parameters as the previously sent

Set Feature command except Enable IEEE1667 Silo, Dword 11 Bit 31, should be set to 0b (Disabled) 6. The Testing Station acting as host should perform a Get Feature for FID 0xC4 with SEL=00b (Current State).

Observable Results:

1. Verify that each Set Feature and Get Feature command completed successfully. 2. Verify that each Get Feature command returned a value in Dword 0 Bits 2:0 of the completion queue entry

indicating that the Current State of the Enable IEEE1667 value matched what was set in the previous Set Feature command.




Test 1.28 – Enable IEEE1667 Silo Get Feature Identifier (0xC4) Purpose: To verify that an NVMe SSD properly follows the requirements for FID 0xC4. References: [1] Cloud SSD Specification 3.11.8 Resource Requirements: Tools capable of monitoring and decoding NVMe traffic. Last Modification: July 29, 2020 Discussion: The specification requires support for Feature ID 0xC4. Test Setup: See Appendix A. Case 1: G1667-1-2 Test Procedure:

1. The Testing Station should perform a Set Feature Command with the following parameters: a. CID - Dword 0 Bits 31:16 set to a valid value b. PSDT – Dword 0 Bits 15:14 set to 00b c. Reserved – Dword 0 Bits 13:10 set to 0 d. FUSE – Dword 0 Bits 9:8 set to 00b e. OPC – Dword 0 Bits 7:0 set to 09h f. NSID – Dword 1 Bits 31:0 set to 0 g. Reserved - Dword 2 and 3 Bits 31:0 set to 0 h. MPTR - Dword 4 and 5 Bits 31:0 set to 0 i. DPTR – Dword 6:9 Bits 31:0 set to 0 j. SV – Dword 10 Bit 31 set to 1b k. Reserved – Dword 10 Bits 30:8 set to 0 l. FID – Dword 10 Bits 7:0 set to C4h m. Enable IEEE1667 Silo Dword 11 Bit 31 set to 0b (Disabled) n. Reserved – Dword 11 Bits 30:0 set to 0 o. Reserved – Dword 12:15 Bits 31:0 set to 0

2. The Testing Station acting as host should perform a Get Feature for FID 0xC4 with SEL=00b (Current State). 3. The Testing Station acting as host should perform a Get Feature for FID 0xC4 with SEL=01b (Default State). 4. The Testing Station acting as host should perform a Get Feature for FID 0xC4 with SEL=10b (Saved State). 5. The Testing Station acting as host should perform a Get Feature for FID 0xC4 with SEL=11b (Capabilities).

Observable Results:

1. Verify that each Set Feature and Get Feature command completed successfully. 2. Verify that the Get Feature command returned a value in Dword 0 Bits 2:0 of the completion queue entry

indicating that the Current State of the Enable IEEE1667 value was 000b (disabled). 3. Verify that the Get Feature command returned a value in Dword 0 Bits 2:0 of the completion queue entry

indicating that the Default State of the Enable IEEE1667 value was 000b (disabled) or 001b (enabled). 4. Verify that the Get Feature command returned a value in Dword 0 Bits 2:0 of the completion queue entry

indicating that the Saved State of the Enable IEEE1667 value was 000b (disabled). 5. Verify that the Get Feature command returned a value in Dword 0 Bits 2:0 of the completion queue entry

indicating that the Capabilities of the Enable IEEE1667 value was 101b (saveable, changeable, not namespace specific).




Group 2: PCIe Requirements

Overview:





Group 3: Reliability Overview:





Group 4: Endurance Overview:





Group 5: Thermal Overview:





Group 6: Form Factor Requirements Overview:





Group 7: SMBus Support Overview:





Group 8: Security

Overview:





Group 9: Labeling

Overview:





Group 10: Compliance

Overview:





Group 11: Shock and Vibration

Overview:





Group 12: NVMe Linux CLI Plug-In Requirements

Overview:





Appendix A: TEST SETUP The diagram below outlines a simple test setup for testing NVMe products for conformance to the tests in this docu-ment.



Appendix B: NOTES ON TEST PROCEDURES There are scenarios where in test procedures it is desirable to leave certain aspects of the testing procedure as general as possible. In these cases, the steps in the described test procedure may use placeholder values, or may intentionally use non–specific terminology, and the final determination of interpretation or choice of values is left to the discretion of the test technician. The following is an attempt to capture and describe all such instances used throughout the procedures.

Ports on Testing Station and Device Under Test

In general, any NVMe capable Port on the Testing Station or Device Under Test may be used as an interface with a test station or interoperability partner. There is assumed to be no difference in behavior, with respect to the protocols involved in this test suite, between any two NVMe ports on the Testing Station or Device Under Test. Hence, actual ports used may be chosen for convenience. However, it is recommended that the port used in the test configuration is recorded by the test technician.

Use of “various” To maintain generality, some steps will specify that “various other values” (or the like) should be used in place of a given parameter. Ideally, all possible values would be tested in this case. However, limits on available time may constrain the ability of the test technician to attempt this. Given this, a subset of the set of applicable values must generally be used. When deciding how many values should be used, it should be noted that the more values that are tested, the greater the confidence of the results obtained (although there is a diminishing return on this). When deciding which specific values to use, it is generally recommended to choose them at pseudo–randomly yet deterministically. However, if there exist subsets of the applicable values with special significance, values from each subset should be attempted.

UNH–IOL NVMe Testing Service · 2020-11-02 · University of New Hampshire InterOperability...

Documents

Transcript of UNH–IOL NVMe Testing Service · 2020-11-02 · University of New Hampshire InterOperability...