Best Practices When Deploying Microsoft Windows Server 2008 R2 or Microsoft Windows Server 2008 SP2 on HP ProLiant DL980 G7 Servers

Technical white paper

Table of contents

Introduction .................................................................................................................................. 2

OS Support Matrix for the HP ProLiant DL980 G7 Server .................................................................. 3

Hyper-V Support Matrix ................................................................................................................. 4 Hyper-threading + Windows Server 2008 R2 + Hyper-V version 2 ................................................. 4

Additional Software Components Recommended When Running Windows on the DL980 ..................... 5 HP Smart Update CD ................................................................................................................. 5 HP ProLiant DL980 System Providers ........................................................................................... 6 HP Enterprise SQL Optimizer (HP ESO) ..................................................................................... 14

Hardware Best Practices for the DL980 When Running Windows ..................................................... 15 Boot Adapter Location ............................................................................................................. 15 Power Consumption Settings .................................................................................................... 15

Windows Server 2008 R2 Best Practices for the DL980 .................................................................. 17 Enable Hyper-threading to Maximize Resource Usage for Large Applications (but NOT for Hyper-V or Windows 2008 SP2 with 8 sockets populated) .................................. 17 Hyper-threading + Windows Server 2008 R2 + Hyper-V version 2 ............................................... 18 Enable 44-bit Mode after OS Installation ................................................................................... 18 Enable x2APIC Mode for Systems with 8 Intel® Xeon E7-family Processors Running Windows Server 2008 R2 (with or without SP1) .................................................................... 20 Install Microsoft Hotfix on Systems with More Than 1 TB RAM ...................................................... 22 Workaround When Enabling Hyper-V on Systems with More Than 512 GB RAM ........................... 22

Windows Server 2008 SP2 Best Practices for the DL980 ................................................................. 25 Disable Hyper-threading to Keep Logical Processor Count at 64 Maximum .................................... 25 Hyper-threading + Windows Server 2008 SP2 + Hyper-V version 1 ............................................. 27 Workaround for Possible Driver Conflict When Logging in Using iLO ............................................ 27

For More Information .................................................................................................................. 28 Documentation Feedback ......................................................................................................... 28

2

Introduction This purpose of this white paper is to facilitate the successful deployment and configuration of Microsoft®

Windows® Server 2008 R2 and Windows Server 2008 Service Pack 2 (SP2) operating systems on HP ProLiant DL980 G7 servers. It includes a description of additional recommended HP software components for the DL980, and best practices you can use when deploying Microsoft Windows on the DL980. Topics include:

• Operating System support matrix • Hyper-V support matrix • Additional software components recommended • Hardware best practices • Windows Server 2008 R2 best practices • Windows Server 2008 SP2 best practices

The HP ProLiant DL980 G7 server is HP’s latest 8 socket, x86-based, 64-bit scale up platform. Available in 4-, 6-, 8-, and now 10-core configurations, it can support up to 160 logical processors (with 8 Intel® Xeon E7-family processors x 10 cores x 2 threads/core, with Intel’s hyper-threading enabled), 4 TB of physical memory (RAM), and up to 16 PCIe/PCI-X I/O cards. In addition, the DL980 utilizes Non-Uniform Memory Architecture (NUMA) and Intel QuickPath Interconnect (QPI) technology.

For a general overview of the technical specifications for this server, see the HP ProLiant DL980 G7 Server Data Sheet at: http://h20195.www2.hp.com/V2/GetPDF.aspx/4AA1-5671ENW.pdf

For detailed technical specifications, see the server QuickSpecs at: http://h18000.www1.hp.com/products/quickspecs/13708_div/13708_div.pdf

The DL980 uses HP PREMA architecture which incorporates a new node controller design with Smart CPU caching and redundant system fabric. Combined with the Microsoft Windows Operating System, these features provide a solution that is fully capable of supporting the most demanding, data-intensive workloads, with the reliability, availability, and efficiency needed to run all of your business-critical applications with confidence.

For a detailed discussion of the HP PREMA architecture, see the technical overview at: http://h20195.www2.hp.com/V2/GetPDF.aspx/4AA3-0643ENW.pdf

3

Figure 1: Front view of the DL980 G7 server

OS Support Matrix for the HP ProLiant DL980 G7 Server To run Windows and Windows applications optimally, you must understand how to leverage the capabilities and resources of the DL980 server. A good starting point is to review the following OS Support Matrix.

HP ProLiant DL980 G7 Windows Operating System Support Matrix

Operating System

Maximum CPUs

Supported by OS

Maximum Memory (RAM)

x2APIC Support

Notes

Microsoft Windows Server 2008 R2 Enterprise Edition

8 Sockets 256 Logical Processors

2 TB Yes Service Pack 1 (SP1) supported.

Microsoft Windows Server 2008 R2 Data Center Edition

64 Sockets 256 Logical Processors

2 TB Yes Service Pack 1 (SP1) supported.

Microsoft Windows Server 2008 Enterprise x64 Edition

8 Sockets 64 Logical Processors

1 TB No Requires Service Pack 2 (SP2) or greater.

Microsoft Windows Server 2008 Data Center x64 Edition

64 Sockets 64 Logical Processors

1 TB No Requires Service Pack 2 (SP2) or greater.

Though the HP ProLiant DL980 G7 server can support up to 4 TB of physical memory with 32 GB dimms, the current maximum memory supported for each Window OS is as listed in the matrix above.

This matrix shows that Windows Server 2008 R2 is necessary to make full use of the DL980 G7’s maximum processors and memory. Note that Hyper-V is also fully supported in both operating systems on the DL980 G7. More information on Hyper-V is found in the next section.

Intel x2APIC interrupt architecture was designed to meet interrupt address scaling needs above 128 logical processors (the previous limit of xAPIC architecture). It provides better performance in the delivery and handling of interrupts, both in hardware and software, and is only supported in Windows Server 2008 R2.

4

For configuration with 8 Intel® Xeon® E7-family processors, if x2APIC is used, it must be enabled through BIOS. For more information, see the Enable x2APIC Mode section (on page 20).

Testing with Windows indicates that x2APIC resolves some of the high contention issues in kernel data structures, when compared with xAPIC support.

Hyper-V Support Matrix Hyper-V is a Microsoft technology that enables the partitioning of a single physical machine into multiple virtual machines (also called logical servers). Based on the hypervisor architecture, Hyper-V allows the creation and management of virtual machines, each one isolated from the others, running the same or different operating systems.

Hyper-V is supported on the ProLiant DL980 G7 server running either Windows Server 2008 R2 or Windows Server 2008 SP2. It was first made available as a download for Windows Server 2008 (known as Hyper-V version 1). A more feature-rich version was released later with Windows Server 2008 R2 (known as Hyper-V version 2). The following table illustrates the major differences between these two versions:

Hyper-V: Version 1 versus Version 2 Hyper-V Feature Version 1 Version 2 Maximum CPUs/Host 24 Logical 64 Logical Maximum Memory/Host 1 TB 1 TB Maximum Guests 192 384 Maximum Virtual CPUs/Guest 4 4 Maximum Virtual Memory/Guest 64 GB 64 GB Maximum Virtual CPUs/Host 192 512 For specific recommendations regarding the use of Hyper-V version 1 and version 2 in combination with hyper-threading, see the following sections:

Hyper-threading + Windows Server 2008 R2 + Hyper-V version 2 • Hyper-threading + Windows Server 2008 R2 + Hyper-V version 2 (on page 18) • Hyper-threading + Windows Server 2008 SP2 + Hyper-V version 1 (on page 27)

In addition, Hyper-V version 2 includes several new features such as live migration, storage hot addition and removal, jumbo frame and TCP chimney, and CPU core parking. For a complete summary of this newer version, see the following website: http://www.microsoft.com/windowsserver2008/en/us/hyperv-r2.aspx

Released in February 2011, Service Pack 1 (SP1) for Windows Server 2008 R2 also adds significant features for Hyper-V, like Dynamic Memory support. With Dynamic Memory, instead of setting a fixed memory amount, you can allocate a range of memory with fixed upper and lower limits, and let the guest operating system inform Hypervisor how much memory it really needs. This allows you to run more virtual machines and consume your memory resources much more efficiently.

The Windows Server 2008 R2 SP1 update is available directly from Microsoft, here: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=C3202CE6-4056-4059-8A1B-3A9B77CDFDDA.

5

Additional Software Components Recommended When Running Windows on the DL980 In addition to the software found on the HP Smart Start CD (with ProLiant Support Pack v8.60 or later for Windows Server 2008 SP2 and Windows Server 2008 R2, or v8.70 or later for Windows Server 2008 R2 SP1), HP provides additional components for running Windows on the HP ProLiant DL980 G7 that ensure optimal performance, availability, and manageability of the solution. These two additional components are the HP Smart Update CD and the HP ProLiant DL980 System Providers for Windows.

To download these components, along with other drivers and software, go to the HP DL980 G7 Support web page: http://www.hp.com/support/dl980g7. From this page, select either “Microsoft Windows Server 2008 R2” or “Microsoft Windows Server 2008 x64”. Then look in the section marked “Software – CD-ROM” for the HP Smart Update package, or in the section marked “Software – System Management” for the HP System Providers package.

Note It is critical that you download and install the HP Smart Update CD and the HP ProLiant DL980 System Providers for Windows on the DL980 for optimal performance and functionality.

The Smart Update CD v9.2 includes the HP ProLiant DL980 System Providers v9.0.5 and HP Enterprise SQL Optimizer 2.0.

HP Smart Update CD One of the challenges when administering servers in mission critical environments is the sourcing and installation of important system updates. The HP Smart Update CD provides a comprehensive source for operating system and security updates, created by Microsoft and recommended by HP, all of which are thoroughly tested to ensure their compatibility on the DL980. Distributing updates in this way offers the following advantages for the administering IT professional:

• It provides a single source for all regularly published, tested fixes • It is the quickest way to update the system, minimizing down time • It is the only way to update systems that are not connected to the Internet

Also known as hotfixes, these updates are grouped into 3 categories in the Smart Update package: Operating System, Security, and Clustering. Each category provides a collection of self-extracting, self-installing system updates. HP recommends that you regularly update your system with the most recent Operating System and Security updates. However, you only need to install the Clustering updates if your server will function as a node in a Microsoft Failover Cluster. Updates are provided for Windows Server 2008 R2, Windows Server 2008 R2 SP1, and Windows Server 2008 SP2, in both English and Japanese.

To download the latest ISO image of the Smart Update CD, go here: http://www.hp.com/support/DL980G7, select the Windows Operating System version , locate the “Software - CD-ROM” section on this page, and then click the Download button next to “Smart Update QFE CD for x64”.

To install the updates, follow these steps:

1. Download the Smart Update ISO package and burn it to CD.

6

2. Insert the CD locally (or use iLO3 Virtual media), on the target server, or remotely using an iLO3 Integrated Remote Console.

3. Select the desired OS (Windows Server 2008 SP2, Windows Server 2008 R2, or Windows Server 2008 R2 SP1).

4. Navigate to the desired category (Operating System, for example) using the links on the left side of the page.

5. Click the Install button and the HP Setup screen displays.

6. Click Install again and a command window displays, listing each update as it installs. These can be large updates, taking several minutes each.

7. When prompted, click Reboot to complete the installation for that category. You must reboot after each category finishes, to ensure correct updating of the system registry.

8. Repeat to install the Security updates (highly recommended). 9. Repeat again to install the Clustering updates (if your server will function as a cluster node).

HP ProLiant DL980 System Providers On DL980s running Windows Server 2008 and Windows Server 2008 R2, the HP DL980 System Providers ensure optimal system performance by verifying that the system memory physical addressing mode, x2APIC mode, and processor hyper-threading settings are correct. They also furnish a convenient way of enumerating processors by matching Windows logical processors to physical processors, and by describing cores and threads (when hyper-threading is turned ON). System settings and processor enumeration information is reported to the HP System Management Homepage (HP SMH). HP SMH is a web-based interface that simplifies single-system management of HP servers running Microsoft Windows, HP-UX, and Linux operating systems.

In System Providers release 9.0.5, a click button is provided on SMH for optimizing system configuration settings, including HP Power Regulator Mode. Once the optimization is completed, a system cold boot is required for changes to become effective. Release 9.0.5 also adds a new property page on SMH listing the recommended OS Quick Fix Engineering (QFEs) of the Smart Update CD v9.x, along with a status icon next to each QFE, indicating whether the QFE is installed on the system. Last but not least, a system reporting tool is included that inventories hardware for the entire system and outputs the results to a text file.

System Memory Physical Addressing Mode The default RBSU (ROM-Based Setup Utility) method for addressing memory on the DL980 G7 is 40-bit, and this setting is perfect for systems running Windows Server 2008. However, the recommended setting for Windows Server 2008 R2 is 44-bit. Therefore, Windows Server 2008 R2 users must change this setting to 44-bit in order to make full use of the available address space and to reduce data faults that can occur if the less-optimum 40-bit setting is used. The procedure for making this change is described in the section below (Enable 44-bit Mode after OS Installation, page 18).

Intel Hyper-Threading Mode Windows Server 2008 SP2 and Windows Hyper-V version 2 only support up to 64 logical processors, therefore hyper-threading should be disabled if the system has 8 sockets populated with 8 or more cores/processor under these operating environments. The procedure for making this change is described in the section, Disable Hyper-threading to Keep Logical Processor Count at 64 Maximum (on page 25). However, if the system has 4 sockets populated, then hyper-threading could be enabled to take full advantage of all 64 logical processors, as long as the application benefits from the additional logical processors.

Click the Download button on this page to download the HP ProLiant DL980 System Providers. Also on this page are the installation instructions and release notes for the System Providers. http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&cc=us&prodTypeId=153

7

51&prodSeriesId=4231377&prodNameId=4268505&swEnvOID=4064&swLang=8&mode=2&taskId=135&swItem=MTX-d639d92e975b42f4951c2a94b9

.

Once installed, the DL980 System Providers detect the current memory addressing mode and, if the non-optimum 40-bit mode is found under Windows Server 2008 R2, report it to the Windows System log (see Figure 2). Similarly, if the DL980 System Providers detect that processor hyper-threading is enabled under Windows Server 2008 SP2 or Windows Hyper-V version 2, it also reports this non-optimum setting to the Windows System Log (see Figure 2A).

Figure 2: Windows System log screen with 40-bit mode warning (Windows Server 2008 R2)

Figure 2A: Windows System log screen with hyper-threading warning (Windows 2008 SP2 or Windows Hyper-V version 2)

In general, the following hyper-threading settings are recommended for optimal performance:

• Enabled – for systems running Windows Server 2008 R2 or Windows Server 2008 R2 SP1, if the application supports more than 64 processors, and if the application benefits from the additional logical processors

• Disabled – for systems running Windows 2008 SP2, or Windows Server 2008 R2 Hyper-V, when there are more than 64 logical processors

Performance testing may indicate a zero performance gain when enabling hyper-threading, and in some cases, even degraded performance. HP always recommends that you test your workloads with hyper-threading both ON and OFF. Whether or not the workload or custom application supports more than 64 processors, it is always good practice to test and benchmark these configurations. Software licensing may

8

dictate that hyper-threading and/or some cores are disabled. Also, you can ignore the hyper-threading alert and Degraded State notice if the server workload does not benefit from hyper-threading.

Note The hyper-threading state displayed by SMH reflects the actual processor hyper-threading state of the system. It may differ from the RBSU setting for the Intel® Hyper-Threading option, because the “Optimize Logical Processor Configuration” button may have optimized the logical processors, including their hyper-threading state.

X2APIC Mode The following values are recommended for optimal performance:

• Enabled – for systems with 8 Intel® Xeon® E7-family processors running Windows Server 2008 R2, or Windows Server 2008 R2 SP1

• Disabled – for systems which do NOT have 8 Intel® Xeon® E7-family processors running Windows Server 2008 R2, or Windows Server 2008 R2 SP1. In particular, x2APIC Mode must be disabled on systems running Windows Server 2008 SP2, or Windows Server 2008 R2 Hyper-V; otherwise, these systems will not boot.

Note On systems running Windows Server 2008 R2, there are several hotfixes that must be downloaded and installed before enabling x2APIC mode. You can find these hotfixes on the Microsoft Support website, or on the HP Smart Update CD, v7.1 or later. They are not required on systems running Windows 2008 R2 SP1.

See the “Enable x2APIC Mode for Systems with 8 Intel® Xeon® E7-family Processors Running Windows Server 2008 R2 (with or without SP1)” section (on page 20) for a more detailed discussion.

Verifying Installation of the ProLiant DL980 System Providers You can confirm the presence of the DL980 System Providers on your systems either by looking at the Windows Services window for the HP ProLiant DL980 System Providers entry (see Figure 3), or by looking at the Start All Programs shortcut (see Figure 3A).

Figure 3: Verifying System Providers installation (Services window)

9

Figure 3A: Start All Programs shortcut showing presence of the DL980 System Providers

Another way to confirm positive installation is to check the Windows System log for the following event (see Figure 4).

Figure 4: Verifying System Providers installation (System log)

To confirm the System Providers v9.0.5 are functional after installation, using HP’s System Management Homepage (SMH) software, follow these steps:

1. From the target server, or from a system with a network connection to the target server, open a web browser window (the firewalls may need to be turned off for remote connection).

2. Enter the following URL into the browser’s address bar: https://machine:2381/, where machine is the IP address or computer name of the target server. If it is the first time using this link, a Security Alert dialog box displays and asks if you want to trust this server. Click Yes to continue, and the Sign In page displays.

3. Enter your user name and password, and click Sign In to access SMH. The SMH Main page displays

4. In the Advanced Server Health Information section, click Advanced Server Settings.

10

5. Locate the System Address Mode, Hyper-Threading State, x2APIC Mode settings, and HP Power Regulator in the Advanced Server Settings window. The current System Address Mode (44-bit/40-bit), Hyper-Threading State (Enabled/Disabled), x2APIC Mode (Enabled/Disabled), and HP Power Regulator Mode (OS Control Mode/HP Dynamic Power Savings Mode/HP Static Low Power Mode/HP Static High Performance Mode) values are indicated there (see Figure 5A, Figure 5B and Figure 5C).

6. In the Advanced Server Health Information section, Operating System QFEs. A list of QFEs on the Smart Update CD is reported (see Figure 6A and Figure 6B).

Figure 5A: Advanced Server Settings window − 44-bit enabled, Hyper-threading enabled, x2APIC enabled (Windows Server 2008 R2), HP Power Regulator Mode set to OS Control Mode, Logical Processor Configuration setting is optimal

Figure 5B: SMH Advanced Server Settings window − 40-bit enabled, Hyper-threading disabled, x2APIC disabled, HP Power Regulator Mode set to OS Control Mode, Logical Processor Configuration setting is optimal (Windows 2008 SP2 on an 8-processor system)

11

Figure 5C: SMH Advanced Server Settings window − 44-bit enabled, Hyper-threading disabled, x2APIC disabled, HP Power Regulator Mode set to OS Control Mode, Logical Processor Configuration setting is optimal (Windows 2008 R2 Hyper-V on an 8-processor system)

Figure 6A: SMH Operating System QFEs page (Windows 2008 R2 and R2 SP1)

12

Figure 6B: SMH Operating System QFEs page (Windows 2008 SP2)

Logical Processor to Physical Hardware Enumeration The DL980 System Providers also display logical processor information in HP SMH, in the “Advanced Server Health” section (see Figure 7). Information includes processor group number (only in Windows 2008 R2 since it groups processors to allow support of more than 64; for group processor information see http://msdn.microsoft.com/en-us/library/dd405503%28VS.85%29.aspx), NUMA node number, APIC ID, and the physical socket location of each logical processor active in the system.

13

Figure 7: Logical Processor Information page displayed in SMH

If the system determines that the logical processor configuration is not optimal, the Optimize System Configuration button displays in the lower left corner of the Logical Processor Information page (see Figure 7). Press this button to optimize the logical processors and better utilize system resources. A system cold boot is required before the changes take effect.

Note • The Optimize System Configuration button is also available on the

Advanced Server Settings page if there is non-optimal setting on that page. • Clicking the Optimize System Configuration button on either Advanced

Server Settings page or Logical Processor Information page would change non-optimal settings and configuration related to both pages.

• The Disable System Optimization Alert button is intended for customers who must leave their system configuration as is due to known limitations or licensing issues. This button disables the alerts normally generated by non-optimal system configurations. One alert not being ignored is the missing UsePhysicalDestination setting on systems with x2APIC enabled. The status icon displayed is informational.

• After the optimization is completed, the Remove Optimization button will be available to remove the most recent optimized changes and to restore the previously saved configuration.

System Reporting Tool The DL980 System Providers also install a command-line tool named SysReportWMI.exe in the \Program Files\MCWBEM\Tools directory. This tool generates a text file containing a full inventory of the system hardware. Since most of the data captured by this tool are provided by the

14

HP Insight WBEM Providers, the WBEM Providers must also be installed prior to running SysReportWMI.exe.

HP Enterprise SQL Optimizer (HP ESO) HP Enterprise SQL Optimizer (HP ESO) is a software tool that greatly simplifies and consolidates the process of tuning HP ProLiant DL980 G7 servers running Microsoft Windows Server and SQL Server 2008 or SQL Server 2008 R2. HP ESO encapsulates the expert knowledge of HP and other industry leaders in a single, multi-faceted application that is ideal for optimizing SQL Server on your DL980. HP ESO is provided free of charge and can be installed using any HP ProLiant DL980 Smart Update CD, version 9.0 or later.

Benefits of using HP ESO include:

• A drastic reduction in the time and complexity needed to configure and tune SQL Server on the DL980

• An easy to use interface that makes recommendations you can either accept in full, or pick and choose those which best suit your needs

• A one-stop configuration utility that eliminates the need for multiple tools • A means of ensuring that available system resources are allocated not just appropriately,

but optimally • Fewer support calls due to misconfigured systems

After installing HP ESO, users should run a data collection process that scans the entire solution (server, OS, SQL Server, storage, and network infrastructure). HP ESO then evaluates the results of its audit and makes configuration recommendations spanning the entire solution, including the following areas:

• SQL Server instance parameters like MAXDOP, CPU Affinity, Lightweight Pooling, and Min/Max Memory

• Storage settings Database File/Log/TempDB locations, RAID Levels, Number of Drives, and Drive Sizes

• Network settings Network Ports, NUMA/Interrupt CPU Affinities • Operating System options such as Power Management • IO Card Configuration and Placement for optimal PCI card placement based on I/O

slot capability • Software version information

HP ESO uses built-in WBEM-based instrumentation for future integration with other management tools such as HP System Insight Manager. It also provides reporting pages that provide you with graphical displays of various data collection parameters, for use in analysis and interpretation, and the ability to export these results to a file. HP ESO can save your current configuration at any time, and with its ability to roll back to previously saved configurations, you can retrieve and apply those configurations later, if necessary.

For a complete description of HP ESO features, along with instructions for installing, configuring, and using this powerful tool, refer to the HP ESO User Guide on this website (go to the “User Guide” section): http://www.hp.com/go/proliant-DL980-docs.

HP ESO is available as a self-installing, self-extracting system update on the HP Smart Update CD. To download the latest ISO image of this CD, go here: http://www.hp.com/support/DL980G7. Select the Windows Operating System version, locate the “Software - CD-ROM” section on this page, and then click the Download button next to “Smart Update QFE CD for x64”.

15

Hardware Best Practices for the DL980 When Running Windows This section describes hardware recommendations to follow when running Windows on the DL980. These apply to both Windows Server 2008 R2 and Windows Server 2008 SP2.

Boot Adapter Location On the DL980 G7, slots 7 through 11 are the first ones to be scanned during the boot phase. Therefore, the boot adapter must be installed in one of these slots in order for it to be enumerated first. Once that is accomplished, boot order is prioritized by the BIOS settings, as usual.

Power Consumption Settings To take advantage of Windows’ ability to adjust server power consumption (and reduce costs) based on workload, follow these steps:

1. During system boot, press F9 to run the ROM-Based Setup Utility (RBSU).

2. Select Power Management Options.

3. Select HP Power Profile.

4. Select Custom.

5. Select HP Power Regulator. 6. Select OS Control Mode (see Figure 8).

Figure 8: Enabling OS Control Mode using RBSU

7. Select Advanced Power Management Options.

8. Select Minimum Processor Idle Power Core State.

9. Select C1E State (see Figure 9).

16

Figure 9: C1E State selection

10. Select Collaborative Power Control. 11. Select Disabled (see Figure 10).

Figure 10: Disabled selection

17

It is worth mentioning here that power savings usually come at the price of performance. They are, by nature, contradictory goals. However, choosing C1E State does provide some degree of power savings without impacting performance too much. If saving power is not a concern, No C-states might be a better choice. HP recommends that you test your representative workload in order to make the right decision.

12. Exit all RBSU menu screens using the Esc key. Then press the F10 key to confirm your changes and reboot the system.

Windows Server 2008 R2 Best Practices for the DL980 When running Windows Server 2008 R2 on the HP ProLiant DL980 G7 Server there are several recommended best practices that are fully explained in this section:

• Enable hyper-threading to maximize resource usage (unless inappropriate, as with Hyper-V) • Enable 44-bit mode after OS installation • Enable x2APIC Mode (for configuration with 8 Intel® Xeon® E7 processors) • Install a Microsoft hotfix on systems with more than 1 TB RAM • Implement Hyper-V workaround for systems with more than 512GB RAM

Enable Hyper-threading to Maximize Resource Usage for Large Applications (but NOT for Hyper-V or Windows 2008 SP2 with 8 sockets populated) As noted in the OS Matrix earlier, Windows Server 2008 R2 can support up to 160 logical processors (8 Intel® E7-family processors x 10 cores x 2 threads/core, with hyper-threading enabled).

In general, it is highly recommended that hyper-threading be enabled on DL980s running Windows Server 2008 R2, in order to take full advantage of these additional logical processors, assuming that your application is aware of processor groups.

The processor group concept was introduced in Windows 7 and Windows Server 2008 R2 to support more than 64 logical processors. Any machine with more than 64 logical processors has more than one processor group by necessity. Drivers and applications are all expected to run properly in the multi-group context if coded properly. For more information, see: http://msdn.microsoft.com/en-us/library/dd405503%28VS.85%29.aspx.

Hyper-threading is controlled by the system BIOS, and is enabled by default on the DL980 G7. If, however, it was disabled for some reason, you should follow these steps to enable it:

1. During system boot, press F9 to run the ROM-Based Setup Utility (RBSU).

2. Select System Options.

3. Select Processor Options.

4. Select Intel® Hyperthreading Options (see Figure 11).

18

Figure 11: Enabling Hyper-threading using RBSU

5. Select Enabled to turn hyper-threading ON.

6. Exit all RBSU menu screens using the Esc key. Then press the F10 key to confirm your changes and reboot the system.

For more information about the RBSU, see the HP ROM-Based Setup Utility User Guide: http://h20000.www2.hp.com/bc/docs/support/SupportManual/c00191707/c00191707.pdf

Hyper-threading + Windows Server 2008 R2 + Hyper-V version 2 Windows Server 2008 R2 supports Hyper-V version 2. But as stated earlier in the Hyper-V support matrix, version 2 supports a maximum of 64 logical processors. In other words, Hyper-V can only use the first 64 processors it sees. So on DL980s running Windows Server 2008 R2 with the Hyper-V role enabled and 8 processor-sockets (8 cores or more) fully populated, you must turn hyper-threading OFF instead (the procedure is the same as described previously, except that you select Disabled in Step 5 instead).

Enable 44-bit Mode after OS Installation The default RBSU method for addressing memory on the DL980 G7 is 40-bit. However, the optimal setting for running Windows Server 2008 R2 is 44-bit. Therefore, Windows Server 2008 R2 users must change this setting to 44-bit to make full use of the available address space and reduce the decoding faults and data coherency issues that can occur if the less-optimum 40-bit setting is used.

To enable the 44-bit System Address Mode, follow these steps:

1. During the boot process, confirm that the system is in the default 40-bit mode by looking at the RBSU splash screen. It indicates a System Address Mode of 40-bit (see Figure 12).

19

Figure 12: Confirming System Address Mode reported by RBSU

2. Install the Windows Server 2008 R2 operating system. 3. Install the Windows Server 2008 R2 operating system updates, using the Smart Update CD, as

described in the HP Smart Update CD section (on page 5). 4. Reboot the system. 5. Enable a System Address Mode of 44-bit as follows:

a. During system boot, press F9 to run the ROM-Based Setup Utility (RBSU). b. Select Advanced Options. c. Select Advanced System ROM Options. d. Select Address Mode 44-bit (see Figure 13).

20

Figure 13: Enabling 44-bit System Address Mode using RBSU

e. Ignore the Warning message: Caution: Enabling 44-bit Address Mode may

cause problems with an operating system if it does not support 44-bit memory addressing.

f. Select Enabled to turn 44- bit memory addressing ON.

6. Exit all RBSU menu screens using the Esc key. Then press the F10 key to confirm your changes and reboot the system.

7. Confirm the new 44-bit System Address Mode by looking at the RBSU splash screen during system boot, as described in Step 1.

Enable x2APIC Mode for Systems with 8 Intel® Xeon® E7-family Processors Running Windows Server 2008 R2 (with or without SP1) x2APIC architecture provides better management of interrupts, but Microsoft hotfixes are required before enabling this feature on systems with the recommended number of Intel Xeon E7-family processors.

The following hotfixes are found either on the Microsoft Knowledge Base (KB) website, or on the HP Smart Update CD, v7.1 or later:

• KB #2398906 A computer running an x64-based version of Windows Server 2008 R2 with more than 128 processors crashes at startup: http://support.microsoft.com/kb/2398906

• KB #2303458 A fix is available for three x2APIC mode issues in x64-based versions of Windows Server 2008 R2: http://support.microsoft.com/kb/2303458

• KB #2508054 Users experience "0x0000005C" stop error code or assertion failure during startup if Driver Verifier is enabled in Windows Server 2008 R2: http://support.microsoft.com/kb/2508054

21

Note Hotfix #2508054 is only needed on systems running Windows Server 2008 R2 (with or without SP1), and only when the Windows Driver Verifier is enabled. The Driver Verifier is used for debugging purposes, and is not normally enabled by default since it affects performance adversely.

After installing the required hotfixes, enable the x2APIC Mode in the BIOS firmware and within Microsoft Windows operating system using the bcdedit command to set the physical destination interrupt delivery mode.

To enable the x2APIC Mode in BIOS firmware, follow these steps:

1. During system boot, press F9 to run the ROM-Based Setup Utility (RBSU).

2. Select Advanced Options.

3. Select Advanced System ROM Options.

4. Select x2APIC Options (see Figure 14).

5. Select Auto to enable x2APIC Mode.

Figure 14: x2APIC selection

To set the x2APIC physical destination interrupt delivery mode within Windows using the bcdedit command, follow this step:

1. Run the following command: bcdedit /set usephysicaldestination yes

22

Note The Optimize System Configuration button on SMH automatically creates the bcdedit entry above, upon detecting the system is running Windows 2008 R2 or Windows 2008 R2 SP1 with x2APIC enabled.

Install Microsoft Hotfix on Systems with More Than 1 TB RAM Windows Server 2008 R2 can sometimes fail to install or run on systems with more than 1 TB of RAM and 44-bit addressing mode enabled because some physical addresses may go beyond 40 bits. This can generate a bug check for some specific memory conditions, causing the installation to fail. Microsoft has documented the issue and its resolution in Knowledge Base Article #980598, titled Windows Server 2008 R2 cannot be installed or started on a computer that has 1 TB or more of RAM, found here: http://support.microsoft.com/kb/980598. This hotfix is included on the Smart Update CD in the Windows Server 2008 R2 updates, and it must be installed to solve this specific problem. It is not required for systems running Windows Server 2008 R2 SP1.

Workaround When Enabling Hyper-V on Systems with More Than 512 GB RAM In Windows Server 2008 R2 (without SP1), if you enable the Hyper-V role on DL980s with between 512 GB and 1 TB of RAM, these systems will hang after rebooting and never return back to the Windows Login prompt. A Microsoft hotfix must be installed, but it cannot be installed until the Hyper-V role is enabled, and once enabled, you run into the problem. There are several workarounds, all of which involve a temporary reduction in the amount of memory made available to Windows. Use one of these following temporary solutions to reduce system memory:

• Method #1: Enable Mirrored Memory using RBSU • Method #2: Decrease maximum memory in Windows using the MSCONFIG utility • Method #3: Reduce addressable memory with a Windows boot option

After the memory is reduced to less than 512 GB using one of the previous methods, you must enable Hyper-V, then apply the Microsoft hotfix for the problem, documented in Knowledge Base Article #2133637, titled The computer stops responding at startup in Windows Server 2008 R2 if you enable the Hyper-V role and if you have more than 512 GB of RAM, found here: http://support.microsoft.com/kb/2133637. Reboot again after applying the fix and then restore the system memory back to its original value.

Method #1: Enable Mirrored Memory using RBSU Mirrored Memory provides maximum protection against uncorrectable memory errors that might otherwise result in system failure. To enable Mirrored Memory using RBSU, follow these steps:

1. During system boot, press F9 to run the ROM-Based Setup Utility (RBSU).

2. Select System Options.

3. Select Advanced Memory Protection.

4. Select Mirrored Memory with Advanced ECC Support (see Figure 15).

23

Figure 15: Enabling Mirrored Memory using RBSU

5. Select Enabled to turn Mirrored Memory ON.

6. Exit all RBSU menu screens using the Esc key. Then press the F10 key to confirm your changes and reboot the system.

Note Enabling or disabling mirrored memory will increase your system boot time, since any change invokes a mandatory test of all system memory during the boot process.

Method #2: Decrease maximum memory using MSCONFIG utility To decrease maximum memory in Windows using the MSCONFIG utility, follow these steps:

1. At the Windows desktop, click Start Run.

2. Type msconfig and press Enter. 3. Select the Boot tab.

4. Click the Advanced Options… button.

5. Select the Maximum Memory check box and lower the value; for example, to something like 48GB (see Figure 16).

24

Figure 16: Decreasing Maximum Memory using MSCONFIG utility

6. Click OK and exit the MSCONFIG utility. 7. Reboot the system.

Method #3: Reduce addressable memory with a Windows boot option To decrease maximum memory in Windows with a temporary boot option, follow these steps:

1. Enter the following command at the Windows command line: bcdedit /enum Various system boot entries will display.

2. Locate the Windows Boot Loader boot entry and write down its GUID number, for example: ba85e57e-8d4e-11df-b805-c70cfe047125

3. Use the following command to create a new Windows Boot Loader entry (by copying the current one): bcdedit /copy GUID /d “Less Memory Config”, where GUID is the GUID number noted in the previous step, and Less Memory Config is a descriptive name given to the new boot entry that will assign less memory to the system.

4. Run the bcdedit /enum command again to verify that the new entry was created. Notice that the new entry is exactly the same as the current one, except it has been assigned a new GUID number. For example, the new boot entry has the following GUID: ba85e580-8d4e-11df-b805-c70cfe047125

5. Alter the new boot entry so it will assign less memory to the system. For example, for a system with 1 TB RAM, use the following command to give it less than 512 GB, by removing 640 GB: bcdedit /set ba85e580-8d4e-11df-b805-c70cfe047125 removememory 640000

6. Reboot the system. The Windows Boot Manager screen displays, showing the new boot entry you just created. Scroll down to select the new entry (Less Memory Config) that will boot the system with reduced memory (see Figure 17).

25

Figure 17: Windows Boot Manager screen showing new boot entry

For a complete description of the BCDEdit command and an explanation of all available options, see the following Microsoft documentation, titled BCDEdit Command-Line Options, found here: http://technet.microsoft.com/en-us/library/cc709667(WS.10).aspx

After you reduce the system memory using one of the previous methods, follow these steps: 1. Enable the Hyper-V role. 2. Download and install the Microsoft Hyper-V hotfix for this problem, documented in Knowledge

Base Article #2133637, titled The computer stops responding at startup in Windows Server 2008 R2 if you enable the Hyper-V role and if you have more than 512 GB of RAM, found here: http://support.microsoft.com/kb/2133637

3. Reboot the system and remove the memory restriction, depending on the method previously used:

a. Method #1: Press F9 during system boot, disable Mirrored Memory within RBSU, and reboot.

b. Method #2: After system boot, use MSCONFIG utility to remove the maximum memory limit, and reboot.

c. Method #3: Let the system reboot using its default boot option (with no memory restriction).

Windows Server 2008 SP2 Best Practices for the DL980 This section describes several recommended best practices when running Windows Server 2008 SP2 on the HP ProLiant DL980 G7 Server.

Disable Hyper-threading to Keep Logical Processor Count at 64 Maximum Hyper-threading refers to Intel’s proprietary technology for increasing parallel computational power (processor multi-tasking) by allowing the OS to see and address each physical processor core as if it were 2 virtual processors. It also enables the OS and applications to share work between those 2 virtual processors whenever possible, thereby making use of resources that would otherwise sit idle.

26

As noted in the OS Matrix, Windows Server 2008 SP2 supports up to 64 logical processors on the DL980 (8 sockets x 8 cores, or more). Therefore, if hyper-threading is enabled on these systems, it increases the logical processor count to 128 logical processors, or well above that limit. This wastes processing power because the system simply cannot recognize processors above the 64 processor limit. For example, on a DL980 G7 with 8 sockets, 8 cores per socket, and hyper-threading enabled, Windows Server 2008 with SP2 sees only the first 16 threads in Processor #1, the next 16 threads in Processor #2, and so on, until it reaches Processor #4 (16 threads x 4 processors = 64 threads or logical processors). Any processor after the fourth processor is a wasted resource. In this example, four entire processors, or half of the available processing power, sit idle.

Hyper-threading is controlled by the system BIOS, and is enabled by default on the DL980 G7. To disable it, follow these steps:

1. During system boot, press F9 to run the ROM-Based Setup Utility (RBSU).

2. Select System Options.

3. Select Processor Options.

4. Select Intel® Hyperthreading Options (see Figure 18).

5. Select Disabled to turn hyper-threading OFF.

6. Exit all RBSU menu screens using the Esc key. Then press the F10 key to confirm your changes and reboot the system.

Figure 18: Disabling Hyper-threading using RBSU

For more information about the RBSU, see the HP ROM-Based Setup Utility User Guide: http://h20000.www2.hp.com/bc/docs/support/SupportManual/c00191707/c00191707.pdf

27

Hyper-threading + Windows Server 2008 SP2 + Hyper-V version 1 Windows Server 2008 SP2 supports Hyper-V version 1. But as stated earlier in the Hyper-V support matrix, version 1 supports a maximum of 24 logical processors. In other words, Hyper-V can only use the first 24 processors it sees. So on DL980s running Windows Server 2008 SP2 with the Hyper-V role enabled, there is an even greater need to turn hyper-threading OFF.

Workaround for Possible Driver Conflict When Logging in Using iLO Another issue, although intermittent, occurs when logging onto the system using iLO (HP’s Integrated Lights Out – a remote console technology). The problem has usually been seen in systems with 1TB of RAM, and results from a conflict with the ATI2dvag video driver. The resulting blue screen displays the following stop code:

0x000000EA (0xFFFFFA86B9A91860, 0xFFFFFA86B9B64910, 0xFFFFFA86B9AF37C0, 0x0000000000000001)

To resolve this issue, you must add the following two keys to the system registry:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Watchdog\Display\ DisableBugCheck REG_DWORD 0x1

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Watchdog\Display\ EaRecovery REG_DWORD 0x0

For More Information For an overview of the HP ProLiant DL980 G7 server: http://www.hp.com/servers/dl980

For more information about the HP ProLiant DL980 G7 server with HP PREMA Architecture: http://h10010.www1.hp.com/wwpc/us/en/sm/WF25a/15351-15351-3328412-241644-4222584-4231377.html

To download the recommended components described in this document, along with other drivers and software, visit the HP ProLiant DL980 G7 Support web page: http://www.hp.com/support/dl980g7

For a complete library of all documentation supporting the HP ProLiant DL980 G7 server: http://www.hp.com/go/proliant_servers-docs (click on the link, “HP ProLiant DL980 G7 Server series”)

For additional best practices whitepapers for the HP ProLiant DL980 G7 server: http://www.hp.com/go/proliant-DL980-docs

Documentation Feedback HP welcomes your feedback. To make comments and suggestions about product documentation, send a message to: docsfeedback@hp.com. Include the document title, part number, and filename found at the end of the URL string (for example: c02861709.pdf). All submissions become the property of HP.

© Copyright 2011-2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.

Microsoft, Windows, and Windows Server are U.S. trademarks of Microsoft Corporation.

Intel and Xeon are U.S. trademarks of Intel Corporation.

4AA3-6553ENW, Created September 2011; Updated June 2012, Rev. 5