Tips for Improving Your Cisco UCS Power EfficiencyFebruary 2015

White Paper

© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public information.

What You Will LearnThis document shows you how to increase the power efficiency of your Cisco UCS equipment with recommendations and best practices as well as guidelines for selecting hardware and BIOS options. It also provides a list of tools that you can use to validate your power-efficiency choices.

© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public information. Page 2

ContentsChoosing Your Hardware ................................................................................. 3

Processor Considerations ..................................................................................... 4

Memory Considerations ........................................................................................ 5

Storage Components....................................................................................... 6

Power Supplies ............................................................................................... 7

BIOS Configuration Choices ............................................................................ 8

BIOS Settings: No Performance Impact ................................................................ 8

BIOS Settings: Moderate Performance Impact ...................................................... 9

BIOS Settings: Maximum Energy Savings with Moderate to Significant Performance Impact ............................................................................ 10

Tools to Help Determine Power Efficiency ........................................................ 10

Cisco UCS Power Calculator ................................................................................ 10

Cisco UCS Performance Manager ........................................................................ 11

Microsoft Windows: CPU-Z .................................................................................. 11

Microsoft Windows Task Manager ........................................................................ 11

Microsoft Windows: powercfg.exe ........................................................................ 11

Linux Tools ............................................................................................................ 11

Improved Power Efficiency .............................................................................. 12

For More Information ....................................................................................... 12

Tips for Improving Your Cisco UCS Power EfficiencyFebruary 2015

Tips for Improving Your Cisco UCS Power Efficiency

White PaperFebruary 2015

© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public information. Page 3

Energy management is critical to efficient data center operation. You are constantly balancing power consumption against the need for availability, scalability, and performance to lower operating costs and support robust service levels. Power consumption is a complex function that depends on your hardware configuration, BIOS settings, and workload. Although optimizing power efficiency can be an overwhelming task, Cisco is dedicated to advancing energy efficiency in your data centers. One example of this priority is the Cisco Unified Computing System™ (Cisco UCS®). With its unified, policy-based, and ecosystem-friendly approach, Cisco UCS helps you easily manage hardware and software components—including power—to improve the efficiency of the entire system and reduce operating expenses.

This document shows you how to increase the power efficiency of your Cisco UCS equipment with recommendations and best practices as well as guidelines for selecting hardware and BIOS options. It also provides a list of tools that you can use to validate your power-efficiency choices.

Choosing Your HardwareThe way you configure your server hardware affects the efficiency with which the server uses power. This section presents best practices for choosing and configuring your server hardware to improve power utilization.

Tips for Improving Your Cisco UCS Power EfficiencyFebruary 2015

© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public information. Page 4

Processor ConsiderationsWith every Intel® Xeon® processor release, you gain energy efficiency improvements. Intel delivers processors in what Intel calls a tick-tock release cycle (Figure 1). With the “tick” release, Intel updates the manufacturing process using smaller transistors and shorter interconnect leads to reduce capacitance. Less capacitance means less delay, which results in lower voltage and less power consumption. With the “tock” release, Intel updates the microarchitecture, introducing new or enhanced power management features such as low idle states, core voltage regulation to optimize power delivery, and link power management states for SATA and PCI Express (PCIe) bus interfaces.

For example, Figure 2 shows the relative power efficiency improvements from the Intel Sandy Bridge processor (tock) to the Ivy Bridge (tick) and the Haswell (tock). The power efficiency improvement from a manufacturing process change was 1.55 times greater than the baseline (Sandy Bridge). The power efficiency improvement from a combination of manufacturing and microarchitecture changes was 1.92 times greater than the baseline. The idle power from the same microarchitecture (Sandy Bridge and Ivy Bridge) is equivalent, but the change in microarchitecture with Haswell resulted in a 14 percent reduction in idle power consumption.

Tick Tock Tick Tock Tick Tock Tick Tock

45 nm

Nehalem HaswellSandy Bridge

32 nm 22 nm 14 nmManufacturing Process

Technology Update Cycle

MicroarchitectureUpdate Cycle

Figure 1. Intel’s “Tick-Tock” Processor Release Cycle

Idle Power C6 State(Lower Is Better)

SPECpower_ssj2008 Power E�ciency(Higher Is Better)

1.00 1.00

0.86

1.00

Pow

er (

Wat

ts)

Pow

er p

er W

att

Sandy Bridge

Ivy Bridge

Haswell

1.55

1.92

1.2

1.0

0.8

0.6

0.4

0.2

0

2.0

1.6

1.2

0.8

0.4

0

Figure 2. Power Efficiency and Idle Power Consumption Improvements Through Updates in Intel Xeon Processor Manufacturing and Microarchitecture Relative to the Sandy Bridge Idle Power

Tips for Improving Your Cisco UCS Power EfficiencyFebruary 2015

© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public information. Page 5

To optimize server efficiency, you need to understand your performance requirements when purchasing a processor. You should select the lowest-power processor that meets your performance, feature, and growth needs. Intel Xeon processor families are divided into six market segments: Advanced, Standard, Basic, High Core Count, Frequency Optimized, and Low Power. Each segment is optimized for a specific market workload. Cisco UCS servers support all Intel processor segment offerings.

In addition, within each segment, lower-power options are available that can provide nearly equivalent performance while consuming less power.

We give you the capability to select the lowest-power processor option that meets your performance and feature requirements. If one of the processors from the Low Power segment meets your performance and feature requirements, Cisco recommends selecting this processor segment to increase your power savings.

Memory ConsiderationsAs with processors, several factors affect memory power consumption, including the amount of memory, the type of memory, the way the memory is organized, the memory technology used, the operating frequency, and the operating voltage. For a small total memory configuration, you can consider using unregistered dual in-line memory modules (UDIMMs). UDIMMs do not have the additional register found on registered DIMMs (RDIMMs) and load-reduced DIMMs (LRDIMMs), resulting in less power consumption per module. However, UDIMMs are limited in capacity, and only two UDIMMs can be configured per memory channel

When possible, you should select DIMMs with x8 dynamic RAM (DRAM). DIMMs with x8 DRAM access half as many DRAM devices at a time compared to DIMMs with x4 DRAM (Figure 3). This configuration reduces the average power consumption. For example, two 8-GB double-date-rate 4 (DDR4) 1Rx8 RDIMMs consume less power than one 16-GB DDR4 2Rx4 RDIMMs (Figure 4). If the DRAM organization is the same, consider higher-capacity DIMMs to reduce the number of total DIMMs being used. Besides reducing total power consumption, reducing the number of DIMMs per channel (DPC) can allow the memory configuration to operate at a higher frequency and improve performance.

Second Rank Is Energized and Consuming Power but Not Being Accessed

Front

Read Access

1Rx8

2Rx4

Back

1

1 2

Figure 3. Memory Organization Comparison

Tips for Improving Your Cisco UCS Power EfficiencyFebruary 2015

© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public information. Page 6

In addition, if you select low-voltage DIMMs that operate at 1.35 volts (V), you can reduce total memory power by as much as 20 percent. Additional power savings are possible through the use of DDR4 memory (Figure 5). Typical DDR3 memory operates at 1.5 or 1.35V, whereas DDR4 memory operates at 1.2V while also providing higher performance.

Storage ComponentsIf you can use fewer, higher-capacity storage devices, you can reduce the total number of storage devices as well as your server’s total power consumption. However, you must balance your power savings against the performance impact on workloads sensitive to the number of I/O operations per second (IOPS). Although

DDR4 8-GB 1Rx8 2133-MHz RDIMM

DDR4 16-GB 2Rx4 2133-MHz RDIMM

DDR4 32-GB 4Rx4 2133-MHz RDIMM

Two DIMMs per ChannelOne DIMM per Channel

Pow

er (W

atts

per

DIM

M)

Bandwidth (GB/Second/Socket)0 060 60

0

2

4

6

8

10

40nm DDR3 30nm DDR3L

8

Core Power

I/O Power

20nm DDR3L 20nm DDR4

6

4

2

0

Pow

er (W

atts

)

Figure 4. Power Consumption Compared to Memory Bandwidth

Figure 5. DDR4 DIMMs Consume 25 to 30 Percent Less Power

Tips for Improving Your Cisco UCS Power EfficiencyFebruary 2015

© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public information. Page 7

solid-state drive (SSD) capacity can be limited, the drives have exceptional performance and typically consume 3 to 6 watts (W) less power than equivalent-capacity hard-disk drives (HDDs). If top-end performance is not necessary for your particular workload, you should select the HDD with the lowest operating speed. A HDD operating at 10,000 rpm typically consumes 1 to 4 watts less power than an equivalent HDD operating at 15,000 rpm.

When possible, select software RAID, which eliminates the need for PCIe hardware RAID adapters and reduces your server power consumption by 8 to 17 watts. The Cisco UCS product portfolio includes FlexFlash technology, a low-power secure-digital (SD) card that allows you to boot locally and eliminates the need for hardware RAID controllers and HDDs.

Power SuppliesThe efficiency with which power supplies convert input power to output varies based on input voltage and load. It may seem reasonable to populate your server with high-capacity power supplies. However, to reduce conversion losses, we recommend that you right-size the power supply capacity for the selected hardware configuration to optimize power-supply efficiency. Use the Cisco UCS Power Calculator at http://ucspowercalc.cisco.com to right-size your power supply selection. Typically, power supplies are more efficient when operating at a higher percentage of their rated output. Thus, selecting a lower-capacity power supply (right-sizing) increases the load per power supply. As shown in Figure 6, the optimal choice for the sample configuration is two power supply units (PSUs) running at 30 percent load, instead of four PSUs running at 15 percent load.

Load (Percent)

Pow

er E

�ci

ency

(Pe

rcen

t)

85

87

89

91

93

95

10 20 30 40 50 60 70 80 90 100

Recommended Operating Zone

Power Savings From 2 PSUs at 30% Load Compared to 4 PSUs at 15% Load

Figure 6. Power Supply Optimal Power Savings Operating Zone

Tips for Improving Your Cisco UCS Power EfficiencyFebruary 2015

© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public information. Page 8

The Cisco UCS portfolio includes power supplies that support both high-line (200 to 240 volts) and low-line (100 to 120 volts) inputs. If you can, use high-line input. High-line operation reduces conversion losses by up to 1 to 2 percent. Figure 7 shows the efficiency of the high-line compared to the low-line input voltages of a typical Platinum-rated Cisco UCS power supply.

BIOS Configuration ChoicesThe way you configure your BIOS options also affects the efficiency with which your server uses power. This section presents some best practices for configuring your BIOS options to improve power utilization in your servers. These general guidelines are not intended as an all-inclusive list of BIOS tuning recommendations. For a complete list of BIOS tuning recommendations, consult the platform-specific BIOS tuning guides documented in the For More Information section at the end of this document.

BIOS Settings: No Performance ImpactWhen you attach USB devices to your server, particularly a mouse and a keyboard, they can affect the processor’s package and core C-state residency, which affects the amount of power consumed during idle periods. If you are not using these devices, you should disable them in the BIOS to reduce overall idle power consumption. You can do this without affecting performance.

Whereas the traditional, manual method of setting BIOS parameters is time consuming and error prone, Cisco UCS uses BIOS policies that automate the process of configuring BIOS parameters, allowing you to easily deploy BIOS changes to many servers or a single server.

Table 1 lists several Cisco UCS BIOS settings that should be disabled to reduce server power consumption.

}

10

Pow

er E

�ci

ency

(Per

cent

)

Load (Percent)

Average Power E�ciency Di�erence:Approximately 1 to 2 Percent

200-240 Volt Input

100-120 Volt Input

20 30 40 50 60 70 80 90 10085

87

89

91

93

95

Figure 7. Power Efficiency Curve Differences Between High-Line and Low-Line Input Voltages

Tips for Improving Your Cisco UCS Power EfficiencyFebruary 2015

© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public information. Page 9

BIOS Settings: Moderate Performance Impact

Setting Description

SATA ports and mode Disable if not used

Onboard server configuration utility (SCU) support Disable if not used

USB ports: External Disable if not used

USB ports: Internal Disable if not used

PCIe slot Disable if not used

Table 1. Cisco UCS BIOS Settings to Adjust, with No Performance Impact, for Optimal Power Efficiency

Setting Suggested State

C6 C6 is the deep power conservation core idle state and is enacted on a per-core basis.

Enable

C1E The C1 idle state occurs when all hardware threads supported by a core have processed a HALT instruction.

Enable

Energy performance This setting is the energy efficiency policy through which the OS can provide hints to the power control unit (PCU) through a model-specific register (MSR) about the desired power-to-performance trade-off. The PCU can use these hints to manipulate the internal behavior of the processor and determine whether to target higher performance or greater power savings.

Balanced Performance

P-state coordinationThis is a setting in which each core shares the same voltage. Because of this sharing, coordination is required to reconcile P-state requests from logical processors. There are three models of P-state resolution defined by Advanced Configuration and Power Interface (ACPI): SW_ANY, SW_ALL, and HW_ALL. (The Haswell server processor has specific support for SW_ANY and HW_ALL, but does not prevent the use of SW_ALL.) The selection of a specific P-state coordination type (e.g., HW_ALL, SW_ALL or SW_ANY) is how the BIOS communicates the P-state support model to the Operating System Power Management (OSPM). In SW_ANY mode, the OS power manager is responsible for coordinating the P-state among logical processors in a package. SW_ANY is the preferred setting for OS deployments that are biased toward power savings, such as Microsoft Windows, and HW_ALL is the preferred setting for OS deployments that are biased toward performance, such as Linux.

SW_ANY

LV DDR mode This setting determines whether the system prioritizes low voltage or high-frequency memory operations. Power Savings Mode prioritizes low voltage memory operations over high-frequency memory operations. This mode may lower the frequency of memory operations to keep the voltage low.

Power Savings Mode

Table 2. Cisco UCS BIOS Settings That Improve Power Efficiency with Minimal to Moderate Performance Impact

Tips for Improving Your Cisco UCS Power EfficiencyFebruary 2015

© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public information. Page 10

Before you start adjusting these settings, you need to determine the trade-off between power consumption and performance requirements for your workload. Table 2 shows the BIOS settings that will improve power utilization, but with minimal to moderate performance impact.

BIOS Settings: Maximum Energy Savings with Moderate to Significant Performance ImpactSome BIOS settings deliver extremely efficient power utilization, but the trade-off is lower performance. If you have workloads that don’t require high performance to function well, these settings can give you the most power savings (Table 3).

Tools to Help Determine Power EfficiencyCisco UCS Power CalculatorYou can use the Cisco UCS Power Calculator to evaluate power savings from the hardware configurations discussed previously: lower-power processors, higher-density DIMMs, UDIMMs, higher-density storage, SDDs, right-sized power supplies, etc. The Cisco UCS Power Calculator provides you with power estimates based on a given hardware configuration and target workload. It is most helpful in configuration planning to right-size your Cisco UCS deployment for the price, power,

Setting Description

Intel Turbo Boost TechnologyThis setting determines whether the processor automatically increases its frequency if it is running below power, temperature, or voltage specifications. If this setting is disabled, the processor never increases its frequency automatically.

Disable to reduce peak power.

Number of enabled coresThis setting determines whether the maximum number of CPU cores are activated.

Reduce the number of active physical cores to reduce power use.

Energy performance policyThe OS can provide hints to the PCU through an MSR about the desired ratio of power to performance. The PCU can use these hints to manipulate the internal behavior of the processor and determine whether to target higher performance or better power savings. Choose Energy Efficient to prioritize power savings over performance.

Energy Efficient

DDR speedThis setting determines the frequency of memory operations.

Set to a lower frequency to reduce power use.

Port link speedThe setting determines PCIe link frequency.

Review for each adapter; set to lower speeds to reduce power use.

Table 3. Cisco UCS BIOS Settings That Provide Maximum Power Efficiency with a Moderate to Significant Performance Impact

Tips for Improving Your Cisco UCS Power EfficiencyFebruary 2015

© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public information. Page 11

and performance that you need for your applications. You can perform what-if comparisons before you make your purchase.

The Cisco UCS Power Calculator is based solely on the power profile of the hardware being configured and assumes that all BIOS settings are configured for maximum performance. Depending on your workload and both firmware and operating system settings, you may be able to obtain even greater power efficiency than that predicted by the calculator.

Cisco UCS Performance ManagerYou can use the Cisco UCS Performance Manager to evaluate the performance impact of power-saving configurations such as BIOS settings. This tool allows you to quantify the impact of power-saving features on performance.

Microsoft Windows: CPU-ZYou can use the CPU-Z application, by CPIUD, or an equivalent application within Microsoft Windows Server, to gather system information to verify your hardware configuration and BIOS settings. Although this tool doesn’t tell you what your power efficiency is, it offers an excellent way to verify your configuration.

Microsoft Windows Task ManagerYou can use the Microsoft Windows Task Manager to verify that your workloads are preforming within the range you desire and expect after you change BIOS power settings that may affect performance. If the performance isn’t what you want, you can readjust system BIOS settings to increase performance.

Microsoft Windows: powercfg.exeYou can use the powercfg.exe application in Microsoft Windows Server to control power settings and configure your server’s default hibernate and standby modes to save power. You can also use the –energy option to analyze the system for common energy-efficiency problems, such as a selective suspend process that prevents the processor from entering a low power state.

Linux ToolsStandard Linux comes with a comprehensive set of simple yet useful applications such as top and free that can be used to validate the platform under test and monitor system performance. In addition, each Linux distribution often delivers its own enhanced tools. The Linux community has numerous open source projects ongoing, with each addressing different management concerns and capabilities. This software—for example, yum or apt-get—can be installed separately by the package manager, or it can be downloaded and installed manually.

These are some of the more useful Linux applications:

• The cpuinfo and meminfo commands retrieve processor and memory information installed in the system.

• The top command provides a dynamic real-time view of a running system, similar to Microsoft Windows Task Manager.

• The free command displays the amount of free and used physical and swap memory in a system, as well as buffers used by the kernel.

Cisco has more than 200 offices worldwide. Addresses, phone numbers, and fax numbers are listed on the Cisco Website at www.cisco.com/go/offices.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks. Third party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R) LE-43505-00 02/15

Americas Headquarters Cisco Systems, Inc. San Jose, CA

Asia Pacific Headquarters Cisco Systems (USA) Pte. Ltd. Singapore

Europe Headquarters Cisco Systems International BV Amsterdam, The Netherlands

Tips for Improving Your Cisco UCS Power EfficiencyFebruary 2015

• The dmidecode tool can dump a system’s desktop management interface (DMI), or SMBIOS, table contents, such as BIOS, system, baseboard, chassis, processor, memory, cache, connector, and slot information, with great detail.

• The lm_sensors tool is associated with the kernel driver for monitoring hardware health, including temperature, fan speed, and voltage level, and other information about the running hardware.

Improved Power EfficiencyWith the combination of Cisco UCS architecture and adjustments in hardware selection and BIOS settings, you can achieve the optimal balance between performance and power consumption to get the most from your investment. Power management can be challenging. For example, changing BIOS settings to tune power and performance is difficult with traditional architecture. Cisco UCS service profiles and BIOS policies make it easy to optimize power management settings. Cisco wants you to achieve the best results for your system by understanding the trade-offs and best practices for improving the overall efficiency of your data center resources.

For More Information• Power Management in the Cisco Unified Computing System: An Integrated

Approach

• Managing Power in Cisco UCS

• Power Management States: P-States, C-States, and PC-States

• Data Center Power and Cooling

• Best Practices for Setting BIOS Parameters for Performance

• Cisco UCS Power Efficiency Beats HP, IBM and Dell Solutions

• Cisco Unified Computing System BIOS Settings

• BIOS Parameters by Server Model

• Configuring BIOS Settings

• Cisco UCS Power Manager