Datacenter Standards,Best Practices,

and Success Stories

charm@ksc.au.edu

www.charm.au.edu

Prof. Dr. Srisakdi CharmonmanPresident Emeritus of the

Computer Association of ThailandCEO of the College of Internet Distance Education

Assumption University of Thailand

Invited Talk for “Datacenter Satellite Conference & Exhibition Series” Auckland, Bangkok, Beijing, Hyderabad, Kuala Lumpur, Melbourne, and Osaka

organized thru VDO Conference by Pacific and Strategy Holding Ptc. Ltd. For Bangkok at Westin Sukhumvit Hotel, July 16, 2009

1. Introduction2. Datacenter Standards3. Datacenter Best

Practices4. Green Datacenter5. Datacenter Success

Stories 6. Concluding Remarks 2

Datacenter Standards, Best Practices,

and Success Stories.

3

1. Introduction.

As of the year 2009, most organizations depend on computer network and the Internet.

Most organizations cannot continue operating without computer network.

4

Introduction (Cont.)

Therefore, most organizations need backup

for their computer network

which may be in the form of datacenter.

According to Wikipedia, “a data center or datacenter or datacentre or server farm is a facility to house computer systems and associated components, such as telecommunications and storage systems.”

5

Introduction (Cont.)

A datacenter generally includes: * Redundant or backup power supplies* Redundant data communications connections* Environmental controls and security devices

6

Introduction (Cont.)

During the 1980s, microcomputers were used in most organizationsbecause of their low cost and ease of operation.

The word “datacenter” appeared in the 1990s, when more powerful microcomputers called “servers” were used to replace mainframe, and intalled in old mainframe room.

7

Introduction (Cont.)

During the 2000s, organizations needed nonstop operation and 24/7 presence on the Internet, and so datacenters became more popular.

As a matter of fact, they were called “Internet DataCenters or IDC”

In 2009, datacenter may also be operated by datacenter service providers.

A well-known datacenter standard is TIA-942 (TIA = Telecommunication Industry Association)

For more information see www.adc.com/Library/Literature/102264AE.pdf

8

2. Datacenter Standards.

TIA- 942 is the first standard dd dddddddddddd ddddddd datacenter infrastructure.

TIA-942 includes g uidelines on a wide range

of subjects. d ddddddd d dddddddd ddd d dd

ddddddd dddddddddd cabling sys tem using standard media.

9

ANSI/TIA-942

Later, ANSI (American National Standards Institute) also approved TIA-942.

TIA-942 became “ANSI/TIA-942 Telecommunications Infrastructure for Data Center”.

10

ANSI/TIA-942 (Cont.)

11

ANSI/TIA-942 (Cont.)

The requirements of ANSI/TIA-942 covers the following.2.1 Site Space and Layout2.2 Cabling Infrastructure2.3 Tiered Reliability2.4 Environmental Consideration

12

ANSI/TIA-942 (Cont.)

The datacenter should be designed with plenty of flexible “White Space”, empty space that can accommodate future racks or cabinets.

For future growth, surrounding empty space should also be available.

13

2.1 Site Space and Layout

According to ANSI/TIA-942, a datacenter should include the following key functional areas.* Entrance Room (ER) –

outside the computer room for better security.

14

Site Space and Layout (Cont.)

* Main Distribution Area (MDA) location of Main Cross-Connect (MC) such as routers and switches.

* Horizontal Distribution Area (HDA) location of Horizontal Cross-Connect (HC), such as racks for fiber and coaxial cable.

15

Site Space and Layout (Cont.)

* Zone Distribution Area (ZDA) location of Zone Outlet (ZO) or Consolidation Point (CP)

* Equipment Distribution Area (EDA) location of equipment cabinets and racks. 16

Site Space and Layout (Cont.)

17

2.2 Cabling Infrastructure

Horizontal Cabling * Should install the highest capacity media available to reduce the need for re-cabling

in the future. * The maximum horizontal distance is 90 m.

independent of media type.* The maximum channel distance

including equipment cords is 100 m.

18

Horizontal C abling Distances (Cont.)

* The maximum backbone fiber optic cabling is 300 m.

* The horizontal copper cabling is 100 m.

19

Cabling Infrastructure (Cont.) Computer Room & Entrance

Room Requirementssuch as * Minimum clear height of 2.6m/8.5 ft* Minimum door size 1m/3ft wide 2.13/7ft high

The ANSI/TIA-942 includes 4 levels, each specifying details: * Architectural* Security* Electrical* Mechanical* Telecommunication

20

2.3 Tiered Reliability

Tier descriptions include information on* Raised floor heights* Watts per square foot* Points of failure

21

Tiered Reliability (Cont.)

Tier 1 – Basic: 99.671% Availability* Single path for power and cooling distribution* Optioned raised floor, UPS, generator* Takes 3 months to implement* Annual downtime of 28.8 hours* Must be shut down completely

for preventive maintenance

22

Tiered Reliability (Cont.)

Tier 2 – Redundant Components: 99.741% Availability* Less susceptible to disruption

from both planned and unplanned activity* Single path for power and cooling distribution * Includes raised floor, UPS, and generator

23

Tiered Reliability (Cont.)

* Takes 3 to 6 months to implement* Annual downtime of 22.0 hours* Maintenance of power path and other parts of

the infrastructure require a processing shutdown 24

Tiered 2 (Cont.)

Tier 3 – Concurrently Maintainable: 99.982% Availability* Enables planned activity without disrupting computer hardware operation

but unplanned events will still cause disruption* Multiple power and cooling distribution paths

but with only one path active

25

Tiered Reliability (Cont.)

* Takes 15 to 20 months to implement* Annual downtime of 16.0 hours* Includes raised floor and sufficient capacity

and distribution to carry load on one path

while performing maintenance on the other

26

Tiered 3 (Cont.)

Tier 4 – Fault Tolerant: 99.995% Availability* Planned activity does not disrupt critical load

and data center can sustain

at least one worst-case unplanned event

with no critical load impact

27

Tiered Reliability (Cont.)

* Multiple active power and cooling distribution paths includes redundant components,

i.e. 2 UPS each with N+1 redundancy* Takes 15 to 20 months to implement* Annual downtime of 0.4 hours

28

Tiered 4 (Cont.)

Several environmental considerations exist within the ANSI/TIA-942 datacenter standard that are* Fire suppression* Humidity levels* Operating temperatures* Architectural* Electrical (power)* Mechanical system specifications

29

2.4 Environmental Consideration

30

3. Datacenter Best Practices.

3.1 Microsoft’s Environmentally Sustainable Datacenters

3.2 IRIDE Environmentally Friendly Datacenter

3.3 Merger & Acquisition-Driven Datacenter

3.4 Datacenter Assessment and Design

31

3.1 Microsoft’s Environmentally

Sustainable Datacenters. Microsoft prepared top ten best business practices for environmentally sustainable datacenters.

32

Microsoft’s Practices (Cont.)

1. Provide incentives that support primary goals.

Microsoft provides specific incentives to reward managers for improving the efficiency of their operations, using metrics such as Power Usage Effectiveness (PUE).

33

Microsoft’s Practices (Cont.)

Incentives help achieve results in short time if apply properly.* Datacenter manager rewarded for efficiency

and not uptime.* Incentives given for low energy usage.

34

Microsoft’s Practices (Cont.)

2. Focus on effective resource utilization.

Using resources effectively.

Utilizing existing datacenter infrastructure.

35

Microsoft’s Practices (Cont.)

3. Use virtualization to improve server utilization and increase operational efficiency.

Migrating application from physical to virtual machines.

Encourage cloud operating system which use virtualization.

36

Microsoft’s Practices (Cont.)

Key benefits of virtualization. * Reduction in capital expenditures.* Decrease in real estate, power,

and cooling costs. * Faster time to market for new products

and services. * Reduction in outage and maintenance windows.

37

Microsoft’s Practices (Cont.)

4. Drive quality up through compliance.

There are many regulatory and security standards.

Compliance must be strictly enforced.

38

Microsoft’s Practices (Cont.)

5. Embrace change management.

Standardized procedures for the request, approval, coordination and execution of changes can greatly reduce the number and severity of unplanned outages.

If not well managed, poorly planned changes may cause disastrous results.

39

Microsoft’s Practices (Cont.)

Other features that change management process should include:* Documented policies around communication

and timeline requirements. * Standard templates for requesting, communicating, and reviewing changes.* Post-implementation review,

including cases where things went well.

40

Microsoft’s Practices (Cont.)

6. Invest in understanding the application workload and behavior.

Require hardware engineering.

Credible and competent in-house expertise.

Provide meaningful feedback to vendors.* To evaluate new hardware properly* Optimizing request for proposal (RFP)

process for servers* Experiment new technology.

41

Microsoft’s Practices (Cont.)

7. Right-size server platforms to meet application requirements.

A major initiative in Microsoft datacenters involves right-sizing the platform.

42

Microsoft’s Practices (Cont.)

There are two forms:(1) Work closely with server manufacturers

to optimize their designs and remove items

that are not used. Focus on high

efficiency power supplies and advanced power

management features.

43

Microsoft’s Practices (Cont.)

(2) Being disciplined about developing the exact specifications for servers to meet the needs, and then not buying machines that exceed specifications.

44

Microsoft’s Practices (Cont.)

8. Evaluate and test servers for performance, power,

and total cost of ownership.

Test on short list candidate servers to calculatetotal cost of ownership.

Not rely on benchmark data, which may not be applicable to needs and environment.

45

Microsoft’s Practices (Cont.)

9. Converge on as small a number of stock-keeping units (SKUs) as possible.

Narrowing the number of SKUs allows Microsoft to make larger volume buys, thereby cutting capital costs.

10. Take advantage of competitive bids from multiple manufacturers

to foster innovation and reduce costs.

Microsoft encourages to analyze proposals from multiple companies that puts most of the weight on price, power, and performance.

46

Microsoft’s Practices (Cont.)

47

Microsoft’s Practices (Cont.)

Competition among multiple manufacturers to drive innovation and provide the most energy efficient & lowest cost.

48

Microsoft’s Practices (Cont.)

Microsoft’s Global Foundation Services’ team was taking significant steps in four areas important to environmental sustainability: * Using recycled resources whenever practical* Using renewable resources whenever available* Reducing waste in operations* Taking part in industry environmental groups

49

Microsoft’s Practices (Cont.)

Microsoft announced in March 2009 that it was taking a proactive corporate approach to reduce its carbon emissions per unit of revenue by at least 30 percent.

50

3.2 IRIDE Environmentally Friendly Datacenter.

Producing electrical power & thermal energy from renewable sources (Hydroelectric).

Two hundred virtual machines on only fifteen physical machines and cost savings up to 70 percent.

Reduce the number of physical servers and improve flexibility.

Using Intel Xeon processors with four cores to meet the need for balance and modularity.

Intel Xenon also provide high performance and energy efficiency.

Intel Multi-core technology has brought several important improvements for IRIDE.

51

IRIDE (Cont.)

Business Challenge* An Insurance holding company

needed to move the IT assets it had acquired

into its own data center.

According to the terms of the acquisitions,

it had to meet a tight deadline in doing so

or must pay penalty.

52

3.3 Merger & Acquisition-Driven Datacenter.

Solution* Establish a parallel operating environment

and migrate the applications and data* Assesses all elements of the infrastructure, networking, backups and restores, security

and systems monitoring, etc.* Integrate the business unit smoothly step by step

53

Insurance Holding Company (Cont.)

Results* The powering down of the new environment, relocation of the equipment,

re-establishment of the environment

in the buyer’s datacenter,

successful cutover done over the weekend

54

Insurance Holding Company (Cont.)

A major book, music and home entertainment. datacenter was* Approaching capacity* Lack a backup generator* Lack comprehensive power source* In a tornado zone 55

3.4 Datacenter Assessment and Design.

Solution * Detailed assessment of datacenter strengths

and weaknesses* Gather information about history

> What the datacenter was designed for

> How demands had grown

> Actual IT utilization > IT asset inventory

56

Datacenter Assessment and Design (Cont.)

Results* A Tier II datacenter operational on time,

on budget and without business interruption.

57

Datacenter Assessment and Design (Cont.)

4. Green Datacenter

58

There are many examples of green datacenters4.1 Amazon Green Datacenter 4.2 Google Green Datacenter4.3 Sun Green Mobile Datacenter4.4 Fujitsu Green Datacenter4.5 Datacenter in Second Life

4.1 Amazon Green Datacenter

59

From “www.greenm3.com”

dated 1 November 2008,Datacenter Knowledge reported on Amazon.com’s new green datacenter in Oregon, USA.

Amazon Green Datacenter (Cont.)

60

The Amazon Datacenter is located on the bank of the Columbia River in Oregon.

The cost of the datacenter was at US$ 100 million.

Amazon Green Datacenter (Cont.)

61

It includes three buildings. The first building will be

116,000 square feet. The hydro electricity on

Columbia River basin uses cheap and clean power.

4.2 Google Green Datacenter

62

From “www.internetnews.com” dated 3 October 2008,Google, the world largest search company announced how its squeeze considerable energy saving out of its datacenter.

Google Green Datacenter (Cont.)

63

The purpose of Google was to reduce datacenter power costs by targeting the center, not the computer.

Google Green Datacenter (Cont.)

64

There are five elements of Google’s strategyfor building management:* Clean Energy* Server Retirement* Efficient Equipment

* Water Management* Efficient datacenter

Google Green Datacenter (Cont.)

65

It could save about $30 per server per year.

Google removes unused componentssuch as sound and discrete graphics.

Google Green Datacenter (Cont.)

66

Evaporative cooling has been used instead of refrigerators.

Cold water * Run thru the datacenter to absorb the heat

from the computer* Run down from the roof to the ground outside the datacenter, similar to at restaurant, to help let the heat dissipate

4.3 Sun Mobile Datacenter

67

From “blog.internetnews.com” dated 18 September 2008, Sun Microsystems * stored the datacenter

in a mine deep underground in Japan

where the climate is cool

Sun Mobile Datacenter (Cont.)

68

* The datacenter consists of Sun systems densely packed in a standard shipping container, 20x40 feet

* The datacenter hold 240 rack unitswhich could be large servers such as SPARC Enterprise M4000 and M5000 or third-party equipments

69

Sun Mobile Datacenter (Cont.)

* The datacenters contain not only the systembut also the power and cooling necessary to operate the system

* One of the most famous examples is 100 meters down an abandoned mine,relying on the natural cool climate

4.4 Fujitsu Green Datacenter

70

From “www.computerweekly.com” in June 2008, Fujitsu has launched Green Infrastructure Solution Service to provide specialists with expertise in both IT and construction to assist customers with planning, design, employment, and operation, as well as building “highly energy-efficient datacenters” or “Green Datacenters”.

Fujitsu Green Datacenter (Cont.)

71

The efficient IT service aims to reduce power consumption and CO2 emission by as much as 50% of the previous level, excluding power savings from IT product themselves.

Fujitsu Green Datacenter (Cont.)

72

Fujitsu said the green datacenter is nearly twice as energy-efficientas previous datacenters it has opened.

The datacenter uses advanced cooling, heating,

and IT systems to tailor power consumption to processing and operational needs.

4.5 Datacenter in Second Life

73

From “www.informationweek.com”dated 14 August 2008, IBM launched “Eco Datacenter” in Second Life.

Visitors who want to check out IBM’s latest eco-datacenter have to do as an avatar in Linden Lab’s Second Life, where the facility “exist”.

Datacenter in Second Life (Cont.)

74

The datacenter is used as a demonstration area for IBM’s eco-friendly technologies and services.

5. Datacenter Success Stories

75

There are many examples of successful datacenters5.1 UC Berkeley, 5.2 IBM’s Boulder Facility5.3 BAA Inc.

5.1 UC Berkeley

76

UC Berkeley’s Datacenter consisting of nearly 40 years of accumulated technology legacy while minimizing the impact on the more than 40,000 members of the UC Berkeley community dependent on those systems.

UC Berkeley (Cont.)

77

The space design process for a datacenter is very complex and is, in fact, the first critical step to ensure success.

UC Berkeley (Cont.)

78

A detailed analysis of all of the systems and the technology trends that would impact future designs was necessary to design a space flexible enough to handle legacy systems as well as whatever new computers would be installed in future years.

UC Berkeley (Cont.)

79

The information gained from that process was provides to the architects and systems planners to crate a leading-edge datacenter design that could be connected to the old datacenter via fiber and high-speed connections to allow for electronic “moving” where possible.

5.2 IBM’s Boulder Facility

80

IBM completed the installation of a 72,000 square foot raised floor datacenter at the Boulder facility and received Silver LEED certification

“LEED” stands for “Leadership in Energy and Environmental Design”

IBM’s Boulder Facility (Cont.)

81

IBM engaged the services of Texas A&M University, Energy Systems Laboratory, for two phases

of commissioning work at the IBM facility.

Commission is the process in which mechanical equipment is analyzed and operating specifications are modified to optimize energy efficiency.

IBM’s Boulder Facility (Cont.)

82

The first survey was performed on the Central Utility Plant equipment.

Recommendations from this survey were:* Operate the backup satellite chiller plant

as the datacenter primary cooling system

instead of the Central Utility Plant

IBM’s Boulder Facility (Cont.)

83

* Automate the chiller and the free-cooling control systems

* Eliminate three way valves in chilled water systems

* Improve boiler efficiency with operational changes

IBM’s Boulder Facility (Cont.)

84

In a second commission effort 70 air handlers were surveys and their operation specifications were modified to include:* Broadening of economizer controls

which utilize outside air for free cooling* Implementing static pressure resets

for cooling airflow which is dependent

on outside air temperature

IBM’s Boulder Facility (Cont.)

85

* Implementing static pressure night setback to reduce airflow for overnight operations

* Chilled water pump operation for start/stop control based on required demand

* Control program changes which eliminate the opportunities for simultaneous heating and cooling

5.3 BAA Inc.

86

Airport titan BAA needed to consolidate its databases and servers, as well as improve the IT environment’s use of space and energy.

Furthermore, the company needed the ability to add more processing power in the future without disrupting business critical services.

BAA Inc. (Cont.)

87

BAA hosts 141 million passengers through its airport in the United Kingdom. With growing numbers of travels and airlines using its facilities. The world’s largest airport operator strives for improve efficiency and customer service

BAA Inc. (Cont.)

88

BAA IT staff must make the best use of space and save energy, which are challenges due to the heat generated, power required, and real estate consumed by numerous datacenter servers.

BAA Inc. (Cont.)

89

BAA IT needed to consolidate about 80 servers running 135 separate Microsoft SQL Server database that contain departmental, parking, environmental, regulatory, financial, and security information.

6. Concluding Remarks.

90

Datacenters are required to keep organization running 24/7.

Standards must be adopted.

Best practices should be followed.

Green datacenter should be implemented.

Success stories should be examined for the benefits of all concerned.