Huawei FusionCube Technical White Paper · Huawei FusionCube Converged Infrastructure is a flagship...
Transcript of Huawei FusionCube Technical White Paper · Huawei FusionCube Converged Infrastructure is a flagship...
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
i
Copyright © Huawei Technologies Co., Ltd. 2013. All rights reserved.
No part of this document may be reproduced or transmitted in any form or by any means without prior
written consent of Huawei Technologies Co., Ltd.
Trademarks and Permissions
and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd.
All other trademarks and trade names mentioned in this document are the property of their respective
holders.
Notice
The purchased products, services and features are stipulated by the contract made between Huawei and
the customer. All or part of the products, services and features described in this document may not be
within the purchase scope or the usage scope. Unless otherwise specified in the contract, all statements,
information, and recommendations in this document are provided "AS IS" without warranties, guarantees or
representations of any kind, either express or implied.
The information in this document is subject to change without notice. Every effort has been made in the
preparation of this document to ensure accuracy of the contents, but all statements, information, and
recommendations in this document do not constitute a warranty of any kind, express or implied.
Huawei Technologies Co., Ltd.
Address: Huawei Industrial Base
Bantian, Longgang
Shenzhen 518129
People's Republic of China
Website: http://enterprise.huawei.com
Technical White Paper on Huawei FusionCube Contents
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
ii
Contents
1 Introduction.................................................................................................................................... 1
2 FusionCube Solution .................................................................................................................... 2
2.1 System Architecture ......................................................................................................................................... 2
2.1.1 Reliable FusionCompute Cloud Platform ............................................................................................... 5
2.1.2 Automatic O&M ..................................................................................................................................... 6
2.1.3 Hardware Convergence ........................................................................................................................... 7
2.1.4 Integration Before Delivery .................................................................................................................... 8
2.1.5 Intelligent Resource Scheduling ............................................................................................................. 9
2.1.6 Elastic Capacity Expansion ................................................................................................................... 12
2.2 High-Performance Distributed Storage System ............................................................................................. 13
2.2.1 High-Performance Snapshots ................................................................................................................ 13
2.2.2 High-Performance Linked Clone .......................................................................................................... 14
2.2.3 Thin Provisioning .................................................................................................................................. 15
2.3 Network Reliability Design ............................................................................................................................ 16
2.3.1 Connection in Layer-2 Mode ................................................................................................................ 16
2.3.2 Connection in Layer-3 Mode ................................................................................................................ 17
3 Computing and Storage Integrated Hardware ...................................................................... 19
3.1 E9000 Subrack ............................................................................................................................................... 19
3.2 E9000 Blade ................................................................................................................................................... 20
3.3 iNIC Acceleration ........................................................................................................................................... 22
3.4 High-Performance Switching ......................................................................................................................... 23
4 FusionCube Configuration ....................................................................................................... 25
4.2 E9000+FusionStorage V50 ............................................................................................................................ 26
4.3 E9000+FusionStorage V100 .......................................................................................................................... 27
4.4 E9000+FusionStorage VX ............................................................................................................................. 28
Technical White Paper on Huawei FusionCube 1 Introduction
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
1
1 Introduction
Overview
Today, more and more enterprises are using virtualization and cloud computing technologies
to build their IT systems. Though being able to provide much higher operation efficiency than
traditional IT systems, cloud computing still encounters the following challenges:
Virtual platform deployment and management is complicated, which pushes operation
and maintenance (O&M) costs up.
The planning, deployment, and optimization skill requirements on operation personnel
are high because hardware devices may be provided by different vendors.
After-sales problems must be handled on operation portals of different vendors.
Therefore, the response to problems may be slow.
The maintenance system is huge because hardware and virtualization platforms from
different vendors are involved in management and maintenance.
The cloud computing has advantages only in large-scale database virtualization
scenarios.
Customers attach importance to cost control, rapid service provisioning, and risk management.
They want a resource-scalable, reliable, and high-performance system with as lower total cost
of ownership (TCO) as possible.
Huawei FusionCube Converged Infrastructure (FusionCube) solution integrates computing,
storage, and network resources and features high system performance, automatic and rapid
service deployment, reliable and secure service operation, unified operation and maintenance,
and intelligent resource scaling. These features satisfy customers' long-term service
development requirements and help them win rigorous market competitions.
This document describes the architecture, software and hardware, and product portfolios of
the FusionCube.
Intended Audience
This document is intended for the following audience:
Sales engineers
Channel distributors
Service managers
FusionCube customers
Technical White Paper on Huawei FusionCube 2 FusionCube Solution
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
2
2 FusionCube Solution
2.1 System Architecture
Figure 2-1 shows the architecture of Huawei FusionCube system.
Figure 2-1 Architecture of Huawei FusionCube system
Huawei FusionCube Converged Infrastructure is a flagship product of Huawei's IT product
line. The FusionCube uses 12 U subrack and integrates distributed storage engine software,
virtualization platform, and cloud management software.
Technical White Paper on Huawei FusionCube 2 FusionCube Solution
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
3
A subrack houses blade servers, distributed storage devices, and switches, requiring no
external storage device or switch. Virtual resources provided by the FusionCube can be
provisioned or expanded on demand.
A FusionCube system consists of the following components:
FusionManager: A core component for cloud platform management. It provides a unified
web-based user interface for performing service operations including user management,
cluster management, service template management, VM management, desktop service
provisioning, unified hardware management, alarm reporting, and system monitoring.
FusionCompute: A virtualization software. It converts physical resources, such as
computing, storage, and memory resources, into dynamically-adjustable virtual resources
for VMs to use.
FusionStorage: A storage software product. It uses distributed cloud storage technologies
to provide high-performance high-reliability block storage services by scheduling local
hard disks on servers.
Hardware: The FusionCube uses E9000 servers that have built-in computing, storage,
and network resources. Computing, storage, switch, and power modules of an E9000
server are designed as separated modules so that these modules can be flexibly
configured in a subrack based on service requirements.
An E9000 server supports I/O acceleration by adding more components, such as the
graphics processing unit (GPU), solid-state drive (SSD) PCIe, and digital signal
processor (DSP). It also supports switch modules running on various interfaces,
including gigabit Ethernet (GE), 10GE, fiber channel (FC), fiber channel over Ethernet
(FCoE), and InfiniBand (IB).
The FusionCube applies to the following service scenarios:
Virtualization converged infrastructure: The required hardware and software are
integrated in the FusionCube system, and no other application software runs on it.
Desktop cloud converged infrastructure: Virtual Desktop Infrastructure (VDI) or
virtualization software runs on the FusionCube to provide desktop access services.
Enterprise OA converged infrastructure: Enterprise office automation (OA) software,
such as Microsoft Exchange and SharePoint, runs on the FusionCube to provide OA
services for enterprise users.
Database or big data systems: Database software, such as Microsoft SQL Server and
SAP HANA, runs on the FusionCube to maximize the database performance using
hardware optimization technologies.
Integrating Huawei-developed hardware platform, distributed storage, and virtualization
software, the FusionCube has the following features:
Easy to use
Hardware devices, including servers, storage devices, and switches, are installed in one
FusionCube subrack or rack before delivery, facilitating sales and maintenance.
Rapid service deployment
The FusionCube offers rapid service deployment and flexible IT resource allocation
because of its following characteristics:
− A universal hardware platform accelerates planning, procurement, and deployment of
IT systems.
Software is decoupled from hardware and is compatible with general servers.
− Resources are combined in a pool for shared use, improving resource utilization.
Technical White Paper on Huawei FusionCube 2 FusionCube Solution
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
4
Users do not need to plan resources based on the peak traffic demands. This helps
customers reduce the initial investment and prevent a waste of resources.
System capabilities can be elastically provisioned and released based on service
requests.
The average IT resource utilization rate increases from 10% to more than 60%.
− Dynamic resource allocation
The platform-based IT resources, pre-installation, and graphical system planning help
reduce system deployment time.
The FusionCube dynamically allocates resources in the IT resource pool based on
service requirements, shortening the service provisioning time.
Clustered and automatic O&M system
The FusionCube transforms the network O&M mode from manual to automatic,
boosting the O&M efficiency by over 10 times and improving the O&M service quality.
The system provides this feature because of its following characteristics:
− The system automatically monitors IT resources and system running status, reports
system faults and potential risks in real time, and automatically rectifies faults and
clears risks, improving service reliability of the entire system.
− The system automatically allocates and dynamically adjusts IT resources based on the
resource allocation policy.
− New IT resources can be added quickly and automatically to the resource pool, and
original resources can be migrated from the resource pool as required.
End-to-end (E2E) solution
The FusionCube provides E2E solutions that involve the following components and
solutions:
− A large variety of hardware products, including servers, network devices, storage
devices, and security devices
− Comprehensive service software solutions
− Comprehensive O&M solutions
In conclusion, the FusionCube has advantages in:
Quickening service rollout, resource adjustment, and capacity expansion, and realizing
faster economic returns.
Improving resource utilization, reducing hardware, power, and O&M costs, and cutting
down the TCO.
Facilitating IT delivery and maintenance and reducing the manpower costs by
automatically adjusting resources and rectifying faults.
Technical White Paper on Huawei FusionCube 2 FusionCube Solution
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
5
2.1.1 Reliable FusionCompute Cloud Platform
Figure 2-2 shows the reliability architecture of the FusionCompute.
Figure 2-2 Reliability architecture of the FusionCompute
The FusionCompute uses Xen technologies to provide optimized VM management functions
throughout the VM lifecycle. With Xen's advantages in performance and security and using
chip-assisted virtualization technologies from Intel and AMD, the FusionCompute supports
virtualization requirements for key applications with high performance, reliability, security,
and adaptation.
On the infrastructure service layer, the FusionCompute provides enhanced functions in the
following aspects:
Computing virtualization: The system uses memory overcommitment and GPU
virtualization technologies to provide optimized performance and user experience.
Storage virtualization: The system uses linked clone and snapshot backup technologies to
help reduce hardware procurement costs.
Network virtualization: The system uses chip-assisted virtualization technologies such as
SR-IOV NIC passthrough to meet user requirements for high input/output (I/O)
performance.
On the applications service layer, the FusionCube provides the following functions to improve
system availability, maintainability, and security.
The VM live migration, VM hot backup, and virtual resource live expansion
technologies help reduce planned or unexpected system downtime and improve service
continuity.
The black box and guest OS fault detection functions help improve system
maintainability.
The VM security hardening, virtual local area network (VLAN), and security group
functions help enhance application security for enterprise users.
Technical White Paper on Huawei FusionCube 2 FusionCube Solution
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
6
The FusionCompute is a light-load virtualization system. It requires a maximum of 4 GB
space to be installed, and a FusionCompute cluster can contain up to 128 servers.
The FusionCompute supports various data storage devices, including small computer system
interface (SCSI) and VIMS storage devices based on block storage technologies.
The FusionCompute uses Huawei-developed intelligent network interface card (iNIC) and
distributed virtual switch to provide virtual networks.
VMs virtualized using the FusionCompute supports various guest OS versions, such as
mainstream Windows OSs and Linux OSs including Red Hat and SUSE. Virtual servers
running on the same physical server can work simultaneously with OSs of different types or
versions installed.
All these virtualization-based features provided by the FusionCompute help optimize key
application operations for enterprises.
2.1.2 Automatic O&M
The automatic O&M of the FusionCube is implemented by the FusionManager. The
FusionManager manages hardware devices, virtualization platforms, resource pools, VM data,
and applications. It also provides a unified portal for the life cycle management of various
resources in the system.
A FusionManager can be installed virtually on a SUSE Linux 11 (64 bits) server. When it is
deployed on a VM, other resources on its physical node can be shared with other service
VMs.
Based on the FusionCompute capabilities, the FusionManager provides the following
functions:
Unified Resource Management: The FusionManager aggregates the resources of different
systems and establishes a unified resource management model to provide unified resource
management, such as monitoring and alarm reporting on various resources.
Role-based Access Control (RBAC): RBAC applies to enterprise security management.
RBAC creates a role collection between users and rights. Each role has a group of rights.
Once a user is allocated a role, this user can perform all the operations authorized to the role.
With RBAC, you only need to allocate a role to a new user without allocating rights. Because
role right changes are fewer than user right changes, RBAC simplifies user right management
and reduces system costs.
Automatic Hardware Discovery and Capacity Expansion: The FusionManager
automatically discovers all hardware in the FusionCube and scans the hardware
specifications.
Users can select the discovered hardware when creating a resource pool. Then the
FusionManager automatically installs software for the hardware, configures the network, and
adds the hardware to the resource pool. During capacity expansion, the FusionManager also
automatically discovers new hardware and completes configuration when you add the
hardware to the resource pool.
Automatic Application Deployment: The FusionManager uses service templates to
automatically deploy applications, including creating VMs, installing an operating system
(OS), configuring a network, and installing applications.
Application Scaling: Users can set different resource policies for applications. The
FusionManager automatically adjusts resources for the applications based on the policies.
Technical White Paper on Huawei FusionCube 2 FusionCube Solution
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
7
VM Migration: When a VM in the system becomes faulty, it can be automatically migrated
to another physical server so that services are restored rapidly.
VM Snapshot: A snapshot is a reproduction of a VM at a specific moment. It can be used to
restore the VM to the state it was when the snapshot was taken. The snapshot records the
entire VM state, including the status of the power supply, memory, hard disk, network
interface card (NIC), and CPU register.
Dynamic Resource Scheduler (DRS): The DRS uses intelligent scheduling algorithms to
continuously monitor the resource utilization in the resource pool, flexibly schedule VM
resources based on configured policies, and dynamically balance load between physical hosts
in one resource cluster.
Data Backup and Restoration: The FusionCube uses an agent-free backup policy for VMs.
By deploying Huawei Hyper Data Protection (HyperDP) on the cloud platform, the system
administrator can use the web interface to easily back up VMs without installing any backup
software on VMs. The backup has no impact on VM operating.
When data loss occurs on a VM, the system administrator can use the local backup system to
rapidly restore the VM data.
Patch Upgrade: The FusionManager provides a unified patch upgrade tool.
2.1.3 Hardware Convergence
Figure 2-3 shows the internally integrated hardware of the FusionCube.
Figure 2-3 Integrated hardware of the FusionCube
Technical White Paper on Huawei FusionCube 2 FusionCube Solution
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
8
Huawei's FusionCube system uses a modular design so that users can customize product
profiles based on service requirements and do not need to assembly hardware components by
themselves onsite.
The computing, storage, and network devices integrated in a FusionCube system are all
developed by Huawei, which provides optimized infrastructure and application performance.
The converged platform also provides high-bandwidth and short-delay switching capabilities
complying with multiple switching protocols. Not only hardware, the FusionCube system also
integrates all the required software before the delivery, providing the following features:
Unified user interface: Users perform all operations, such as configuration, alarm
management, and performance statistics collection, related to computing, storage, and
network resources through a unified user interface, simplifying user-system interaction
and improving user experience.
One-stop services: Huawei provides E2E one-stop support for hardware, software, and
applications, ensuring that users can obtain optimal services from Huawei.
2.1.4 Integration Before Delivery
Huawei's FusionCube has all its components installed before delivery, simplifying onsite
installation and commissioning, and shortening the commissioning duration from several
weeks or even months to only hours.
The following measures implement FusionCube integration before delivery:
Hardware installation: Devices are installed in the cabinets and cables are bundled
properly.
Software installation: Required software including the following is installed in the
system: customized basic input and output system (BIOS), virtualization software
FusionCompute, virtualization management software FusionManager, and storage
management software FusionStorage or IP storage area network (IP SAN).
Integrated device commissioning: Before delivery, the FusionCube system will go
through an ordered-specifications compliance check and a connectivity check.
Cabinet shipment: The entire FusionCube device will be shipped in a cabinet.
After the FusionCube is delivered onsite, users can implement a hardware check,
ordered-specifications compliance check, connectivity check, and service configuration and
commissioning for the system. Then, the FusionCube can start to work.
Figure 2-4 shows the integration and delivery process of the FusionCube.
Technical White Paper on Huawei FusionCube 2 FusionCube Solution
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
9
Figure 2-4 Integration and delivery process of the FusionCube
2.1.5 Intelligent Resource Scheduling
The FusionCube supports on-demand allocation of computing, storage, and network resources,
improving resource utilization and saving energy.
Scheduling Actions
Resources that can be intelligently scheduled include CPU, memory, VMs, servers, and
resource clusters. Different resources use different scheduling actions, which are as follows:
CPU: dynamic frequency modulation, capacity expansion or reduction, resource
limitation, and physical CPU binding.
Memory: memory overcommitment, dynamic expansion or reduction, and resource
limitation
VM: automatic hibernation and wakeup, timed VM startup and stop, restart, forcible stop,
and migration
Server: power-off, timed power-off and power-on, and load balancing
Resource cluster: application migration and resource priority control
Scheduling Modes
Resources can be scheduled in the following modes:
Timed scheduling: The system starts scheduling tasks at the preset time based on the
configured scheduling policy. The following tasks can be implemented in timed
scheduling mode: VM migration, VM stop, VM start, VM migration, server power-off,
and server power-on.
Conditional scheduling: The system periodically detects resource running status based on
the configured scheduling policies. It starts scheduling tasks when the scheduling
conditions are met.
Scheduling policies can be flexibly configured based on the scheduling objects,
scheduling operation, and scheduling mode.
Technical White Paper on Huawei FusionCube 2 FusionCube Solution
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
10
The system supports creation, deletion, modification, activation, and deactivation of
scheduling policies.
Scheduling Technologies
The FusionCube system uses the following technologies to implement intelligent resource
scheduling:
Memory overcommitment
Memory overcommitment allows a host to provide more memory space to VMs than
what it physically provides. The commonly used memory overcommitment technologies
include memory ballooning, memory sharing, and memory swapping.
Based on the memory overcommitment policies, a host uses appropriate
overcommitment technologies to release its free memory space to more VMs and
balance the memory usage of these VMs.
In this manner, the total memory space used by all VMs served by a host is always larger
than the memory space the host physically provides. This is a method of improving VM
density in a FusionCube system.
The FusionCube supports memory overcommitment rate settings (0% to 50%) by logical
clusters.
Take an E9000 server as an example. If this server physically provides 48 GB memory
and it uses 7 GB space for memory management, the maximum memory space that can
be used by VMs is 61.5 GB.
(48 GB – 7 GB) x (1 + 50%) = 61.5 GB.
Virtual CPU (vCPU) quality of service (QoS) control technology
This technology limits the physical CPU usage of a vCPU so that the system can ensure
CPU resource allocation to VMs with higher QoS priorities.
When CPU resources are shared by multiple VMs running on the same physical server
and the VMs are running concurrently, the following settings help ensure resource
provisioning for higher-priority VMs:
− Number of CPUs used by a VM
− Maximum physical CPU resource that can be used by a vCPU. This limits resource
preemption when the physical CPU is shared by multiple VMs. For example, if the
maximum physical CPU resource usage of a vCPU is set to 60%, the VM cannot
consume more physical CPU resource when its vCPU already has used 60% of the
physical CPU resources.
A vCPU of a VM can also be bound to a physical CPU to prevent its CPU resources
from being preempted by other VMs.
The implementation process of this binding technology is as follows:
A physical CPU is virtualized into multiple vCPUs. If the vCPUs are allocated to one
VM, the VM uses the resources of the whole physical CPU exclusively. If the vCPUs are
allocated to multiple VMs, these VMs share the physical CPU resources.
If a vCPU of a higher-priority VM is bound to the physical CPU, the vCPU resource of
this VM will not be preempted by other VMs.
This resource assurance method has been included in the FusionCube's service-level
agreement (SLA).
Intelligent resource scheduling for load balancing
Technical White Paper on Huawei FusionCube 2 FusionCube Solution
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
11
The FusionCube system uses an intelligent resource scheduling algorithm to periodically
check the load on each physical host in a cluster and migrate VMs among hosts as
needed. This achieves load balancing among hosts in the cluster.
The implementation process of this function is as follows:
The system periodically monitors the CPU usage and memory usage of all physical hosts
and VMs in a cluster and compares the current load with the configured scheduling
threshold. If the load of 50% of all checked hosts and VMs is greater than the threshold
or meets the resource scheduling requirements, the system migrates VMs among hosts
based on the scheduling policies.
Intelligent power management
The system periodically checks the load of physical hosts in a resource cluster. When
detecting that the load of a host is heavy, the system powers on a new host to share the
load of existing hosts.
When detecting that the existing host load is light, the system migrates VMs on a
light-load host and powers off this host. This ensures that the power usage of the entire
resource cluster is always efficient.
Automatic resource sharing (ARS)
If a resource cluster serves multiple service groups, the system allocates idle resources in
a service group to other groups based on the configured service group priority.
In addition, the system can add or reduce resources in one service group at different time
periods based on scheduling policies.
This technology is also called time-division resource overcommitment technology
because resources can be used by different services in different time periods.
When resources are insufficient, the system reclaims resources based on the service
group priority and resource share. This ensures resource provisioning to high-priority
and high-resource-share services.
Resource overcommitment can only be implemented for service groups in the same
logical cluster. Therefore, the minimum unit for resource overcommitment is a VM.
Advantages of the ARS Technology
The resource overcommitment technology has the following advantages:
Resources can be shared among applications, reducing customers' IT investment cost.
Resources can be flexibly and automatically expanded or reduced according to service
requirements.
Resource overcommitment polices can be flexibly configured based on multiple rules.
Resources can be adjusted automatically.
Application Scenario of the ARS Technology
Resource overcommitment allows virtual desktop services and other services to use system
resources at different time periods.
For example, a user uses a virtual desktop dealing with office work at daytime and releases
the occupied computing resources at night. Then, the system can use the released computing
resources to process other services, such as continuous integration (CI) and supercomputing,
at night. After finishing these services, the system releases the computing resources again for
virtual desktop users to use the next day.
This technology greatly improves the resource utilization of the entire FusionCube system.
Technical White Paper on Huawei FusionCube 2 FusionCube Solution
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
12
2.1.6 Elastic Capacity Expansion
The cloud platform supports elastic capacity expansion based on service requirements without
interrupting running services.
If the cluster resources are insufficient, users can add computing or storage resources to
the cluster.
If the servers are insufficient, users can add computing servers to the cluster.
If the storage resources are insufficient, users can add storage blade servers or
independent storage disks to the cluster.
The entire FusionCube solution can be designed with a certain amount of redundant capacity
so that system resources can be smoothly expanded when more VMs are required.
Huawei's cloud platform is based on a distributed expandable architecture, providing
independent computing pool by clusters. When the host resources in a cluster are insufficient,
the system adds blade servers, subrack, or cabinets to the system and connects the added
devices to the cloud network. Using a hardware automatic discovery function, the cloud
platform can manage new devices immediately after discovering them.
The cluster expansion is implemented in a linear on-line mode. For each cluster, the
computing and storage resources are independent, but the O&M and service provisioning are
implemented by one system in a unified manner.
Technical White Paper on Huawei FusionCube 2 FusionCube Solution
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
13
2.2 High-Performance Distributed Storage System
High-performance becomes a key requirement on enterprise storage systems. However,
traditional storage systems cannot meet the demanding storage requirements of increasing
data through performance expansion. Moreover, high-end storage devices that serve as
independent NEs are not only expensive but also need dedicate management and
maintenance.
By deeply integrating computing and storage, the FusionStorage delivers perfect performance,
sound reliability, and high cost-effectiveness. It can be deployed on a general server to
combine local disks on all servers into a virtual storage resource pool. Therefore, the
FusionStorage can completely take the place of external storage area network (SAN) device in
some scenarios.
Innovative architecture and design: The innovative architecture empowers the
FusionStorage to achieve optimal performance. This performance optimization allows you to
fully use system resources and automatically allocates workload among all hard disk drives.
In addition, the FusionStorage supports a series of advanced functions, such as thin
provisioning and snapshot, without deteriorating system performance.
Consistent and predicable performance expansion: The load balancing among all disks
and distributed cache structure of the FusionStorage allow you to expand system performance
linearly and smoothly by adding server nodes. The FusionStorage ensures consistent high
performance at peak and valley hours regardless of the change of the service amount and
snapshot mode or even though the component becomes faulty.
Flexibility and self-recovery: The FusionStorage retains high flexibility and runs properly
even if hardware becomes faulty. In addition, the self-recovery capability empowers the
FusionStorage to defend against more hardware faults that may occur after its initial recovery
from failures.
Deep integration of computing and storage: The FusionStorage integrates the local disks on
all servers as a virtual storage resource pool and delivers automatic management, high
running performance, and out-of-the-box packaging for quick deployment. The FusionStorage
does not serve as an independent NE for dedicate configuration or management. In this way,
resources on the computing nodes can be fully used, which not only provides users with high
storage performance, but also reduces costs.
2.2.1 High-Performance Snapshots
The FusionStorage provides the snapshot mechanism and stores the status of the data stored
on the logical volumes for possible data export and restoration.
The FusionStorage creates snapshots based on the distributed hash table (DHT) technology.
Creating snapshots does not have any adverse impact on the volumes. For example, 24 MB
space is required for constructing 2 million keys in the memory for a 2 TB hard disk. You can
determine whether any snapshot has been created for the disk in a hash query operation. If a
snapshot has been created, the hash query operation enables you to determine the storage
location of the snapshot.
The snapshot counts depend on the partition capacity. The FusionStorage supports infinite
snapshots because the partition capacity is not restricted.
Figure 2-5 shows a snapshot working model supported by the FusionStorage.
Technical White Paper on Huawei FusionCube 2 FusionCube Solution
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
14
Figure 2-5 High-performance snapshots provided by the FusionStorage
2.2.2 High-Performance Linked Clone
The FusionStorage provides the linked clone mechanism for incremental snapshots and
creates multiple cloned volumes for a snapshot. The data in the cloned volumes is the same as
that in the snapshot. Subsequent modification to a cloned volume does not affect the snapshot
or other cloned volumes.
The FusionStorage supports linked clone rate of 1:256 and improves utilization of storage
space.
The FusionStorage supports batch volume creation for VMs and can create hundreds of
volumes in seconds.
A cloned volume inherits all the functions of a common volume. You can create snapshots for
a cloned volume, use the snapshot to restore the data in the cloned volume and clone the data
in the cloned volume.
Figure 2-6 shows a working process of linked cloning supported by the FusionStorage.
Technical White Paper on Huawei FusionCube 2 FusionCube Solution
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
15
Figure 2-6 Linked cloning that the FusionStorage supports
2.2.3 Thin Provisioning
The FusionStorage uses the thin provisioning mechanism and provides virtual storage
resources more than physical storage resources, improving storage utilization.
The FusionStorage uses the distributed hash technology and supports automatic thin
provisioning.
Thin provisioning effectively avoids data migration that is required for external storage
devices.
Figure 2-7 shows a working process of thin provisioning supported by the FusionStorage.
Figure 2-7 Thin provisioning that the FusionStorage supports
Technical White Paper on Huawei FusionCube 2 FusionCube Solution
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
16
2.3 Network Reliability Design
The system networks are logically divided into the service plane, storage plane, and
management plane, which are isolated to protect the basic platform.
Storage plane: The storage devices on the server communicate with each other at layer 2 over
the storage network. Storage devices provide storage resources to VMs through the
virtualization platform but do not communicate with VMs.
Service plane: It provides a channel for users to obtain services, for virtual NICs of VMs to
communicate with each other, and for external applications to interact with the desktop cloud
system. Access can be separated by the VLANs configured for VMs.
Management plane: It provides such functions as system management, service deployment,
and system loading. The baseboard management controller (BMC) plane manages servers. It
is isolated from the management plane.
Each blade on the server is equipped with two 10 GE network ports for networking. The
service, management, and storage planes are aggregated through two NICs, ensuring link
redundancy. The E9000 server has two 10 GE switch modules in its subrack. They serve as
the access and aggregation switches for connecting to all server blades in the subrack and
cascading inter-subrack switch modules. By assigning VLANs on the switch modules, you
can logically isolate the management, service, and storage planes. In the subrack, the switch
modules connect to the customer network in the uplink.
On the FusionCube, the BMC plane is assigned to VLAN 4030, management plane to VLAN
4040, service plane to VLAN 4050, and service plane to VLAN 2-4000 by default. The subnet
and VLAN division for the management plane can be adjusted based on customers'
requirements.
The FusionCube provides unified services and management. The internal network of the
FusionCube follows the layer-2 network architecture so that the FusionCube can connect to
the customer network in either layer-2 or layer-3 mode.
2.3.1 Connection in Layer-2 Mode
In layer-2 mode, the subnet gateway is set to the customer's network, and only the switch for
the FusionCube serves as the access switch. This mode applies to scenarios in which
management and service planes must be isolated. Users can configure different subnets to
ensure secure isolation. Figure 2-8 shows layer-2 networking diagram.
Technical White Paper on Huawei FusionCube 2 FusionCube Solution
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
17
Figure 2-8 Connection in layer-2 mode
Networking description:
1. The switch for the FusionCube serves as the customer's access switch, on which the
uplink port, service VLAN, and management VLAN must be configured.
2. The IDs of service VLANs range from 2 to 4000.
3. The management VLAN ID is 4040, which can be modified.
4. VLANs whose IDs are greater than 4000 are used by the internal network only.
2.3.2 Connection in Layer-3 Mode
In layer-3 mode, the gateways of the management and service planes are both configured on
the FusionCube switch module so that the user network communicates with the FusionCube
through a layer 3 route. This mode applies to scenarios in which user networks must be
directly connected to the FusionCube. Figure 2-9 shows layer-3 networking diagram.
Technical White Paper on Huawei FusionCube 2 FusionCube Solution
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
18
Figure 2-9 Connection in layer-3 mode
Networking description:
An aggregated uplink port, a port IP address, and a static uplink route are configured on the
FusionCube.
Technical White Paper on Huawei FusionCube 3 Computing and Storage Integrated Hardware
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
19
3 Computing and Storage Integrated Hardware
3.1 E9000 Subrack
Tecal E9000 (E9000 for short) is a 12-U server developed by Huawei. It can house E9000
series computing nodes, service processing nodes, storage nodes, and capacity expansion
nodes. The E9000 server has the following features and functions:
Houses eight horizontal full-width computing nodes or 16 half-width computing nodes
and supports slot division.
Supports 850 W heat dissipation for half-width slots.
Supports 1700 W heat dissipation for full-width slots.
Supports a maximum of 2 CPUs+24 DIMMs in half-width slots.
Supports a maximum of 4 CPUs+48 DIMMs in full-width slots.
Supports 32 CPUs/256 cores on a single subrack, with a maximum of 12 TB memory.
Supports a maximum of 5.76 Tbit/s switching capacity on the backplane.
Provides four (two pairs) switching slots, supports multiple types of switching protocols,
such as Ethernet, Fibre Channel (FC), fiber channel over Ethernet (FCoE), and
InfiniBand (IB), and supports straight-out I/O interfaces.
Figure 3-1 shows the appearance of an E9000 server.
Figure 3-1 Appearance of an E9000 server
Technical White Paper on Huawei FusionCube 3 Computing and Storage Integrated Hardware
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
20
A FusionCube cabinet can house three E9000 subracks.
3.2 E9000 Blade
The FusionCube supports the following blades: 2-CPU CH121 computing blade, 2-CPU
CH222 computing+storage blade, 2-CPU CH220/CH221 computing I/O expansion blade, and
4-CPU CH240 computing blade.
Figure 3-2 shows the appearance of a CH121 computing blade.
Figure 3-2 Appearance of a CH121 computing blade
The CH121 computing blade has the following features:
1. Uses Intel® Xeon® E5-2600 quad-core/hexa-core/octa-core processors, supports all
product families, and provides 2.9 GHz octa-core computing capability.
2. Provides 24 DIMM slots and up to 768 GB DDR3 memory.
Figure 3-4 shows the appearance of a CH222 computing blade.
Figure 3-3 Appearance of a CH222 computing+storage blade
The CH222 computing+storage blade has the following features:
Technical White Paper on Huawei FusionCube 3 Computing and Storage Integrated Hardware
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
21
1. Uses two Intel® Xeon® E5-2600 processors.
2. Provides 24 DIMM slots and up to 768 GB DDR3 memory.
3. Supports a maximum of fifteen 2.5-inch SSD, SAS, or SATA disks.
4. Uses hot swappable architecture, supports in-service replacement of a single hard disk,
and provides one full-height 3/4-length PCIe x16 slot.
Figure 3-4 shows the appearance of a CH220/CH221 computing blade.
Figure 3-4 Appearance of the CH220/CH221 computing+storage blade
The CH220/CH221 computing+storage blade has the following features:
1. Uses two Intel® Xeon® E5-2600 quad-core/hexa-core/octa-core processors.
2. Provides 24 DIMM slots and up to 768 GB DDR3 memory.
3. Provides two PCIe x16 full-height full-length slots or four PCIe x8 full-height
half-length slots holding four GPU cards or SSDs.
Figure 3-5 shows the appearance of a CH240 computing blade.
Figure 3-5 Appearance of the CH240 computing+storage blade
The CH240 computing+storage blade has the following features:
1. Uses four Intel® Xeon® E5-2600 quad-core/hexa-core/octa-core processors.
2. Provides 48 DIMM slots and up to 1.5 TB DDR3 memory.
3. Supports eight 2.5-inch SSD, SAS, or SATA disks.
Technical White Paper on Huawei FusionCube 3 Computing and Storage Integrated Hardware
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
22
4. (Optional) Provides one full-height 3/4-length PCIe x16 slot.
3.3 iNIC Acceleration
The FusionCube uses Huawei-developed iNICs that provide the virtual switching capability
for hypervisors.
iNICs support various functions, such as multi-queue, virtual layer-2 switching, and quality of
service (QoS) and security control over virtual interfaces. iNICs apply to the following
modes:
Common mode: iNICs serve as common NICs to send and receive packets only.
Virtualization mode: iNICs bear the network capabilities of VMs, significantly reducing
the VM CPU load.
Advantages of iNICs:
Server virtualization functions, such as switching, ACL, QoS, can be processed by iNICs,
reducing the CPU load, enhances the I/O straight-through performance of the VM, and
prevents the network performance bottleneck of the host. When the Linux bridge is used,
the forwarding performance of a host core reaches 2 Gbit/s. If the iNIC passthrough
function is used, network processing services do not consume the CPU resource of the
host. Therefore, the forwarding performance reaches 10 Gbit/s during basic virtual
switching and 7.5 Gbit/s during full-service operation (including virtual switching, static
ACLs, and stateful ACLs).
iNICs integrate the functions of static ACLs and stateful ACLs and use high-performance
ACL matching algorithms, therefore improving the ACL matching performance.
Figure 3-6 shows a comparison between common NICs and iNICs.
Technical White Paper on Huawei FusionCube 3 Computing and Storage Integrated Hardware
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
23
Figure 3-6 Comparison between common NICs and iNICs
3.4 High-Performance Switching
The FusionCube uses the CX310 switch to support the 10 GE network. Each subrack is
configured with two switches. Figure 3-7 shows the appearance of a CX310 switch.
Figure 3-7 Appearance of the CX310
Technical White Paper on Huawei FusionCube 3 Computing and Storage Integrated Hardware
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
24
Performance indicators of the CX310 are as follows:
Table 3-1 Performance indicators of the CX310
Item Description
Number of ports on the panel One BMC serial port and one SYS serial port.
Sixteen 10 GE uplink optical ports.
Switching capacity 1.28 Tbit/s
Packet forwarding rate 960 Mpps
Key data center features TRILL
FCoE
DCBX, PFC, and ETS
VLAN Supports the Access, Trunk, and Hybrid modes.
Supports the default VLAN.
Supports QinQ and enhanced QinQ.
Supports the MUX VLAN.
QoS Supports flow classification based on the combination of
layer-2, layer-3, layer-4, and 802.1p protocols.
Supports ACLs, committed access rate (CAR), remark, and
traffic scheduling.
Supports various queuing modes, such as priority queue
(PQ), weighted round robin (WRR), deficit round robin
(DRR), PQ+WRR, and PQ+DRR.
Supports anti-congestion mechanisms, such as weighted
random early detection (WRED) and tail drop.
Supports traffic shaping (TS).
Security and management Provides hierarchical protection using commands,
preventing unauthorized access
Defends against Denial of Service (DoS), Address
Resolution Protocol (ARP), and Internet Control Message
Protocol (ICMP) attacks
Supports port isolation, port security, and sticky MAC
Supports binding between IP addresses, MAC addresses,
ports, and VLANs
Provides various authentication modes, such as AAA,
RADIUS, and HWTACACS
Supports remote monitor (RMON)
Technical White Paper on Huawei FusionCube 4 FusionCube Configuration
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
25
4 FusionCube Configuration
The FusionCube provides different product portfolios to adapt to various application scenarios.
Table 4-1 describes the application scenarios available to different FusionCube portfolios.
Table 4-1 Application scenarios available to different FusionCube portfolios
No. Product Portfolio Application Scenario
1 E9000
server+FusionStorage
storage (recommended)
Applies to high-performance applications to meet
infrastructure virtualization requirements.
2 E9000 server+IP SAN
storage
Applies to the converged infrastructure scenario in
which customers use the SAN device. The
configuration schemes for this scenario depend on
the actual project.
The E9000 server+FusionStorage application scenarios are V50 and V100. Moreover, the
FusionCube solution provides customers with a customizable VX scheme to meet various
service load and storage requirements of different data centers.
V50: half-subrack E9000 server, with no external switch.
V100: one E9000 subrack, with no external switch.
VX: E9000 servers ranging from a half subrack to eight subracks, with no external switch.
Table 4-2 lists the types of blades supported by the FusionCube.
Table 4-2 Physical components of the FusionCube that is composed of the E9000 server and the
FusionStorage
Blade Type Description
MCNA The Manage Compute Node Agents (MCNA) work in active/standby
mode and provide computing and storage resource pools. The
FusionManager, FusionStorage Manager, active and standby Virtual
Resource Manager (VRM) VMs, FusionStorage Agent (including the
metadata control process), and Computing Node Agent (CNA) run on the
MCNA.
Technical White Paper on Huawei FusionCube 4 FusionCube Configuration
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
26
Blade Type Description
LCNA The Lite Manage Compute Node Agent (LCNA) is a host supporting
light-load storage management and computing and providing computing
and storage resource pools. The FusionStorage Agent (including the
metadata control process) and CNA run on the LCNA.
SCNA The Storage Compute Node Agent (SCNA) is a host providing computing
and storage functions. The FusionStorage Agent (excluding the metadata
control process) and CNA run on the SCNA.
CNA The CNA is a host providing the computing function. The FusionStorage
Agent (excluding the metadata control process and object-based storage
process) and CNA run on the CNA.
4.2 E9000+FusionStorage V50
The deployment of the FusionCube in V50 mode (half-subrack E9000) is as follows:
Figure 4-2 Physical deployment in V50 mode
The specifications of this deployment mode are as follows:
Table 4-3 V50 configuration specifications
Specifications
Number of VMs 100 to 255
Subrack 1 x 12 U
Computing resource E5 2640 with eight processors (four blades)
Memory 512 GB (128 GB x 4)
Technical White Paper on Huawei FusionCube 4 FusionCube Configuration
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
27
Specifications
Network 10 GE
Storage capacity 7.2 TB SAS (valid capacity)
Virtualization and cloud
management
FusionSphere 3.X
Power consumption 1600 W
In the V50 configuration mode of the FusionCube, the specifications can be smoothly
expanded within the scope of the VX scheme. The number of VMs depends on the specific
service model.
4.3 E9000+FusionStorage V100
The deployment of the FusionCube in V100 mode (one E9000 subrack) is as follows:
Figure 4-3 Physical deployment in V100 mode
The specifications of this deployment mode are as follows:
Table 4-4 V100 configuration specifications
Specifications
Number of VMs 300 to 600
Subrack 1 x 12 U
Computing resource E5 2640 with 16 processors (eight blades)
Technical White Paper on Huawei FusionCube 4 FusionCube Configuration
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
28
Specifications
Memory 1024 GB (128 GB x 8)
Network 10 GE
Storage capacity 14.4 TB SAS (valid capacity)
Virtualization and cloud
management
FusionSphere 3.X
Power consumption 2700 W
In the V100 configuration mode of the FusionCube, the specifications can be smoothly
expanded within the scope of the VX scheme. The number of VMs depends on the specific
service model.
4.4 E9000+FusionStorage VX
In this portfolio, the FusionCube needs to be configured based on the site requirements
because the computing nodes can be either half-width or full-width blades. Figure 4-4 shows
the physical deployment (eight-subrack E9000 servers) using one-subrack full-width CH222
blades and seven-subrack half-width CH121 blades.
Figure 4-4 Physical deployment in VX mode
Technical White Paper on Huawei FusionCube 4 FusionCube Configuration
Issue 1.3 (2013-05-08) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
29
The specifications of this deployment mode are as follows:
Table 4-5 VX configuration specifications
Specifications
Number of VMs 100 to 3000
Subrack (1 to 8) x 12 U
Computing resource E5 26XX with 8 to 248 processors (maximal)
Memory 92.6 TB (maximal)
Network 10 GE
Storage Built-in distributed FusionStorage
Storage capacity 43.2 TB (maximal valid capacity, 96 900 GB SAS disks)
Virtualization and cloud
management
FusionSphere 3.X
Power consumption 5600 W for one subrack
Because blades can be either half-width or full-width, the number of blades also changes. In
this case, computing resources, memory resources, and storage resources will not reach the
upper limit at the same time.