Installing

IBM SmartCloud ProvisioningVersion 1.2

Installing and configuring

��

This edition applies to version 1 release 2 modification level 0 of IBM SmartCloud Provisioning (program number5725-C88) and to all subsequent releases and modifications until otherwise indicated in new editions.

© Copyright IBM Corporation 2011.US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contractwith IBM Corp.

Note

Before using this information and the product it supports, read the information under “Notices” on page 59.

Contents

Installing and configuring . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1Physical architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1Installing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Installation overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4Before you begin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4Installing using the GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5Installing using the CLI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9Verifying the installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21Restarting all services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22Shutting down a running environment . . . . . . . . . . . . . . . . . . . . . . . . . 24Recovering kernel services virtual machines after a restart of your cloud . . . . . . . . . . . . . . 25

Upgrading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25Upgrading using the GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25Upgrading using a script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26Upgrading HBase and other nodes using a script . . . . . . . . . . . . . . . . . . . . . . 27

Configuring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29Reconfiguring nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29Other configuration tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

Default ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49Securing the communication. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49Installing and configuring additional options . . . . . . . . . . . . . . . . . . . . . . . . 50

Installing topology management . . . . . . . . . . . . . . . . . . . . . . . . . . . 50Installing and configuring the Iaas API . . . . . . . . . . . . . . . . . . . . . . . . . 51

Appendix. Support information . . . . . . . . . . . . . . . . . . . . . . . . . 55Using IBM Support Assistant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55Obtaining fixes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55Receiving weekly support updates . . . . . . . . . . . . . . . . . . . . . . . . . . . 56Contacting IBM Software Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

Determine the business impact of your problem . . . . . . . . . . . . . . . . . . . . . . 57Describe your problem and gather background information . . . . . . . . . . . . . . . . . . 58Submitting problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

© Copyright IBM Corp. 2011 iii

iv IBM SmartCloud Provisioning: Installing and configuring

Installing and configuring

The following topics describe how to install and configure SmartCloudProvisioning.

The product file is CBH6H7ML.iso.

Physical architectureThis topic describes the topology of SmartCloud Provisioning.

There are two types of servers in a typical SmartCloud Provisioning environment:Compute Nodes and Storage Nodes. The Storage Nodes are optimized for diskand network I/O, while the Compute Nodes are optimized for CPU and RAM,although I/O is also critical. The Storage Nodes host all images and persistent uservolumes, while the Compute Nodes are used to host the virtual machines.

The SmartCloud Provisioning environment requires two or more Storage Nodesand two or more Compute Nodes, and it also requires a machine as a PXE Serverif you are going to use the automation installer. There are approximately 6:1(average ratio) Compute Nodes to Storage Nodes. This is a recommended ratiofrom practice, and you can adjust it according to your system workload.

SmartCloud Provisioning requires an isolated network because it has an internalDHCP server. You must configure your network to guarantee that the DHCPservice is not affected and does not affect your existing network.

The following figure shows a typical SmartCloud Provisioning environment:

© Copyright IBM Corp. 2011 1

SmartCloud Provisioning has the following components:

PXE ServerProvides DHCP, DNS, HTTP, and TFTP services. It is used to quickly setup all SmartCloud Provisioning components over the network.

ZookeeperZookeeper is part of the Apache Hadoop project. It is a scalable resilientdistributed lock service. It must be deployed in odd numbers of serverswith a minimum of three nodes, as shown in the figure.

Web ServiceAll administrative and user access to SmartCloud Provisioning is providedthrough a RESTful web service interface. At least one copy of this virtualmachine must be running. Multiple copies can run in parallel with a loadbalancer (virtual machine or physical device) spreading out http traffic tothe group.

Web ConsoleAt least one copy of the web console must be deployed to run the web UI.As with the web service, multiple copies can be run in parallel with afront-end load balancer.

LDAP ServerAn open LDAP provides user authentication for the Web Console. It'soptional and configurable, though if you follow the installation procedure

2 IBM SmartCloud Provisioning: Installing and configuring

document in this set it is always installed. If you have no LDAP server, theSmartCloud Provisioning installer can help you install an LDAP server.

Hadoop and HBaseHBase is part of the Apache Hadoop project. It is a distributed, scalable,column-oriented data store modeled after Google's BigTable. HBase hasdependency on HDFS (Hadoop Distributed File System). To guarantee highavailability of the service, the minimum recommended deployment is threecopies of this virtual machine, but it is highly recommended to have morenodes, for example, 7.

REST ServerThe REST Server is a gateway for the HBase cluster. All requests to HBaseare processed by this component. The minimum is 1 node, but you canhave more nodes for balance purpose.

Storage NodesThe Storage Nodes provide images and volumes services through the iSCSITarget Server. The default iSCSI Target Server is Linux tgtd. The minimumis 2 nodes. Each Storage Node is equipped with a storage bot. A storagebot is an agent that manages all storage-oriented resources and tasks.

Compute NodesThe Compute Nodes host the deployed virtual images. The minimum is 1node but, depending on your environment and forecasted workload, youcan have more nodes. Each Compute Node is equipped with a computebot. A compute bot is an agent that manages all computing-orientedresources and tasks.

Virtual Image LibraryThe Virtual Image Library component provides image managementservices such as, for example, federation among heterogeneous imagerepositories, comparison among images both pre installed products and perfiles, inventory, search, versioning, and replication.

Image Construction and Composition ToolThe Image Construction and Composition Tool helps you build images thatare reusable, self-descriptive, customizable, shareable, and manageable.Images can contain the basic operating system plus additional softwarebundles.

In a typical deployment:v The PXE Server is installed on a physical box, the so-called First Box, but it can

also be installed on a virtual machine.v The Storage Nodes are deployed on physical machines, because this improves

their performances, but, if you have limited hardware resources, they can also bevirtualized.

v The Compute Nodes are deployed on physical boxes.v If you choose VMware ESXi Server as system for the Compute Node, a

management virtual machine is created to host the compute bot on the VMwareESXi Server.

v The other components, including ZooKeeper, Hadoop plus HBase, REST Server,Web Service, Web Console and Administrative Console are all installed onvirtual machines. These virtual machines, often referred to as kernel services, canrun in any physical machines installed as KVM host, but the CPU of thephysical machines must support virtualization.

Installing and configuring 3

InstallingThe following topics deal with the installation of SmartCloud Provisioning and allof its nodes.

Installation overviewThe PXE Server is used to build the images for the other servers so that they canbe installed quickly over the network.

At a very high level, the installation is performed as follows:1. Set up the PXE Server using an automation script. This task sets up DHCP,

DNS, TFTP, HTTP, and NTP services on the PXE Server, which acts as a bootingand running environment for the cloud.

2. Install the Storage Nodes. This task installs the operating system on the nodeand then configures it as a Storage Node.

3. Install the Compute Nodes. This task boots the node through the network,loads the operating system image from the PXE Server and configures it as aCompute Node.

4. Install the kernel service virtual machines. This task deploys a few virtualmachines, each with a kernel service (for example, a Zookeeper, an HBase, anLDAP) that supports the cloud.

The installation can be performed in two ways:v Using the GUI on the PXE Server. This is the recommended approach to perform

an installation. It can be minimal or custom.v Using the CLI as described in the Installing and configuring set of topics, see

“Installing using the CLI” on page 9.

Before you beginComplete the following prerequisites before you start an installation.

Before starting an installation, ensure that you have configured your environmentfor PXE booting.

Before setting up the PXE Server, ensure that the following prerequisites are meton its physical machine:v Prepare an ISO file of Red Hat Enterprise Linux 6.0 or 6.1 x86_64, and place it

into a directory on the PXE Server, for example, place it in /tmp/ISCP. This file isused to make the kernel services images and to make Red Hat as an RPMrepository for the whole environment, and it is also used to install ramdisk forthe Storage Nodes and Compute Nodes. All these steps are automaticallyperformed during the installation, and no manual actions are needed beyondmaking an ISO file available in a directory on the PXE Server.

v Prepare the following packages and place them into a directory on the PXEServer, for example, place them in /tmp/ISCP. The following packages, requiredfor the installation, correspond to Red Hat Enterprise Linux 6.0 or 6.1, which canbe downloaded from the Red Hat website:for rhel 6.1:ruby-devel-1.8.7.299-7.el6.x86_64.rpmruby-rdoc-1.8.7.299-7.el6.x86_64.rpmrubygems-1.3.7-1.el6.noarch.rpmruby-ri-1.8.7.299-7.el6.x86_64.rpmruby-docs-1.8.7.299-7.el6.x86_64.rpm


for rhel 6.0:ruby-ri-1.8.7.299-4.el6.x86_64.rpmrubygems-1.3.7-1.el6.noarch.rpmruby-rdoc-1.8.7.299-4.el6.x86_64.rpmruby-devel-1.8.7.299-4.el6.x86_64.rpmruby-docs-1.8.7.299-4.el6.x86_64.rpmruby-irb-1.8.7.299-4.el6.x86_64.rpm

Do not put packages for RHEL 6.0 and RHEL 6.1 together.v Prepare the following packages and place them in a directory on the PXE Server,

for example, place them in /tmp/ISCP:livecd-tools-031-1.el6.1.x86_64.rpmpython-imgcreate-031-1.el6.1.x86_64.rpm

The livecd-tools package can be downloaded from: http://ftp.redhat.com/pub/redhat/rhel/beta/6/x86_64/os/Packages/livecd-tools-031-1.el6.1.x86_64.rpm.The python-imgcreate package can be downloaded from: http://ftp.redhat.com/pub/redhat/rhel/beta/6/x86_64/os/Packages/python-imgcreate-031-1.el6.1.x86_64.rpm.It is at your sole discretion to decide to download thesepackages or not. If you elect to download the packages, you must comply withthe licenses governing the use of the packages. IBM does not make anywarranties to the packages, including but not limited to fitness for a particularpurpose, merchantability and non-infringement. IBM does not indemnify youagainst claims of any third party that the packages infringe the third party'sintellectual property right.

v Ensure that iptables does not block http, SSH, TFTP, DHCP, DNS:chkconfig iptables off&&/etc/init.d/iptables stop

v Ensure that SELinux is disabled. You can disable SELinux in the following way:1. Open the file /etc/selinux/config in edit mode.2. Change SELINUX=enforcing to SELINUX=disabled.3. Reboot.

Installing using the GUIThese topics describe how you perform an installation using the graphical userinterface on the machine that you want to set up as a PXE Server.

The installation through the GUI is the recommended approach for this task.

Launching the installationMount the media (.iso format or CD) or extract the package (.tar format) to anydirectory on the PXE Server with at least 10 GB of disk space available.

For example, extract to $your_dir, then run .install.sh to start the installationprogram. If you are using extracted packages, the install.sh file is located under$your_dir/IBM_SmartCloud_Provisioning.

There are two types of installation: minimal and custom.

Minimal:

A minimal installation only supports the characteristics described in this topic.v The only supported hypervisor is KVM.v You need at least two Storage Nodes and one Compute Node.


http://ftp.redhat.com/pub/redhat/rhel/beta/6/x86_64/os/Packages/livecd-tools-031-1.el6.1.x86_64.rpm

http://ftp.redhat.com/pub/redhat/rhel/beta/6/x86_64/os/Packages/livecd-tools-031-1.el6.1.x86_64.rpm

http://ftp.redhat.com/pub/redhat/rhel/beta/6/x86_64/os/Packages/python-imgcreate-031-1.el6.1.x86_64.rpm



Follows these steps to perform a minimal installation:1. In the Prerequisites panel, provide the path to the Red Hat Enterprise Linux 6.0

or 6.1 installation image.2. In the Network options panel, type:v The DNS suffix (in other words, the domain name of your environment, for

example your.cloud.com).v The address of the parent DNS server. This is the upstream DNS server to

which the local DNS server forwards non-authoritative requests.v The network address that indicates the network segment for the whole

environment (for example, 10.10.0.0).v The network mask (for example, 255.255.0.0).v The DHCP range to be used by the virtual machines (for example,

10.10.192.10-10.10.192.254).3. Read the pre-installation summary, modify anything if needed, and click Next

to start the installation.4. The installation program first sets up the PXE Server performing the following

actions:v Copying files to the /data directory.v Setting up a YUM (RPM) repository.v Setting up a DNS and DHCP server.v Creating a Live CD image.v Setting up the PXE Server.

5. When the PXE Server has been set up, you get a prompt that tells you that it issafe to power on the Storage Nodes and you can proceed to install the twoStorage Nodes:a. Power on the Storage Nodes and set the BIOS for PXE network booting.

This enables the installation of the operating system on the Storage Nodes.The kernel service virtual machines are installed on Storage Nodes, so it isrequired that the physical machines of the Storage Nodes supportvirtualization and that this feature is turned on before the installation.

b. On the console of the systems to be used as Storage Nodes, a PXE BootOption menu is displayed. Select storage_node on the menu.

c. If the workstation has multiple network devices, choose the device thatconnects to the cloud.

d. Reboot the Storage Node when prompted and change the BIOS settings tolet the hard disk be the first boot device.

6. During the setup of the Storage Nodes, you can also set up the ComputeNodes on the KVM hypervisor:a. Power on the physical machine.b. Set the BIOS for PXE network booting. Since the Compute Nodes are using

a ramdisk with a stateless boot. Each boot means a fresh install on the rootpartition. This means you must keep PXE booting as the first choice always.

7. The installation program detects that the Storage Nodes and the ComputeNodes are booting up. A minimal installation requires at least 2 Storage Nodes(with virtualization support). When they are installed and booted up, theinstallation program starts the kernel services deployment.

Note: Do not restart any Storage Nodes until the whole installationcompletes. The default password for Storage Nodes is passw0rd.


Note: Based on the available memory and CPU, the required virtualmachines are deployed on the Storage Nodes.

At the end of the installation, a post-installation summary panel is shown toindicate that the installation completed successfully. At this point, you can accessthe web UI and the other components according to the information shown on thesummary panel .

Custom:

With a custom installation, you can install the optional Virtual Image Librarycomponent, customize kernel services, and configure KVM and VMwarehypervisors.

Follow this steps to perform a custom installation:1. Select the options that you want to install:

Deploy a Virtual Image LibraryDeploy the Virtual Image Library component if you want to takeadvantage of extended services for image management. Thiscomponent supports the VMware VMFS repository in addition to therepository of SmartCloud Provisioning.

After you define the operational repository to the Virtual ImageLibrary and index the related images, all the information about theimages is stored in the Virtual Image Library DB. You can perform thefollowing operations on the indexed images:v Searching images for specific files and products.v Comparing two images and determining the differences in files and

products.v Finding similar images.

The Virtual Image Library component also provides the ability to trackimage version and provenance.

Note: Do not select this option if you are installing SmartCloudProvisioning in an environment different from RHEL 6.1, 64-bit.Virtual Image Library can be deployed only on a RHEL 6.1,64-bit machine. To manually install Virtual Image Library, referto the Manually installing topic in the Virtual Image Librarydocumentation.

Managing VMware ESXi serversSelect this option if you want SmartCloud Provisioning to manageyour existing ESXi servers. The installation program sets up anhypervisor on the servers.

2. In the Prerequisites panel, provide the path to the Red Hat Enterprise Linux6.0 or 6.1 installation image.

3. In the Network options panel, type:v The DNS suffix (in other words, the domain name of your environment, for

example your.cloud.com).v The address of the parent DNS server.v The network address that indicates the network segment for the whole

environment (for example, 10.10.0.0).


v The network mask (for example, 255.255.0.0).v The DHCP range to be used by the virtual machines (for example,

10.10.192.10-10.10.192.254).4. In the Kernel Services Settings panel:

a. Select the option Only deploy Kernel Service virtual machines to StorageNodes if you want to deploy the kernel services only on Storage Nodes,and not on Storage Nodes and Compute Nodes. This is the default andrecommended selection.

5. If you selected the option to manage VMware EXSi servers, you are shown apanel requesting additional information as follows:v VMware vSphere CLI installation package path: the VMware vSphere CLI

installation package is required to set up a Compute Node on the ESXiservers.

v ESXi Server Information: you can type the information for multipleVMware ESXi servers. Provide the IP Address, user name, and rootpassword and click Add. If you want to delete or update an existing entry,select the entry in the table and click the corresponding button.

6. Read the pre-installation summary, modify anything if needed, and click Nextto start the installation. At this point the installation program sets up the PXEServer. This action includes:v Copying files to the /data/directory.v Setting up the YUM (RPM) repository.v Setting up the DNS DHCP server.v Creating a Live CD image.v Setting up the PXE Server.

7. After the PXE Server has been set up, you can install the Storage Nodes andthe Compute Nodes. There must be two or more Storage Nodes in yourcloud. As opposed to the minimal installation, the custom installationcalculates the necessary resources according to the number of kernel servicesthat you selected.a. Power on the Storage Nodes and set the BIOS for PXE network booting.

This enables the installation of the operating system on the Storage Nodes.The default password for storage nodes is passw0rd. It is required that thephysical machines of the Storage Nodes support virtualization if you selectOnly deploy Kernel Service virtual machines to Storage Nodes.

b. Choose storage_node on the boot menu.c. If the workstation has multiple network devices, choose the device that

connects to the cloud.d. Reboot the Storage Node when prompted and change the BIOS settings to

let the hard disk be the first boot device.8. During the setup of the Storage Nodes, you can set up also the Compute

Nodes on a KVM hypervisor. The setup of the Compute Nodes is done asfollows:a. Power on the physical machine.b. Set the BIOS for PXE network booting. Because the Compute Nodes are

using a ramdisk with a stateless boot, each boot means a fresh install onthe root partition. This means you must always keep PXE booting as thefirst choice.

9. The installation program detects when the Storage and Compute Nodes arebooting up. At that point, the installation program starts deploying the kernelservices.


Note: Do not restart any Storage or Compute Nodes, until the wholeinstallation completes.

10. If you selected the option to manage the VMware ESXi servers, the installationprogram sets up an hyper program on the ESXi servers.

11. If you selected the option to deploy a Virtual Image Library, the installationprogram sets up the Virtual Image Library automatically. The instance can beidentified with the 'Tag' information that is set to image-library. In addition, apersistent volume with the same tag is attached to the machine.The default credentials to access Virtual Image Library are wasadmin/passw0rd.

At the end of the installation, a post-installation summary panel is shown toindicate that the installation completed successfully. At this point, you can accessthe web UI and the other components according to the information shown on thesummary panel .

Post-installation steps

If Virtual Image Library is deployed by the system, it is not a persistent image,therefore, if it is terminated, the instance is lost. It is recommended to proceed bycapturing it as a new image template, to obtain a new template with Virtual ImageLibrary already installed and available for future needs.

Installing using the CLIThe following topics describe how SmartCloud Provisioning can be installed usingthe command line interface.

Setting up the PXE ServerYou setup a PXE Server using an automation script.

The following topic describe how to set up the PXE Server and then use PXEnetwork booting to install the Compute Nodes and the virtual machines.

Using an automation script:

This topic describes how to set up the PXE Server using an automation script.

Procedure

1. Prepare the repository and ensure that all prerequisites (ruby and livecd-tools)are in this directory:mkdir -p /data/repos/isaac

2. Extract install_media/IBM_SmartCloud_Provisioning/data/IBM_SmartCloud_Provisioning-1.2.0.0-*.tgz to /data.

3. Navigate to /data/IBM_SmartCloud_Provisioning-1.2.0.0-*/.4. Modify the settings in ISAAC.cfg:

Note: If the MAC address of a workstation is not specified in ISAAC.cfg, itwill automatically be installed as a Compute Node. If you plan on usingvirtual machines to run kernel services, which is the defaultconfiguration, make sure their MAC addresses start with 52:54, forexample, 52:54:00:00:00:01.


Field Description and sample value

iso_location The location of the RHEL6 iso, for example"/data/RHEL6.0-20100922.1-Server-x86_64-DVD1.iso"

dns_suffix The domain name of your environment, for example"your.cloud.com"

#gateway Optional. The gateway address. If not specified, the gatewaytakes the first address of the network. For example, if thenetwork address is 10.10.1.0, the gateway address becomes10.10.1.1.

dns_world The address of the parent DNS server

network The network address, for example "10.10.0.0"; it indicatesthe network segment for the whole environment

netmask The network mask, for example "255.255.0.0"

dhcp_range The dhcp range to be used by the virtual machines, forexample "10.10.192.10-10.10.192.254".

storage_mac_addrs The comma-separated MAC addresses that the StorageNodes use, for example"e4:1f:13:ef:36:8a,e4:1f:13:ef:36:de", or leave it blankto identify the Storage Nodes from the boot menu.

zookeeper_mac_addrs The MAC addresses that the zookeepers virtual machinesuse, for example"52:54:00:00:00:01,52:54:00:00:00:02,52:54:00:00:00:03"

ldap_mac_addr The MAC address that the LDAP server virtual machineuses, for example "52:54:00:00:00:04"

hbase_mac_addrs The MAC addresses that the HBase virtual machines use,for example"52:54:00:00:00:05,52:54:00:00:00:06,52:54:00:00:00:07"

webconsole_mac_addr The MAC address that the web console virtual machine uses

webservice_mac_addr The MAC address that the web service virtual machine uses

ntp_svr_addrs Comma separated NTP server addresses. Leave this blank ifPXE Server is used as NTP Server.

adm_console_mac_addrs The MAC address that the Administrative Console’s virtualmachine uses.

storage_hd The block device on which the Storage Node OS is installed.

5. Run the script:./firstbox_run

Note: If you must change ISAAC.cfg again and run the script again, youmust remove all content in /var/lib/tftpboot.

Results

The PXE Server has been set up. A private DNS and DHCP service, required bythe cloud, have been setup and started in the PXE Server.

To verify if the installation was successful, check if there is content in the directory/var/lib/tftpboot and check if the services dhcpd, named, httpd, xinetd arerunning. If any of these services is not running, you must reconfigure ISAAC.cfg,remove all content in /var/lib/tftpboot, and run the script again.


Installing the Storage NodesAfter the PXE Server has been set up, you can install the Storage Nodes. Theremust be two or more Storage Nodes in your cloud.

About this task

When the PXE Server is ready and available, power on the Storage Nodes and setthe BIOS for PXE network booting. This enables the installation of the operatingsystem on the Storage Nodes.

The default password for storage nodes is passw0rd.

It is recommended that the physical machines of the Storage Nodes supportvirtualization.

Procedure1. Power on the physical machine.2. Set the BIOS for PXE network booting.3. Choose Storage Node if the boot menu prompts.4. If the workstation has multiple network devices, choose the device that

connects to the cloud.5. Reboot the Storage Node when prompted and change the BIOS settings to let

the hard disk be the first boot device.

Installing Compute NodesThe following topics describe how to install Compute Nodes on the differenthypervisors.

Installing Compute Nodes on a KVM hypervisor:

The Compute Nodes must be installed on physical machines.

Before you begin

(Optional) How to enable automatic rack configuration:

The Compute Node can identify its own rack automatically besides reading it from/iaas/etc/hyper_bot.conf. It uses the host name, where the first four letters servesas the rack name and the remaining letters as the node name. Follow these steps toachieve this:1. Modify /var/lib/tftpboot/startup.sh and comment out lines 170 and 171 so

that the code snippet looks as follows:167 if [ -z "$HNAME" ]; then168 if [ "$ROLE" = "storage" ]; then169 HNAME=`ssh -o StrictHostKeyChecking=no $SERVER \

/var/lib/tftpboot/add_storage.sh $CLOUD_IFACE_MAC | \grep "new storage node" | \cut -d’ ’ -f5`

170 # elif [ "$ROLE" = "compute" ]; then171 # HNAME=$(echo "v`echo "$CLOUD_IFACE_MAC" | \

tr ’[A-Z]’ ’[a-z]’ | tr -d ’:’`")172 fi173 fi

2. Configure the Compute Node with host names that meet the required rule.This must happen after the PXE Server has been set up and before theCompute Node is booted. Modify /etc/dhcp/dhcpd.static on the PXE Server


and add blocks according to your Compute Node. An example follows. Ensureto restart the dhcpd service after the update.host r001pc01 {

hardware ethernet 1c:6f:65:0b:80:fc;ddns-hostname = pick (option host-name, "r001pc01");fixed-address 10.0.1.13;

}

3. Remove the rack configuration from hyper_bot.conf.Modify /var/lib/tftpboot/common/etc/hyper_bot.conf on the PXE Server andremove the nameand rack lines so that the file looks similar to what follows:{

"main":{},

"monitoring":{"data_collect_interval":30,"file_collect_interval":60

}

}

4. Prepare and boot the Compute Node.

You can also accomplish this after the Compute Nodes have been booted. A bootedCompute Node registers itself into /etc/dhcp/dhcpd.static on the PXE Serverafter it finishes the installation typically as this:host v1c6f650b80fc {

hardware ethernet 1c:6f:65:0b:80:fc;ddns-hostname = pick (option host-name, "v1c6f650b80fc");fixed-address 10.0.128.106;

}

Then you only need to replace the name with an appropriate name. v1c6f650b80fcis a typical name if the node is booted from the PXE Server. If the node ismanually installed, the name might be set according to r001pc01, r002pc09 ... andso forth. After this file is modified, you must restart the dhcpd service with thecommand /etc/init.d/dhcpd restart.

About this task

Note: Except for Compute Nodes, the MAC addresses of all nodes are storedin the PXE Server. This means that all nodes whose MAC address is not inthe PXE Server are all set up as Compute Nodes when booted from thenetwork.

The default password for Compute Nodes is passw0rd.

The Compute Node is up and running after the installation. There is no need torestart it.1. Power on the physical machine.2. Set the BIOS for PXE network booting. Since the Compute Nodes are using a

ramdisk with stateless boot, each boot means a fresh install on the rootpartition. This means you must keep PXE booting as the first choice always.

The installation of a Compute Node can be done on a pristine machine or on amachine that already has an operating system or that has partitions on hard disks:


Installation on a pristine machineIf the machine has no operating system already installed, power it on. ThePXE installation makes the following partitions automatically:v /dev/sda1 with 8 GBv /dev/sda2 with 8 GBv /dev/sda3 with 16 GBv /dev/sda4 with the remaining space of the first disk

Installation on a machine that has an OS or that has partitionsThe are three choices:

Change FORCE_PARTITION=no to FORCE_PARTITION=yes in the file/var/lib/tftpboot/pxelinux.cfg/default

The default value for FORCE_PARTITION is no, if set to yes, theramdisk partitions the hard disk as mentioned before. This is themost convenient way, but you must ensure that this setting issuitable for all Compute Nodes

Remove the existing partitionsRemove the partitions and set the BIOS for PXE network booting.

Use the existing partitions or create partitions manually

1. Edit /var/lib/tftpboot/pxelinux.cfg/default on the PXEServer and set:ROOTDEV=/dev/sda1 LOGDEV=/dev/sda2 SWAPDEV=/dev/sda3 IAASDEV=/dev/sda4

where ROOTDEV is the root partition (≥ 8 GB), LOGDEV is mountedon /var/log, and IAASDEV is mounted on /iaas/local-storage.SWAPDEV works as a swap partition, the current auto-partitiondoes not support a hard disk larger than 2T. If you have suchkind of disk, a workaround is splitting the disk into two disksin the BIOS, where the first disk must be less than 2T. Theramdisk recognizes them as /dev/sda and /dev/sdb, then youcan partition the disks manually and edit the parameters in thedefault file accordingly. Also, if you have plenty of memory,you can remove the ROOTDEV=xxx parameter to make the rootpartition inside the memory entirely.

2. Reboot and set the BIOS for PXE network booting.

Note: All data in the root partition is removed.

What to do next

The Compute Nodes, unlike the Storage Nodes, are expected always to PXE boot.When the PXE boot completes, you see a CLI on the Compute Node screen. Youcan sign on as root and start working. If the system reboots, you get the sameresult. The physical machine is not intended to be a permanent store. Somepartitions are still alive after the reboot, while some partitions are overwritten.

Installing Compute Nodes on a Xen hypervisor:

The following topics describe how to install and configure a Compute Node thatruns on a Xen hypervisor.


Before you begin

This following tutorial explains how to install and configure Red Hat EnterpriseLinux to run as Xen dom0, which is not supported by Linux by default. You mustcompile Xen and the dom0 kernel from sources and RPMs. For reference, theversion list of the components is:

XEN 4.0.1dom0 kernel 2.6.32.43libvirt 0.8.7

Installing RHEL6:

This topic describes how you can perform an RHEL6 installation.

Procedure

1. Boot from the Red Hat Enterprise Linux 6.0 or 6.1 x86_64 DVD, and install asusual..

2. Choose Basic Server install.3. Register your system to RHN, so that you can fetch updates and install

additional packages using yum. However, you can also do so by setting up aRed Hat repository for packages. Disable SElinux by editing/etc/selinux/config and make SELINUX=disabled.

Installing the prerequisite packages:

There are a number of prerequisite packages that you must install.

About this task

Run following commands in this sequence:yum groupinstall "Development Tools"yum install transfig wget texi2html libaio-devel dev86 glibc-devele2fsprogs-devel gitk mkinitrd iaslyum install xz-devel bzip2-devel pciutils-libs pciutils-develSDL-devel libX11-devel gtk2-devel bridge-utils openssl-develyum install PyXML qemu-img mercurialyum install ncurses-devel texinfo libuuid-devel python-develwget http://www.debath.co.uk/dev86/Dev86bin-0.16.18.tar.gzcd /tar -zxvf ~/Dev86bin-0.16.18.tar.gzwget http://acpica.org/download/acpica-unix-20101209.tar.gztar -zxvf acpica-unix-20101209.tar.gzcd acpica-unix-20101209cd compiler/makecp iasl /usr/bin/

Rebuilding and installing QEMU:

Run the following commands to rebuild and install the QEMU process emulator.

About this taskyum install pulseaudio-libs-develmkdir -p /root/src && cd /root/srcwget http://ftp.funet.fi/pub/mirrors/fedora.redhat.com/pub/fedora/linux/ \updates/13/SRPMS/qemu-0.13.0-1.fc13.src.rpmrpm -i qemu-0.13.0-1.fc13.src.rpmcd /root/rpmbuild/SPECS


rpmbuild -bb qemu.speccd /root/rpmbuild/RPMS/x86_64rpm -Uvh qemu-common-0.13.0-1.el6.x86_64.rpm

Rebuilding and installing Xen:

Run the following commands to rebuild and install Xen.

About this taskyum install glibc-devel.i686 kpartxmkdir -p /root/src && cd /root/srcwget http://ftp.funet.fi/pub/mirrors/fedora.redhat.com/pub/fedora/ \linux/releases/14/Everything/source/SRPMS/xen-4.0.1-6.fc14.src.rpmrpm -i xen-4.0.1-6.fc14.src.rpmcd /root/rpmbuild/SPECSrpmbuild -bb xen.speccd /root/rpmbuild/RPMS/x86_64/rpm -Uvh xen*4.0.1-6*.rpmwget http://bits.xensource.com/oss-xen/release/4.0.1/xen-4.0.1.tar.gztar -zxvf xen-4.0.1.tar.gzcd xen-4.0.1make xenmake toolsmake stubdommake install-xenmake install-toolsmake install-stubdommake world

Note: The command make world takes a long time to download the pvopskernel. After download, press Ctrl+C when prompted for the configurationof the kernel.

At this point you have the kernel ready to be configured and compiled. Clone thekernel git repository from kernel.org and verify the xen/stable-2.6.32.xlong-term maintained branch. The configuration file used is version 2.6.32.43, but itis supposed to work also with newer version.cd linux-2.6-pvops.gitwget -O .config http://pasik.reaktio.net/xen/kernel-config/ \config-2.6.32.43-pvops-dom0-xen-stable-x86_64makemake modules_installmake install

Installing libvirt and virt-manager:

This topic describes how you install libvirt and virt_manager.

Before you begin

Before rebuilding libvirt, you must boot into a non-Xen environment.

The libvirt version should be 0.8.7 or higher.

About this taskyum -y install augeas radvd systemtap-sdt-devel gnutls-utilsmkdir -p /root/srccd /root/srcwget ftp://ftp.redhat.com/pub/redhat/linux/enterprise/ \6Server/en/os/SRPMS/libvirt-0.8.7-18.el6.src.rpm


rpm -i libvirt-0.8.7-18.el6.src.rpmwget http://pasik.reaktio.net/xen/patches/libvirt-spec-rhel6-enable-xen.patchcd /root/rpmbuild/SPECScp -a libvirt.spec libvirt.spec.origpatch -p0 < ~/src/libvirt-spec-rhel6-enable-xen.patchyum install libnl-devel xhtml1-dtds libudev-devel libpciaccess-devel yajl-devel \libpcap-devel avahi-devel parted-devel device-mapper-devel numactl-devel \netcf-develyum install lzop cyrus-sasl-md5 gnutls-utils ncrpmbuild -bb libvirt.speccd /root/rpmbuild/RPMS/x86_64/rpm -Uvh --force libvirt-0.8.7-18.el6.x86_64.rpm \libvirt-client-0.8.7-18.el6.x86_64.rpm libvirt-python-0.8.7-18.el6.x86_64.rpmlibvirt-devel-0.8.7-18.el6.x86_64.rpm

Configuring Xen DOM0:

This topic describes how to configure Xen DOM0.

Procedure

1. Create the network bridge of br0 for the Xen Compute Node:vim /etc/xen/xend-config.sxpMake sure:(network-script ’network-bridge bridge=br0’)(xend-unix-server yes)

2. Configure the network by editing /etc/sysconfig/network-scripts/ifcfg-ethX, and defining ifcfg-eth0, for example:cat /etc/sysconfig/network-scripts/ifcfg-eth0DEVICE=eth0HWADDR=00:21:86:17:27:96NM_CONTROLLED=noONBOOT=yesTYPE=EthernetBOOTPROTO=staticIPADDR=10.0.0.63USERCTL=no

Make sure that NM_CONTROLLED="no" and ONBOOT="yes". Then run ifup eth0 toactivate networking.

3. Make sure that your internet connection works fine, the make sure that the/etc/hosts file has an entry for your hostname:# cat /etc/hosts127.0.0.1 localhost localhost.localdomain localhost4 \localhost4.localdomain4::1 localhost localhost.localdomain localhost6 localhost6.localdomain6172.16.0.1 hostname

4. Make sure that the network service is set to start automatically, and that theKSM services do not work with Xen:/sbin/chkconfig --add xend/sbin/chkconfig --add xendomains/sbin/chkconfig xend on/sbin/chkconfig xendomains on/sbin/chkconfig ksm off/sbin/chkconfig ksmtuned off

5. Make sure that the blktap driver and iptables work in Xen by editing/etc/rc.local and adding rules such as:modprobe blktapiptables -Fip addr add 169.254.169.254/32 dev br0


iptables -A FORWARD -d 169.254.169.254 -j DROParptables -A FORWARD -d 169.254.169.254 -j DROPecho "1" > /proc/sys/net/bridge/bridge-nf-call-arptables

6. Edit /etc/grub.conf in a similar way:default=0 #reboot choose xen as defaulttitle XEN 4.0 Red Hat Enterprise Linux Server (2.6.32.43)root (hd0,0)kernel /xen-4.0.gz dom0_mem=1024Mmodule /vmlinuz-2.6.32.43 ro \root=/dev/mapper/vg_hyper06-lv_rootrd_LVM_LV=vg_hyper06/lv_root \rd_LVM_LV=vg_hyper06/lv_swap rd_NO_LUKS rd_NO_MD rd_NO_DM \LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us \crashkernel=auto rhgb quiet console=hvc0module /initramfs-2.6.32.43.img

7. Reboot the system.

Verifying Xen and libvirt:

This topic describes how to verify Xen and libvirt.

About this task

Do the following:# uname -aLinux xen03 2.6.32.43 #4 SMP Wed Aug 17 01:03:12 CST 2011 \x86_64 x86_64 x86_64 GNU/Linux# xm listName ID Mem VCPUs State Time(s)Domain-0 0 xxx xxx r----- 36.0# virsh listId Name State----------------------------------

0 Domain-0 running# virsh versionCompiled against library: libvir 0.8.7Using library: libvir 0.8.7Using API: Xen 3.0.1Running hypervisor: Xen 4.0.0# ifconfigbr0 Link encap:Ethernet HWaddr 00:21:86:17:27:96inet addr:10.0.0.63 Bcast:0.0.0.0 Mask:255.0.0.0inet6 addr: fe80::221:86ff:fe17:2796/64 Scope:LinkUP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1RX packets:4080286 errors:0 dropped:0 overruns:0 frame:0TX packets:2740994 errors:0 dropped:0 overruns:0 carrier:0collisions:0 txqueuelen:0RX bytes:3936981533 (3.6 GiB) TX bytes:455803655 (434.6 MiB)lo Link encap:Local Loopbackinet addr:127.0.0.1 Mask:255.0.0.0inet6 addr: ::1/128 Scope:HostUP LOOPBACK RUNNING MTU:16436 Metric:1RX packets:5424 errors:0 dropped:0 overruns:0 frame:0TX packets:5424 errors:0 dropped:0 overruns:0 carrier:0collisions:0 txqueuelen:0RX bytes:744553 (727.1 KiB) TX bytes:744553 (727.1 KiB)peth0 Link encap:Ethernet HWaddr 00:21:86:17:27:96inet6 addr: fe80::221:86ff:fe17:2796/64 Scope:LinkUP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1RX packets:4082757 errors:0 dropped:0 overruns:0 frame:0TX packets:2749305 errors:0 dropped:0 overruns:0 carrier:0collisions:0 txqueuelen:100RX bytes:3996289089 (3.7 GiB) TX bytes:456788001 (435.6 MiB)Memory:d0300000-d0320000virbr0 Link encap:Ethernet HWaddr 52:54:00:2E:0F:49


inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1RX packets:0 errors:0 dropped:0 overruns:0 frame:0TX packets:147762 errors:0 dropped:0 overruns:0 carrier:0collisions:0 txqueuelen:0RX bytes:0 (0.0 b) TX bytes:22649407 (21.6 MiB)

Installing the Compute Node:

Follow these steps to install the Compute Node.

Before you begin

Before installing a Compute Node, you can optionally enable automatic rackconfiguration as described in “Installing Compute Nodes on a KVM hypervisor”on page 11.

Procedure

1. Configure the repository for Red Hat and SmartCloud Provisioning. You cancopy the repository definition files from the PXE Server. The folder is/var/lib/tftpboot/, copy the repo files to /etc/yum.repos.d.

2. Yum install the following packages:rubyruby-develrubygemsdevice-mapper-multipathhttpdsquashfs-toolsarptables_jf gccibm-java-x86_64-sdk

3. Gem install the package yajl-ruby-0.7.8.gem that you can get from the PXEServer, from folder /var/lib/tftpboot/, by extracting deps.tar.gz.

4. Install the Compute Node with yum install isaac-hyper-bots-xen.5. To configure multipath, copy the multipath configuration file multipath.conf

from the PXE Server. The folder is /var/lib/tftpboot/compute-node/. Copymultipath.conf into the folder of /etc/. After copying it, run the commands:multipath/etc/init.d/multipathd startchkconfig multipathd on

6. Modify /etc/resolv.conf to set the correct name server and domain nameconfigured during PXE Server setup. For example:search test.isaac.ibm.com isaac.ibm.comnameserver 172.16.0.16

You can copy this file from other hyper nodes with a KVM hypervisor in thesame environment installed by the PXE Server.

7. To start Apache, copy the file /iaas/apache-data/iaas.conf to the folder/etc/httpd/conf.d/ and restart the service httpd:cp -f /iaas/apache-data/iaas.conf /etc/httpd/conf.d//etc/init.d/httpd start

To prepare the Compute Node configuration:a. Modify /iaas/etc/hyper_bot.conf to use the following Xen configuration

settings: "hyper_type":"xen". For reference, see the sample located in/iaas/etc/hyper_bot.conf.sample.


b. Modify /iaas/etc/isaac.conf to set the correct environment settings. Youcan copy this file from other hyper nodes with a KVM hypervisor in thesame environment installed by the PXE Server.

Install vhd-util on a Storage Node:

To support the capturing of images with a vhd format, vhd-util must be installedon a Storage Node.

Procedure

1. Copy vhd-util which is located in /usr/sbin/vhd-util from the Xen ComputeNode to all Storage Nodes. Make sure to copy in the same folder of/usr/sbin/vhd-util.

2. On a Storage Node, you must install:xen-libs-4.0.1-6.el6.x86_64.rpmxen-licenses-4.0.1-6.el6.x86_64.rpm

You can copy these rpm files from the Xen Compute Node. The files are locatedin the folder /root/rpmbuild/RPMS/x86_64/.

3. Verify vhd-util installation by running:# vhd-util . The usage of this commandis:vhd-util COMMAND [OPTIONS]where COMMAND := { create | snapshot | query | read | set | repair | \resize | fill | coalesce | modify | scan | check | revert }

Testing and verifying the overall installation:

This topic describes how to verify the installation of a Compute Node on Xen.

Before you begin

Start the Compute Bot using the command :/iaas/hyper_bots/rubybots/re-run.sh

You can use the CLI to check that the node is online and that it can connect to theXen server.

Procedure

1. Use the command iaas-describe-nodes to see the following ouput and verifythat this hyper bot is a Xen hypervisor:jid | cpu_load | instances | hyper_type | ipaddress | disk_used_pct | \is_manager |availability | enabled | version | free_memory | start_time

rack4.xen01 | 2.06 | 0 | xen | 10.0.0.62 | 71 | y | xen-9-10-0-6910 \| yes | 1.1.0.206 | 6910 | Tue Aug 23 10:52:42 +0800 2011rack4.xen02 | 0.26 | 0 | xen | 10.0.0.61 | 66 | n \| xen-4-4-0-1999 | yes | 1.1.0.206 | 1999 | Mon Aug 22 18:43:53 +0800 2011

2. When registering an image for Xen, you must specify a kernel for Xen. Use theCLI to register:a. Register the image kernel. You must specify that the type of the image is

kernel:./iaas-register-virtual-image -i vmlinuz-xen -t kernel -a x86_64 -p linux \-d "centos xen kernel" -v no

b. Register the image ramdisk. You must specify that the type of the image isramdisk:


./iaas-register-virtual-image -i initrd-xen -t ramdisk -a x86_64 -p linux \-d "centos xen ramdisk" -v no

c. Register the master image. You must specify the kernel and the ramdiskregistered in previous steps:./iaas-register-virtual-image -i centos-xen -t image -a x86_64 -p linux \-d "centos xen" -v no -k vmlinuz-xen -r initrd-xen

3. Run iaas-describe-virtual-image to verify that the output is:super-centos |image|available|admin|linux|x86_64|1.0|no|raw|idevmlinuz-xen |kernel|available|xen-kernel|admin|linux|x86_64|0.0|no|raw|ideinitrd-xen |ramdisk|available|xen-kernel|admin|linux|x86_64|0.0|no|raw|ide

4. Use the CLI to run the virtual machine:./iaas-deploy-vms -i vmlinuz-xen -k vmlinuz-xen -r initrd-xen

You must specify the kernel and the ramdisk of the image, otherwise themanager bot launches it with a KVM hypervisor.

Installing Compute Nodes on a VMware hypervisor:

These topics describe how to install a Compute Node that runs on a VMwarehypervisor.

Prerequisites

The prerequisites to installing on VMware are as follows:v A previously set up PXE Server.v The following vSphere CLI 4.1 package for x86_64 Linux available on the PXE

Sever at /data/repos/isaac:VMware-vSphere-CLI-4.1.0-*.x86_64.tar.gz

v An ESXi 4.1 server already installed.v A password was added to the ESXi 4.1 server.

Installing VMware Management VM:

To install VMware Management VM, you run a script.

About this task

Run the script add_compute_vmw.sh that is located in /data/IBM_SmartCloud_Provisioning-1.2.0.0-*/deploy-scripts on the PXE Server:add_compute_vmw.sh <esxi_ip> [-f][esxi_password] [vmnic]

where:

esxi_ipIs the IP address of the ESXi server.

f This option forces an overwrite of an existing VMware management VM.

esxi_passwordIs the root password of the ESXi server. If the ESXi server has beenconfigured with password passw0rd, this parameter can be omitted.

vmnic Is the NIC address with which the ESXi server connects to the cloud. Thiscan retrieved by checking Configure Management Network --> NetworkAdapters from the direct console of the ESXi server. If not provided, thefirst vmnic among the active ones is used as default.


Testing and verifying the installation:

At conclusion of the installation of the Compute Node, you can use the CLI toverify if the node is online and if it can connect to the VMWare ESXi server.

About this task

You can use the CLI to check that the node is online and can connect to theVMware ESXi server. This task can only be done after the installation of the kernelservices is finished. Refer to this topic to accomplish this: “Installing kernelservices.”

Procedure

1. Use ./iaas-describe-nodes to see the following ouput and verify that thishyper bot is a VMware hypervisor:rack3.test | 0 | 1 | vmware | 172.16.0.146 | 0 | y | vmware-31-0-25479| yes| 1.1.0.39 | 25479 | Wed Jun 22 07:12:42 -0400 2011

2. Run iaas-describe-virtual-image to verify that the output is:rh61vmdk | image | available | vmware 6.1 mini | admin| linux | x86_64 | 1.0 | no | vmdk

3. Use CLI to run the virtual machine and specify the vmdk format images:./iaas-deploy-vms -i rh61vmdk

Note: When registering an image for VMware, the vmdk format image issupported and must be only one file. Only dhcp is supported for thenetwork configuration of the image. You can upload the image file intoeach Storage Node. Use the CLI to register:./iaas-register-virtual-image -i rh61vmdk -t image -a x86_64 -p linux-d "vmwaretest" -f "vmdk"

You must specify that the format of the image is vmdk, otherwise themanager bot might launch it with KVM hypervisor.

Installing kernel servicesAfter you have installed the Storage Nodes and Compute Nodes, you can installthe kernel service virtual machines.

About this task

Do this on the PXE Server:1. cd /data/IBM_SmartCloud_Provisioning-1.2.0.0-*/deploy-scripts

2. ./deploy_all.sh

This script deploys all kernel service virtual machines to Storage Nodes andCompute Nodes. Kernel service virtual machines include HBase, Zookeeper,LDAP Server, Web Service, Web Console, and administrative console.

Verifying the installationWhen the installation is complete, you can work with your cloud.


About this task

Verifying the Web Console:

To try the new environment accesshttp://<web_console_ip_address>:8080/welcome.html

orhttp://<web_console_ip_address>/welcome.html

.

You can retrieve the IP address of the web console from /etc/hosts on a StorageNode or Compute Node. In there, you can find a line like the following:’10.0.0.22 webconsole webconsole.xxxxxxxx.com’

Verifying the CLI:

The CLI interface is installed automatically on the first Storage Node (hostnamestorage-1). The IP address of first Storage Node can be retrieved from /etc/hostson Compute Nodes or Storage Nodes. With the CLI, you can view nodes, images,register images. For detailed usages, see Using and administering.

Verifying the administrative console:

At completion of the installation, the administrative console is also installed as oneof the kernel services virtual machines and can be accessed from:http://<admin_console_ip_address>:9080/login

The default credentials to access the console are adminuser/passw0rd.

Similarly, you can retrieve the IP address of the Web Console from /etc/hosts on aStorage Node or Compute Nodes. In these nodes and at that location, you can seea line like the following:’10.0.0.23 admconsole admconsole.xxxxxxxx.com’

Restarting all servicesAfter everything has been installed, all services are up and running automatically.

About this task

Though all services are up and running whenever a Compute Bot or a virtualmachines boots up, there might be situations in which you must restart theservices manually.

Before trying to restart any services, you must SSH into the node first. For the IPaddresses of the Storage Bots and kernel services, map the addresses to the hostnames. On the PXE Server, you can view the file /var/lib/tftpboot/hosts, on anyother node (whether it’s a physical machine or a virtual machine), you can viewthe file /etc/hosts. For the Compute Bots IP addresses, you need to view the file/etc/dhcp/dhcpd.static. Then restart the services in this order:1. Start the DNS and DHCP services on the PXE Server:

a. SSH into the PXE Server.b. Run /etc/init.d/named start.


c. Run /etc/init.d/dhcpd start.

2. Restart the LDAP server:a. SSH into the LDAP server.b. /etc/init.d/slapd restart

3. Restart all Zookeeper Servers:a. SSH into each Zookeeper node.b. /opt/zookeeper-3.3.3/bin/zkServer.sh start

Note: Run/opt/zookeeper-3.3.3/bin/zkServer.sh stop

to stop the service.4. Restart the HBase Nodes. Refer to the HBase documentation for doing so:

a. SSH into hbase-1 (if you have less than 5 HBase Nodes) or into hbase-3 (ifyou have 5 or more HBase Nodes).

b. /opt/IHC-0.20.2/bin/start-dfs.sh && /opt/hbase-0.90.4/bin/start-hbase.sh

Note: Run/opt/hbase-0.90.4/bin/stop-hbase.sh && /opt/IHC-0.20.2/bin/stop-dfs.sh

to stop it.5. Restart all REST Servers.

If you have less than 5 HBase nodes, a REST Server is installed on each HBasenode except hbase-1. If you have 5 or more HBase nodes, a REST Server isinstalled on each HBase node except hbase-1, hbase-2 and hbase-3.Each REST Server is installed under /iaas/iaas-rest-srv.To start a REST Server:a. SSH into the node.b. cd /iaas/iaas-rest-srv/bin && ./rest_server.sh start

Note: Run cd /iaas/iaas-rest-srv/bin && ./rest_server.sh stop tostop it.

6. Restart all storage bots:a. SSH into each Storage Node.b. killall -9 ruby zooadapter && /iaas/storage_bots/rubybots/re-run.sh

7. Restart all compute bots:a. SSH into each Compute Node.b. killall -9 ruby zooadapter && /iaas/hyper_bots/rubybots/re-run.sh

8. Restart the Web Service:a. SSH into the Web Service node.b. /iaas/webservices/re-run.sh

9. Restart the Web Console:a. SSH into the Web Console node.b. cd /iaas/webconsole && nohup ./re-run.sh && service httpd start

10. Restart the administrative console:a. SSH into the administrative console node.b. cd /iaas/web-adminconsole && nohup ./re-run.sh


Shutting down a running environmentThis topic describes how to shutdown a running SmartCloud Provisioning.

Procedure1. Shutdown all bots and related services:

a. SSH to the virtual machine running the administrative console, and run:# killall -9 ruby

b. SSH to the virtual machine running the web console, and run:# killall -9 ruby

c. SSH to the virtual machine running the web service, and run:# killall -9 ruby

d. SSH to each Compute Node, and run:# killall -9 ruby

e. SSH to each Storage Node, and run:# killall -9 ruby

2. Shutdown all kernel services:a. SSH to the virtual machines running the REST server, and shutdown the

REST server:# cd /iaas/iaas-rest-srv/bin# ./rest_server.sh stop

b. SSH to the virtual machine running the HMaster node, and shutdownHBase cluster:# cd /opt/hbase-0.90.4/bin# ./stop-hbase.sh

c. SSH to the virtual machine running the Hadoop NameNode, and shutdownthe HDFS cluster:# cd /opt/IHC-0.20.2/bin# ./stop-dfs.sh

d. SSH to each virtual machine running a ZooKeeper, and shutdown theZooKeeper server:# cd /opt/ibm-zookeeper-3.2.2/bin# ./zkServer.sh stop

3. Shutdown the DNS and DHCP services on the PXE Server. To do so, SSH to thePXE Server:# /etc/init.d/dhcpd stop# /etc/init.d/named stop

4. Shutdown all virtual machines. To do so, you can SSH to all virtual machinespreviously mentioned, and execute the following command to shut them downone by one:# shutdown -h now

5. Shutdown all physical machines. To do so, you can SSH to all physicalmachines, and execute the following command to shut them down one by one:# shutdown -h now


Recovering kernel services virtual machines after a restart ofyour cloud

About this task

In a typical SmartCloud Provisioning environment, the kernel service virtualmachines are hosted on Storage Nodes or Compute Nodes. When these physicalmachines are rebooted, the kernel service virtual machines, as well as the kernelservices themselves, can automatically recover for as long as all physical machinesthat hold kernel service virtual machines are shutdown simultaneously, or within ashort interval.

Likewise, these physical boxes need to be powered-on almost simultaneously whenrecovering. It is recommended to back up important data, like HBase tables, beforeproceeding rebooting physical machines.

On the other hand, when a kernel service virtual machine is no longer needed, it isbetter to terminate it before the physical machine reboots. This is to prevent it frombeing recovered after system rebooting. Follow these steps to terminate a runningvirtual machine:1. Get the list of running virtual machines:

# virsh list

2. Terminate the vm that is no longer needed:# virsh destroy <vm_name>

UpgradingThe following topics describe how to upgrade SmartCloud Provisioning.

There are two ways this task can be done:v Using the GUI.v Using a script.

Upgrading using the GUIIf the installation program detects that a previous version of SmartCloudProvisioning is installed, then it starts the upgrade process.

Before you begin

Before launching the installation program, verify the following:v Red Hat Enterprise Linux 6.0 or 6.1 x86_64 is installed, with X Window System.

Set the system time correct and do not change it until after the environment isup and running. To install X Windows System, run:yum groupinstall "x windows system"

v The GUI installation can only be run on the PXE Server.

You also must set up automatic authentication with all of the management nodesand Compute Nodes involved in the upgrade, or these nodes will keep requestingthat you type a password.


Procedure1. Unpack the installation package and run install.sh.2. Select Upgrade IBM® SmartCloud Provisioning.3. If the upgrade fails to detect a Zookeeper server, you need to define the

Zookeeper server’s IP address before continuing.4. The upgrade starts detecting all nodes and services.5. In the Upgrade Options panel, click re-detect if you want to detect all nodes

again.6. If you want to upgrade the Web Console and if the upgrade fails to detect any

component, you can specify the Web Console’s IP address.7. You get the option to install Virtual Image Library.8. The upgrade starts.

Results

Verify the status of Virtual Image Library.

The IP address of the Virtual Image Library virtual machine can be obtainedthrough the CLI with the filtering tag of image-library:./iaas-describe-instances | grep image-library

And the link https://<ip_of_image_library>:9443/ImageLibraryUI/ can be usedto access the Virtual Image Library GUI.

Virtual Image Library might take 30 minutes or more to be fully functionaldepending on the hardware performance.

Upgrading using a scriptFollow these steps to upgrade the PXE Server and your cloud environment usingan automated script.

Before you begin

Before you run the upgrade script, make sure that you are using the PXE Serverand that all nodes are active.

You also must set up automatic authentication with all of the management nodesand Compute Nodes involved in the upgrade, or these nodes will keep requestingthat you type a password.

About this task

You must get the iso package or the DVD for SmartCloud Provisioning and searchfor a file named IBM_SmartCloud_Provisioning-1.2.0.0-*.tgz included in the CDimage. This file is in the directory ./data/. Untar the file to find a file namedupgrade.sh in the uncompressed directory. To upgrade, follow either of thefollowing procedures:

Procedure A (interactive):

Run ./upgrade.sh.

Provide the following information. Press Enter to accept the default values:


1. Specify a Zookeeper Server IP. The default value is detected from the PXEServer.

2. Specify a web console IP. The default value is detected from the PXE Server.3. The upgrade process automatically detects all nodes information. You must

specify if the values are correct. If the values are correct, the upgrade startsand, if it is successful, the PXE Server is upgraded. If the upgrade process isnot successful, error messages are issued.

Procedure B (non interactive):

Follow these steps:1. 1. Modify the settings in upgrade.cfg:

Example of upgrade.cfg:

storage_node=’10.0.0.13 10.0.0.14’storage_manager=’10.0.0.13’compute_node=’10.0.0.3 10.0.0.4’compute_manager=’10.0.0.3’webservice=’10.0.0.3’rest_server=’10.0.0.3’webconsole=’10.0.0.3’

Note: You must add these node IPs manually if you want to upgrade thenodes. Add a blank space among the IPs. However, if you plan toupgrade some nodes without using the script, you can delete their IPsfrom upgrade.cfg.

2. Run:./upgrade.sh upgrade.cfg

Note: If the detection of the Zookeeper’s IP from the PXE Server fails, youmust run the script and specify an IP:./upgrade.sh upgrade.cfg 10.0.XX.XX

When you complete the above steps, the PXE Server has been upgraded. All kernelservices (Compute Nodes, Storage Nodes, Web Service, REST Servers, and WebConsole) have been upgraded and restarted.

To verify if the upgrade was successful, check if there is content in the directory./upgraded_result. Use the CLI to check all nodes:1. cd /iaas/cli

2. source ./iassrc

3. ./iaas-describe-nodes

4. Verify that nodes show an upgraded version.

If any of these upgrades failed, you can perform a yum upgrade manually andrestart the services.

Upgrading HBase and other nodes using a scriptFollow these steps to upgrade the HBase, hadoop, REST server, and the Zookeeperservers using an automated script.


Before you begin

Before you run the upgrade script, make sure that you are using the PXE Serverand that all HBase nodes are active. You also must set up automatic authenticationwith all of the management nodes and Compute Nodes involved in the upgrade,or these nodes will keep requesting that you type a pa@ssword.

About this task

You must get the iso package or the DVD for SmartCloud Provisioning and searchfor a file named IBM_SmartCloud_Provisioning-1.2.0.0-*.tgz included in the CDimage. This file is in the directory ./data/. Untar the file to find a file namedupgrade_hbase.sh in the directory ./deploy_scripts/.

To upgrade, follow either of the following procedures:

Procedure A (interactive):

Run ./upgrade_hbase.sh.

Provide the following information. Press Enter to accept the default values or quit:1. Specify a Zookeeper Server IP. The default value is detected from the PXE

Server.2. The upgrade process automatically detects all nodes information. You must

specify if the values are correct. If the values are correct, the upgrade startsand, if it is successful, the Zookeeper servers, hadoop, HBase, and Rest serverare upgraded. If the upgrade process is not successful, select Quit and switchto Procedure B.

Procedure B (non interactive):

Follow these steps:1. 1. Modify the settings in upgrade.cfg:

Example of upgrade.cfg:

storage_node=’10.0.0.13 10.0.0.14’storage_manager=’10.0.0.14’compute_node=’10.0.0.3’compute_manager=’10.0.0.3’webservice=’10.0.0.3’rest_server=’10.0.0.23 10.0.0.22’webconsole=’10.0.0.3’hbase_ips=’10.0.0.21 10.0.0.22 10.0.0.23’hadoop_ips=’10.0.0.21 10.0.0.22 10.0.0.23’hbase_master_ip=’10.0.0.21’hadoop_namenode_ip=’10.0.0.21’zookeeper_ips=’10.0.0.3’

Note: You must add HBase and hadoop manually if you want to upgradethem. Add a blank space among the IPs.

2. Run:./upgrade_hbase.sh upgrade_hbase.cfg

Note: If the detection of the Zookeeper’s IP from the PXE Server fails, youmust run the script and specify an IP:


./upgrade.sh upgrade.cfg 10.0.XX.XX

When you complete the mentioned steps, the upgrade starts. The upgrade processdetects all nodes, all Zookeeper servers, HBase, and hadoop. Then the processstops all kernel services and backs up the data. It upgrades and restarts theZookeeper servers, it performs a backup of hadoop (to IHC-0.20.2_bak_1.1) thenupgrades and restarts it, it upgrades and restarts HBase from 0.20.6 to 0.90.4, itupgrades and restarts the REST server. Finally, the upgrade process restores thedata and restarts all services.

Also, all kernel services (Compute Nodes, Storage Nodes, Web Service, and WebConsole) are restarted without upgrade.

Results

To verify if the upgrade was successful, visit the HBase master web athttp://<HBase-IP>:60010. Make sure the version of HBase is 0.90.4.

Use the CLI to check that all nodes are active:1. cd /iaas/cli

2. source ./iassrc

3. ./iaas-describe-nodes

4. ./iaas-describe-images to make sure that the data of the images is correct.

ConfiguringThese topics deal with the configuration of the SmartCloud Provisioning nodes, aswell as other configuration aspects.

Reconfiguring nodesThis topic describes how you can customize your environment to use differentconfiguration values.

About this task

The SmartCloud Provisioning installation sets default values when configuring thevarious nodes. Later on, you might want to change these settings to meet specificrequirements that your environment has.

Most configuration files can be found under /iaas/etc/.

Common settingsThe file is /iaas/etc/isaac.conf.

This configuration file defines data which is common to all components. Thefollowing is an example:{

"region":"iaas",

"zookeeper":{"zookeeper_servers":["10.0.0.1:2181,10.0.0.2:2181,10.0.0.3:2181"]},

"db":{"type":"hbase","rest_domain":"iaas_rest",


"cache_bucket":"cache"},

"messaging":{"type":"http","http_ports":[8880,8881]}

}

region It defines the region name of the SmartCloud Provisioning environment.An environment only has one region. The REST Servers, hyper bots,storage bots and web services information is kept in the Zookeeper. Theregion defines the root node of REST Servers, hyper bots, storage bots andweb services configuration in a Zookeeper. If you want to reuse an existingZookeeper or want to use a cluster of Zookeepers for two or moreSmartCloud Provisioning clouds, you must define different regionsaccording to your Zookeeper environment. The default value of a region isiaas. If this option is not defined in the configuration file, the default valueis used.

zookeeper

zookeeper_serversIt defines the zookeeper cluster addresses. You must add all yourzookeeper cluster nodes to this configuration.

db

type It defines the database type. The default value is hbase, and itmeans that it uses HBase as database.

rest_domainIt defines the context node of the Iaas REST Server domain. If thehyper bots want to find the Iaas REST Server, they look up theREST Server from this context node in zookeeper.

cache_bucketIt defines the cache table name in the database.

messaging

type A value of http means that this bot uses the HTTP protocol.

http_portsIt defines the port range used for communication. The defaultvalue is [8880, 8881]. If your system has a firewall configured, youmust modify the port range value that the firewall allows.

Compute Nodes settingsThe file is /iaas/etc/hyper_bot.conf.

This configuration file defines data for the Compute Node. The following is anexample:{"main":{

"rack":"rack1","name":"$(hostname)","per_cpu_instance":2,"per_instance_mem":512},

"hyper_type":"kvm","default_bridge" => "br0","trunk_interface"=>"eth0",


"cow_type":"dm","keypair":"mount_write","instance_type":{"xsmall" : {"ram":512, "vcpu":1, "swap":512},"small" : {"ram":1024, "vcpu":1, "swap":1024},"medium" : {"ram":2048, "vcpu":2, "swap":2048},"large" : {"ram":4096, "vcpu":2, "swap":4096},"xlarge" : {"ram":8192, "vcpu":2, "swap":4096}},//log_level: 0,1,2,3,4,5 debug,info,warn,error,fatal,unknown"regular_logging":{

"log_level": 1,"max_log_file_num": 10,"max_log_file_size": 1048576

},"manager_logging":{


},//event: new,terminated,lost,failed_to_launch,captured,attach_volume,\detach_volume,attach_address,detach_address,ip_update"instances_monitoring":{

"event":[],"data_writer":{

"writer":"DailyFileWriter","file_path":"/var/log/iaas/monitoring"

}},"vls_weight_policy":{

"timely_reduce":1,"timely_reduce_interval":3600,"launch_fail_reduce":6,"threshold_to_disable":100,"disable_action"=>"disable"

},"vmware":{

"nvram_path":"[datastore1]/iaas/iaas.nvram","swap_templates_dir":"[datastore1]/swap_templates","rdm_dir":"[datastore1]/iaas/rdm","instances_dir":"[datastore1]","iscsi_adapter":"vmhba37","mvm_name":"vmware_mvm","data_center":"cvt","esx":{

"vmware_op_timeout":60,"server":"10.0.139.80","username":"root","password":"xxxx"},

"vcenter":{"vmware_op_timeout":180,"server":"10.0.139.100","username":"Administrator","password":"xxxx"}

}}

main.nameIt defines the ID of Compute Nodes. It is unique in the same rack.

main.rackIt defines the rack of Compute Nodes for HYPER Rack Awareness. If theCompute Nodes are in same rack, you must define the same rack. It isunique in the same domain.

main.per_cpu_instance(Optional) It defines the number of virtual machines supported by oneprocessor. The default is 3.


main.per_instance_mem(Optional) It defines the average memory used by a virtual machine. Thedefault is 2048 MB.

hyper_typeIt defines the type of hypervisor.

kvm This bot is a kvm hypervisor.

xen This bot is a xen hypervisor.

vmwareThis bot is a VMware hypervisor.

default_bridgeIt defines the bridge network name on the node.

trunk_interfaceIt defines the trunk interface of the node.

cow_typeIt defines the type of copy-on-write format. Only valid for a kvm or xenhypervisor.

dm It means that a Device Mapper (DM) snapshot format is used bythe hyper bot.

qcow2 It means that a QEMU Copy On Write 2 format is used by thehyper bot.

keypairIt defines the method to enable the ssh keypair for a virtual machine.

mount_writeIt means that the bot mounts the virtual machine's cow file andappends the ssh public key to /root/.ssh/authorized_keys. Thisoption is only for Device Mapper (DM) snapshot format(cow_type=dm).

vmware_mount_writeThis option is for a VMware hypervisor.

instance_typeDefines the type of virtual machine to be provisioned. It includes ram size,virtual-cpu number and swap size. This configuration must be same in allthe hyper nodes within the same environment. This option is used tolaunch virtual machines with the IaaS API or the CLI.

loggingThere are two configurations: regular bot and manager bot loggingsettings.

regular_loggingThe logging settings for the regular bot.

manager_loggingThe logging settings for manager bot.

log_level(Optional) It defines the level of log. The default is 1 (INFO). Theoptions are: 0 (DEBUG), 1 (INFO), 2 (WARN), 3 (ERROR), 4(FATAL), 5 (UNKNOWN).


max_log_file_num(Optional) It defines the maximum number of log files. The defaultis 5.

max_log_file_size(Optional) It defines the max size of the log files. The unit is bytes.For example, a value of 41943040 means that the maximum size ofthe log files is 40 MB. The default is 20 MB.

instances_monitoringThis configuration defines how the monitoring data is collected fromvirtual machines.

event Defines what are the events that must be captured to collectmonitoring data. A value of [] (empty array) means that no eventmust be captured (disable data collecting). Possible events:

new The event of launching a virtual machine.

terminatedThe event of terminating a virtual machine.

lost The event of losing a virtual machine from the hypervisor.

failed_to_launchThe event of not being able of launching a virtual machine.

capturedThe event of capturing a virtual machine.

attach_volumeThe event of attaching a volume to a virtual machine.

detach_volumeThe event of detaching a volume from a virtual machine.

attach_addressThe event of attaching a public address to a virtualmachine.

detach_addressThe event of detaching a public address from a virtualmachine.

ip_updateThe event of changing the private IP of a virtual machine.

data_writerDefines how the monitoring data is going to be recorded.

writer A value of DailyFileWriter means that the data file nameis changed every day.

file_pathDefines the data file path for the writer. This value must bean existing directory.

vls_weight_policyDefines the virtual machine’s launch success weight policy. The virtualmachine’s launch success weight value is represented by vls_weight andits initial value is 0. You can retrieve this value by running the commandiaas-describe-nodes. It can be retrieved from the availability field. Forexample, if availability is kvm-6-30-0-18199, then the number 6 is the


available virtual machine’s number * (100+vls_weight) / 100. Note that 100is the value of option threshold_to_disable and that the availability for thevirtual machine in this example is 30.

timely_reduceReduces the vls_weight value. The default value is 1.

timely_reduce_intervalIt defines the time interval after which the vls_weight value isreduced by the value of timely_reducel. The default is 1 hour(3600 seconds), that is, when the hyper bot runs for an hour,vls_weight is reduced by the value of timely_reduce.

launch_fail_reduceReduces the vls_weight value if a virtual machine cannot belaunched. The default value is 6.

threshold_to_disableThreshold of vls_weight to run the action defined bydisable_action. The default value is 100. For example, if a hyperbot fails many times, then the vls_weight value is set to the valuedefined by threshold_to_disable. The hyper bot is rerun ordisabled.

disable_actionIt defines the action when vls_weight reaches the value ofthreshold_to_disable.

re-run The hyper bot is rerun.

disableThe hyper bot is disabled.

wmwareThis entry is for a VMware hypervisor.v nvram_path: BIOS template file for the virtual machines on the ESXi

server.v swap_templates_dir: directory to store swap templates for virtual

machines on the ESXi server.v rdm_dir: directory to store image files for virtual machines on the ESXi

server.v instances_dir: directory to store data of virtual machines on the ESXi

server.v iscsi_adapter: iscsi adapter name of the ESXi server.v mvm_name: virtual machine name of the ISAAC hyper bot running.v data_center: data center name that includes the ESXi server (only if you

want the hyper bot to manage the ESXi server through VCenter).v esx: This entry contains access information of the ESXi server.

– vmware_op_timeout: timeout value of invoking the VMware CLI forthe ESXi server. The default value is 60 seconds.

– server: IP address of the ESXi server.– username: user name to access the ESXi server.– password: password to access the ESXi server (the password supports

encrypted text, you can use the encrypt utility /iaas/bin/configuration_encrypt to encrypt your password).


v vcenter: This entry contains the access information of the VCenter server(only if you want the hyper bot to manage the ESXi server throughVCenter).– vmware_op_timeout: timeout value of invoking the VMmware CLI, for

the VCenter server. The default value is 180 seconds.– server: The IP address of the VCenter server– username: user name of the VCenter server.– password: password of the VCenter server (the password supports

encrypted text, you can use the encrypt utility /iaas/bin/configuration_encrypt to encrypt your password).

Changing the size of a virtual machine template

When you create a virtual machine, you cannot set or modify the size of itstemplate.

If you want to do so, you must modify the file /iaas/etc/hyper_bot.conf and youmust do so on each Compute Node of your cloud.

To change the configuration file, identify the instance_type tag inside the file:"instance_type":{

"xsmall" : {"ram":512, "vcpu":1, "swap":512},"small" : {"ram":1024, "vcpu":1, "swap":1024},"medium" : {"ram":2048, "vcpu":2, "swap":2048},"large" : {"ram":4096, "vcpu":2, "swap":4096},"xlarge" : {"ram":8192, "vcpu":2, "swap":4096}}

and modify the values to meet your needs.

Note: The hyper bots on the Compute Nodes must be restarted afterchanging the configuration file.

Storage Nodes settingsThe file is /iaas/etc/storage_bot.conf.

This configuration file defines data for the Storage Node. The following is anexample:{"id":"storge1","max_volume_size":50,"file_dirs":[

{"dir":"/iaas/iaas-store", "weight":"1"},{"dir":"/iaas/iaas-store1", "weight":"1"}],

//log_level: 0,1,2,3,4,5 debug,info,warn,error,fatal,unknown"regular_logging":{


},

"manager_logging":{"log_level": 1,


"max_log_file_num": 10,"max_log_file_size": 1048576

}}

id Defines the ID of the Storage Node.

max_volume_sizeDefines the maximum size of the volume that can be created. The unit isGB. The default is 50 GB.

file_dirsDefines the layout of files (images and volumes) on the file system.

dir For example, a "dir":"/iaas/iaas-store" layout for an imageresults into /iaas/iaas-store/files/img-01237.

Each file inside files/ dir is the actual image, not a directory.${mount_dir}/files/ is the real path.

weightValid only if you have multiple mount points. A higher weightmount directory is used to store the volume first.

loggingThere are two configurations: regular bot and manager bot loggingsettings.

regular_loggingThe logging settings for the regular bot.

manager_loggingThe logging settings for the manager bot.

log_level(Optional) It defines the level of log. The default is 1 (INFO). Theoptions are: 0 (DEBUG), 1 (INFO), 2 (WARN), 3 (ERROR), 4(FATAL), 5 (UNKNOWN).

max_log_file_num(Optional) It defines the maximum number of log files. The defaultis 5.

max_log_file_size(Optional) It defines the max size of the log files. The unit is bytes.For example, a value of 41943040 means that the maximum size ofthe log files is 40 MB. The default is 20 MB.

Web Console settingsThe file is /iaas/etc/webconsole.conf.

This configuration file defines data for the web console. The following is anexample:{"main": {"read_timeout": 30,"default_region": "vm_service","default_session_timeout": 1800,"allow_account_creation": true,"account_creation_url": "http://webservice:5678/registry/"},

"logging": {"level": "DEBUG"


},

"ldap":{"host":"ldap-server","port":389,"rootdn":"cn=admin, ou=user, ou=people, dc=example, dc=com","rootpw":"admin","oudn":",ou=user,ou=people,dc=example,dc=com","datasource":"internal"

},

"region":{"vm_service":{"description" : "VM Services","available": true,"web_service_url": "http://webservice:5678/query/","web_registry_url": "http://webservice:5678/registry/"}}}

Before you start the web console, navigate to its root directory and copy thesample configuration file:cp etc/webconsole.conf.example webconsole.conf

You can modify the configuration file as follows:v Change the URL of the web service as needed in your deployment.v Append additional service regions under attribute "region".v Disable the creation of accounts by setting "allow_account_creation": false

under attribute "main".

Web Service settingsThe file is /iaas/etc/webservice.conf.

This configuration file defines data for the Web Service. The following is anexample:{

"ws_port":5678,"worker_number":"10",

"ldap":{"host":"172.16.0.32","port":389,"rootdn":"cn=admin,dc=hslt,dc=cn-ibm-com","rootpw":"secret","oudn":",ou=user,ou=people,dc=hslt,dc=cn-ibm-com","datasource":"openldap"

},"quota_enabled":false,"nogroup":{

"quota":{"instnum":5, "nr_virt_cpu":5, \"memory":10240, "volume":20, "image":20, "eip":1}

},

"service_region":{"region_id":"vm_service","description":"VM Services"

}//log_level: 0,1,2,3,4,5 debug,info,warn,error,fatal"webservice_logging":{

"log_level": 1,


"max_log_file_num": 10,"max_log_file_size": 1048576

}}

ws_portDefines the port of the web service. The default is 5678.

workers_numberDefines the number of workers. The default is 5.

ldap.hostDefines the IP address of the LDAP server.

ldap.portIt defines the port of the LDAP server.

ldap.rootdnIt defines the LDAP rootdn of admin.

ldap.rootpwIt defines the LDAP rootpw of admin.

ldap.oudnIt defines the LDAP oudn of admin.

ldap.datasourceIt defines the type of data source type for LDAP:v bluepages: The data source is bluepages. Bluepages only supports user

authentication and it does not support the creation, modification,deletion, and the search of accounts in LDAP.

v openldap: The data source is openldap. Openldap also supports thecreation, modification, deletion, and the search of accounts in LDAP.

quota_enabledDefines if the resource quota checking function is enabled (true) ordisabled (false).

nogroup.quotaIt defines the default quota of an account. For example, {"instnum":5,"nr_virt_cpu":5, "memory":10240, "volume":20, "image":20, "eip":1}means that the default user quota is 5 virtual machines, 5 virtual CPU, 10GB of memory, 20 GB volumes, 20 GB images, 1 EIP. In other words, theuser can only run 5 virtual machines, can only have 5 virtual processors,cannot have more than 10 GB of total memory for his virtual machines, canonly create volumes for a total of 20 GB, can only capture or register atotal of 20 GB images, and can only allocate 1 elastic IP address.

service_region.region_idIt defines the ID of the service region. This service region is not same asthe region field in isaac.conf, but it is the region that defines theSmartCloud Provisioning service type. Its value can only be vm_service.

service_region.descriptionIt provides a description of the service region.

webservice_loggingDefines the web service logging settings.

log_level(Optional) Defines the level of log, The default is 1 (INFO). Othervalues are: 0 (DEBUG), 1 (INFO), 2 (WARN), 3 (ERROR), 4(FATAL), 5 (UNKNOWN).


max_log_file_num(Optional) Defines the maximum number of log files. The default is5.

max_log_file_size(Optional) It defines the maximum size of log files. For example, avalue of 41943040 means that the maximum size of log files is 40MB. The default is 20 MB.

Iaas REST Server settingsThe file is /iaas/iaas-rest-srv/conf/restserver.xml.

This configuration file defines data for the Iaas REST Server. The REST Server issimilar to a router that tells the Web Console where to forward requests (forexample, where the Web Service runs).

The following is an example:<?xml version="1.0" encoding="UTF-8"?><restconfig>

<input name="Region" value="iaas" /><input name="Domain" value="iaas-rest" /><input name="BindIP" value="" /><input name="MgrPort" value="33090" /><input name="DataPort" value="33080" /><input name="MaxWorkerNumber" value="50" /><input name="MaxClients" value="1000" /><input name="ZookeeperServer" \

value="10.10.30.221:2181,10.10.30.222:2181,10.10.30.223:2181"/><input name="ZookeeperTimeout" value="5000"/><input name="StartMR" value="false" />

</restconfig>

RegionDefines the region name of the SmartCloud Provisioning environment. Itmust be the same as the region value in isaac.conf on the hyper, storageand web services nodes.

DomainIt defines the domain of the REST Server. A SmartCloud Provisioningcloud has only one REST Server domain. The information of REST serversare kept in the Zookeepers. The domain defines the context node of theREST servers.

BindIPIt defines the REST Server socket listening IP address. Normally, this fieldis empty, meaning that the server socket will bind on 0.0.0.0.

MgrPortIt defines the REST Server management port. The REST Server managerthread listens on this port.

DataPortIt defines the REST server data service port. Web Service, Compute Nodes,and Storage Nodes use this port to transfer data with REST servers.

MaxWorkerNumberIt defines the REST Server maximum thread number for data service.

MaxClientsIt defines the REST Server maximum accept queue size for data service.


ZookeeperServerIt defines the Zookeeper cluster addresses. You must include the addressesfor all your Zookeeper nodes.

ZookeeperTimeoutIt defines the timeout value between Zookeeper Server and ZookeeperClient used in the REST server.

StartMRIt defines whether the metering service must be started in the REST Server.

HBase Client performance tuning

This configuration file can be modified by advanced users, who want to tune theHBase client performance.

The file is /iaas/iaas-rest-srv/conf/services.d/hbase.rsc. and the following isan example:<IRestServicesContainer url="/hbase/" scancache="30" hbaseClientPause="500" \hbaseClientRetriesNumber="3" tablePoolSize="200" \class="com.ibm.iaas.rest.service.hbase.HBaseServicesContainer">

Customizing the size of a virtual machineThis topic describes how you can customize the size of a virtual machine.

About this task

Follow this procedure to set the size of a virtual machine to extra small, small,medium, large, and extra large.

Procedure1. Edit /iaas/etc/hyper_bot.conf on a Compute Node to set the size to your

preferred value:"instance_type":{"xsmall" : {"ram":512, "vcpu":1, "swap":512},"small" : {"ram":1024, "vcpu":1, "swap":1024},"medium" : {"ram":2048, "vcpu":2, "swap":2048},"large" : {"ram":4096, "vcpu":2, "swap":4096},"xlarge" : {"ram":8192, "vcpu":2, "swap":4096}}

Only 5 virtual machine sizes are supported.

Note: Make sure each Compute Node has the same configuration.2. Run again the compute bot process on each Compute Node using the following

command that makes the new value valid:/iaas/hyper_bots/rubybots/re-run.sh

Other configuration tasksWhat follows is a series of topics that describe other configuration tasks.

How to configure the IP address of the PXE ServerThis topic describes how to configure networking for the PXE Server.


About this task

The PXE Server has RHEL installed. In this topic, it is assumed that the IP addressof the PXE Server is 192.168.96.2, its netmask is 255.255.255.0, and its gateway is192.168.96.1.

Do the following:# cd /etc/sysconfig/network-scripts# vi ifcfg-eth0

DEVICE=eth0TYPE=EthernetBOOTPROTO=staticIPADDR=192.168.96.2NETMASK=255.255.255.0NETWORK=192.168.96.0BROADCAST=192.168.96.255GATEWAY=192.168.96.1ONBOOT=yesNM_CONTROLLED=no

# cd /etc/sysconfig# vi network

NETWORKING=yesNETWORKING_IPV6=noHOSTNAME=firstboxGATEWAY=192.168.96.1

# service network restart

Configuring failover in DHCPThis topic explains how to set up an ISC DHCP daemon with load sharing andfailover capabilities.

Before you begin

Before implementing this task, you must have a fully functional SmartCloudProvisioning environment and a host computer with RHEL 6, which will functionas the secondary DHCP server.

About this task

Small-sized networks tend to have a single DHCP server, which can become asingle point of failure for a large number of hosts in the network. When the DHCPserver goes offline, DHCP client hosts lose their addresses and ability tocommunicate with the rest of the network. Since most desktop computers, andeven some servers, get their networking configuration via DHCP, such an outagecan result in a lot of downtime.

A UNIX network often uses the Internet Systems Consortium (ISC) DHCP server,which is widely available on Linux systems. The ISC DHCP server providesfailover capabilities that allow network administrators to offer a more robustDHCP service. A failover setup requires a little care, but it's fairly straightforwardto implement.

This task describes how to implement a failover setup.


Procedure

To configure the primary DHCP server:1. The primary DHCP server is the one that already serves the existing

SmartCloud Provisioning environment.2. Create the failover configuration file /etc/dhcp/dhcpd.failover that includes a

similar content:failover peer "example" { # Failover configuration

primary; # I am the primaryaddress 10.0.0.2; # First box IP addressport 647;peer address 10.0.0.3; # Peer’s IP addresspeer port 647;max-response-delay 60;max-unacked-updates 10;mclt 3600;split 128; # Leave this at 128, only defined on Primaryload balance max seconds 3;

}

3. Modify /etc/dhcp/dhcpd.conf to contain the line that defines the failoverconfiguration file. Make sure that the failover configuration file is included inthe local one, that is, make sure that dhcpd.local includes /etc/dhcp/dhcpd.failover.

4. Modify /etc/dhcp/dhcpd.local and add the failover option into the pooldeclaration so that it is similar to what follows:pool {

range 10.0.0.21 10.0.0.254;failover peer "example";deny dynamic bootp clients;

}

To configure the secondary DHCP server, which needs to be set up manually:1. Install DHCP on the secondary server:

yum install dhcpchkconfig dhcpd on

2. Create the failover configuration file /etc/dhcp/dhcpd.failover to containsimilar content:failover peer "example" { # Fail over configuration

secondary; # I am the secondaryaddress 10.0.0.3; # My ip addressport 647;peer address 10.0.0.2; # Peer’s ip addresspeer port 647;max-response-delay 60;max-unacked-updates 10;mclt 3600;load balance max seconds 3;

}

3. Copy dhcpd.conf, dhcpd.local, dhcpd.static from the primary server to/etc/dhcp and replace the existing ones, if any.

4. Copy the /etc/rndc.key from the primary server to the secondary.

To restart the DHCP service, run what follows on both primary and secondaryservers:/etc/init.d/dhcpd restart

Adding Storage NodesThis topic describes how to add a Storage Node to an existing environment. Youcan follow either of the described procedures.


About this task

Procedure ABoot the machine using the PXE Server. From the PXE boot menu selectStorage Node.

Procedure B

1. Plug in the power cable and Ethernet cable, and execute the followingcommand on the PXE Server:# cd /path/to/hslt-installer/deploy-scripts# ./add_storage.sh <MAC_address>

where <MAC_Address> is the MAC address of the new Storage Node.2. Press the power button on this machine, and select network bootup in

the bootup sequence list on the bootup screen. The machine is installedand configured automatically through the PXE service provided by thePXE Server. After the OS installation is complete, change the bootdevice from PXE to hard disk and then press Reboot.

Adding Compute NodesThis topic describes how to add a Compute Node to an existing environment.

About this task

Procedure ABoot the machine using the PXE Server. From the PXE boot menu selectCompute Node.

Procedure B

1. Adjust the BIOS settings of your new Compute Node to enable PXEbooting, and save the MAC address of the ethernet card with whichyou have enabled PXE.

2. Login to the DHCP DNS server and edit the DHCP configuration file.This file is often located at /etc/dhcp/dhcpd.conf. Add this code withyour hostname and IP address to enable PXE booting from your DHCPDNS server:host <compute_node_hostname> \{hardware ethernet xx:xx:xx:xx:xx:xx, \fixed-address 10.0.0.xxx;}

3. Verify the dhcpd configuration and restart the dhcpd service if there areno configuration errors:# service dhcpd configtest# service dhcpd restart

4. Reboot the newly-added Compute Node.

Setting up a redundant DHCP-DNS serverThis topic describes how you can set up a DHCP-DNS server to have a master orslave mode.

About this task

When you complete the SmartCloud Provisioning installation, the PXE serverbecomes the DHCP-DNS server with master mode and its IP address becomes10.0.0.1.


This task explains how you can set up another DNS server in slave mode with IPaddress 10.0.0.30.

Procedure1. For the DNS slave, you can start a virtual machine or use a physical machine.2. On the slave node:

yum install bind

Copy /etc/named.conf and /etc/rndc.key from the PXE server.3. On the PXE server, edit /etc/named.conf as follows:

zone "MoonRiver.cloud.com" {type master;file "forward-lookup.db";allow-transfer { 10.0.0.30; };allow-update {key "rndc-key";};notify yes;

};

zone "0.0.10.in-addr.arpa" {type master;file "reverse-lookup.db";allow-transfer { 10.0.0.30; };allow-update {key "rndc-key";};notify yes;

};

4. On the slave server with IP address 10.0.0.30:a. Edit /etc/named.conf as follows:

zone "MoonRiver.cloud.com" {type slave;file "forward-lookup.db";masters { 10.0.0.1; };notify yes;

};

zone "0.0.10.in-addr.arpa" {type slave;file "reverse-lookup.db";masters { 10.0.0.1; };notify yes;

};

b. Remove the block for zone "0.0.127.in-addr.arpa".c. Run:

touch /var/named/cache.db

d. Run:chown -R named.named /var/named

5. Run the following command on both servers to restart the dns service:/etc/init.d/named restart

Backing up and restoring HBaseThere are two utilities for Cloud administrators to backup and restore HBase.

They are iaas-db-backup.sh and iaas-db-restore.sh. These utilities are stored inthe REST Server, and installed in the directory /iaas/iaas-rest-srv/bin in thevirtual machines with a REST Server installed. There can be one or more RESTServers.


Note: If you have less than 5 HBase nodes, a REST Server is installed on eachHBase node except hbase-1. If you have 5 or more HBase nodes, a RESTServer is installed on each HBase node except hbase-1, hbase-2 and hbase-3.

You can select any of these to execute the backup and restore of HBase.

To do so, SSH to a virtual machine with a REST Server installed, and:# cd /iaas/iaas-rest-srv/bin# ./iaas-db-backup.sh /mydata/hbase iaas

Note: iaas-db-backup.sh accepts two arguments:v /mydata/hbase where you put the backup data.v iaas that defines the service region to backup.

You can find the service region name of your deployment in/iaas/etc/isaac.conf, which can be found in any Compute Node.

After issuing the given backup command, you must get a screen output as follows:/iaas/iaas-rest-srv/bin /iaas/iaas-rest-srv/bin- /mydata/hbase exists, creating /mydata/hbase/isaac-dbbak-2011-08-05-16-11

to save backup data ...- Directory /mydata/hbase/isaac-dbbak-2011-08-05-16-11 created successfully.- Check existing tables in DB ...- done.- Parsing tables ...- Tables to backup: addresses iaas_cache iaas_job_stats iaas_jobs

images instances key_pair persistent_instances quota_templates rolessecret_keys service_region settings test_hbase usergroups usersvlans volumes

- Exporting data of table iaas.addresses ...- done.- Exporting data of table iaas.iaas_cache ...- done.- Exporting data of table iaas.iaas_job_stats ...- done.- Exporting data of table iaas.iaas_jobs ...- done.- Exporting data of table iaas.images ...- done.- Exporting data of table iaas.instances ...- done.- Exporting data of table iaas.key_pair ...- done.- Exporting data of table iaas.persistent_instances ...- done.- Exporting data of table iaas.quota_templates ...- done.- Exporting data of table iaas.roles ...- done.- Exporting data of table iaas.secret_keys ...- done.- Exporting data of table iaas.service_region ...- done.- Exporting data of table iaas.settings ...- done.- Exporting data of table iaas.test_hbase ...- done.- Exporting data of table iaas.usergroups ...- done.- Exporting data of table iaas.users ...- done.- Exporting data of table iaas.vlans ...


- done.- Exporting data of table iaas.volumes ...- done.

- DB backup complete.

The backup is complete.

If your HBase runs into problems or is corrupted, you want to restore it to acertain backup. You can issue the following command:# cd /iaas/iaas-rest-srv/bin# ./iaas-db-restore.sh /mydata/hbase/isaac-dbbak-2011-08-05-16-11 iaas

Note: iaas-db-restore.sh accepts two arguments:v /mydata/hbase/isaac-dbbak-2011-08-05-16-11 , which is where the data

is restored.v iaas, which defines the service region to be restored.

Moving kernel services virtual machinesThis topic describes how to move kernel services virtual machines among physicalmachines.

About this task

All kernel services run on virtual machines that might be distributed on differentphysical machines. Moving a kernel service means moving the virtual machinerunning the kernel service. To describe the procedure of moving virtual machinesamong physical machines, assume that the kernel service to be moved is a webconsole, running on compute1, and being moved to compute2.

Procedure1. Log on to the compute1 console, using SSH. Use virsh to display the list of

virtual machines running on compute1:# virsh listId name status----------------------------------

2 webconsole running

You can see the web console virtual machine in the list being displayed.2. Log on to the web console virtual machine and stop it:

a. # virsh console webconsole

b. Log on using the virtual machine’s root password and shutdown the virtualmachine:# shutdown -h now

c. Press Ctrl-] return to the physical machine console of compute1.3. Copy the image file webconsole.par from compute1 to compute2. The file's

location is /iaas/local-storage/vm. Copy the virtual machine’s definition XMLfile (webconsole.xml) from compute1 to compute2. The file's location is/iaas/inst-scripts/webconsole.xml.

4. Log on to the compute2 console, using SSH. Use virsh to create a virtualmachine:# virsh create /iaas/inst-scripts/webconsole.xml

5. After the new web console virtual machine is booted up, you can log on to themachine to start the web console application by running /iaas/webconsole/re-


run.sh. Close the console on the virtual machine, and press Ctrl-] to return tothe physical machine compute2's console.a. # virsh console webconsole

b. Log on with the virtual machine’s root password and start the web consoleapplication:# /iaas/webconsole/re-run.sh

c. Close the virtual machine’s console and press Ctrl-] to return to thephysical machine compute2's console.

Integrating an external LDAP serverThe LDAP support from SmartCloud Provisioning allows the web servicecomponent to authenticate users via an OpenLDAP Server.

About this task

Assume that you have an OpenLDAP Server running on localhost and configuredwith the following directory:o=foo,dc=hslt,dc=cn-ibm-com

|+ + ou=people,o=foo,dc=hslt,dc=cn-ibm-com

|+ cn=user1,ou=people,o=foo,dc=hslt,dc=cn-ibm-com|+ cn=user2,ou=people,o=foo,dc=hslt,dc=cn-ibm-com

The following sample configuration in /iaas/etc/webservice.conf is for theprevious sample directory:"ldap":{

"host":"localhost","port":389,"rootdn":"cn=admin,dc=hslt,dc=cn-ibm-com","rootpw":"<admin-password>","oudn":",ou=user,ou=people,dc=hslt,dc=cn-ibm-com","datasource":"openldap"

}

Note: OpenLDAP, Netscape Directory SDK, and Windows Active Directoryare supported.

Note: Also userDN is supported, which means there is no configurable filter.For example, for username user1, it searches for cn=user1 combined withoudn value.

Connecting existing storage devicesIn the majority of cases, enterprise users have some existing storage devices, forexample, IBM DS 3000/5000/8000 series. This topic describes how to add thesestorage devices into a SmartCloud Provisioning environment.

Procedure1. Add HBA cards to the Storage Nodes that are going to use the storage devices.2. Connect the Storage Nodes and storage devices with a SAN Switch.3. Create LUNs in the storage devices, and use or mount LUNs in the Storage

Nodes, as if they were a local hard drive.4. Configure the file_dirs in the Storage Nodes’ configuration file

/iaas/etc/storage_bot.con.


5. Rerun the Storage Nodes, by doing so all LUNs created in the storage devicesare used by SmartCloud Provisioning.

Configuring a virtual LANThis topic describes how you can configure a VLAN.

Before you begin

To implement a virtual LAN, you need a switch with VLAN support. Each portconnected to a Compute Node must be configured as trunk mode. For each VLANthat you want to use, create it in the switch and add all the Compute Node portsto it.

VLAN 0 through 4 are reserved for SmartCloud Provisioning. Use other VLANports, from 5 to 4095.

Note: SmartCloud Provisioning does not manage the configuration of thephysical switch. You must configure the switch manually. The configurationof the switch port as trunk mode is different according to different switchmodels. For example, for some switches, you can configure the port as trunkmode by enabling VLAN tagging.

About this task

Useful information about VLANs can be found at: http://www.linuxhorizon.ro/vlans.html.

Procedure1. Register a VLAN:

As an administrator, you register a VLAN by running the commandiaas-register-vlan. For example, to register a VLAN using port 11 and vlan_11as VLAN name, run:./iaas-register-vlan -m 11 -n vlan_11

Note: If multiple users want to deploy virtual machines in the sameVLAN, the administrator can register the same VLAN number multipletimes with unique VLAN names. When multiple users have allocated theVLANs that have the same VLAN number, they can deploy the virtualmachines in the same VLAN.

2. Allocate the VLAN by running the command:./iaas-allocate-vlan

3. Run the virtual machines:As a user who has allocated a VLAN, you can launch virtual machines withinthe VLAN. For example, to launch two RHEL virtual machines in vlan_11, run:./iaas-deploy-vms -i rhel6-x64 -n 2 -b "default,vlan_11"

Note: When launching virtual machines, the default VLAN must bedesignated using the -b option.

4. Configure a VLAN IP for each virtual machines. After you have launched thevirtual machines, a VLAN IP is set to eth1 on each virtual machine.


http://www.linuxhorizon.ro/vlans.html

http://www.linuxhorizon.ro/vlans.html

5. Verify the isolation of the VLAN. The expected result is that all virtualmachines within the same VLAN are accessible, and that virtual machines indifferent VLANs are isolated.

What to do next

To release a VLAN, you must be the user who allocated it and you must terminateall virtual machines in the VLAN before releasing it. To release a VLAN, run thecommand iaas-release-vlan. For example, to release a VLAN with name vlan_11,run:./iaas-release-vlan -n vlan_11

To deregister a VLAN, you must be an administrator and you must release theVLAN before deregistering it. To deregister a VLAN, run the commandiaas-deregister-vlan. For example, to deregister a VLAN with name vlan_11, run:./iaas-deregister-vlan -n vlan_11

Releasing and deregistering a VLAN can also be accomplished using the web UI.

Default portsThe bot software running on the various nodes, with standard and manager roles,uses specific ports to serve requests.

The following ports are being used for listening:

Table 1. Ports used by the bot software on the SmartCloud Provisioning nodes.

Node and bot software Port number Comment

Compute Node with standard role 8880, 80 v 8880 is used to accept commands

v 80 is used to serve user requests fromvirtual machines

Compute Node with manager role 8880, 808881

8881 is used in addition

Storage Node with standard role 8880

Storage Node with manager role 88808881

8881 is used in addition

REST Server 33080

Web Service 8880 and higher Starting from 8880, each worker uses aunique port to serve a request (worker 0uses 8880, worker 1 uses 8882, worker 2uses 8883, and so forth)

Securing the communicationThis topic describes how to enable secure communications.

About this task

NGINX, a high-performance open source web server, is used to provide httpsconnection to some kernel services (web services, web console, and admin webconsole). Using NGINX does not prevent continuing to have plain http, which canbe blocked by leveraging standard techniques, such as IP table rules.


Web services are secured by signing requests via the private key, therefore even ifthe information sent in the http posts is not encrypted, this does not represent asecurity exposure. SmartCloud Provisioning can provide https connection to theweb services through NGINX.

Note: Using NGINX causes the command line not to work anymore becausethe CLI is not enabled to handle certificates and https requests. SmartCloudProvisioning supports both options and you might want to use the CLI onprivate networks but only enable SSL on public networks for example.

This task provides samples on how to configure the NGINX, and how to set IPtables to block http traffic.

In SmartCloud Provisioning, the Web Services, Web Console, and theadministrative console virtual machines default are installed and configured tohave a running instance of the NGINX web server. The NGINX web serverconfiguration is stored in a specific configuration file under /etc/nginx/conf.d. Forexample, for the administrative console, the file is called adminconsole.conf andhas the following content:server {

listen 443;server_name adminconsole;ssl on;ssl_certificate /etc/nginx/iscp.cert;ssl_certificate_key /etc/nginx/iscp.key;location / {

proxy_pass http://<admin_console:<port>;/;}

}

The configuration tells NGINX to redirect secure traffic (https on port 443) to thedefault http port used by the administrative console. The certificates are used toenable SSL communications between external clients and the NGINX web server.

To block incoming HTTP traffic, you can set the following iptables rules (theexample is for a service running on port 8080):#allow local connection from NGINXiptables -A INPUT -s 127.0.0.1 -p tcp --dport 8080 -j ACCEPT#block all the other incoming connectionsiptables -A INPUT -p tcp --dport 8080 -j REJECT

Installing and configuring additional optionsThe following topics describe how to install and configure additional componentsof SmartCloud Provisioning.

Installing topology managementThis topic describes how to install and launch the topology managementcomponent for the web console. This is an optional task.

Before you begin

A standard installation of SmartCloud Provisioning must be available.


About this task

The web console cluster must use the public address capability provided bySmartCloud Provisioning. Each node in the cluster, including the Apache balancernode and the web console node, needs a public address. Additional publicaddresses can be optionally provided to support the scaling out of the cluster.

Modify the file /var/lib/tftpboot/topo/address-pool on the PXE server to use thepublic addresses, one address per line. The following is a possible version ofsample address-pool file for a web console cluster of 6 nodes (1 Apache balancerand 5 web console servers):10.0.0.1110.0.0.1210.0.0.1310.0.0.1410.0.0.1510.0.0.1610.0.0.1710.0.0.1810.0.0.1910.0.0.20

What to do next

To launch the topology management component and the web console cluster, runtopo_run.sh in /iaas/inst-scripts in one of the Storage Nodes or ComputeNodes.

The CLI client comes from package isaac-topo-cli-xxx.rpm.

Installing and configuring the Iaas APIThe IaaS API is an optional component in the product environment.

The installation of the API is not performed as part of the product installation. Youmust follow these topics to deploy the API in your environment.

The following prerequisites are required to install the IaaS API:v RHEL 6.0 or laterv Ruby 1.8.7 or laterv RubyGems 1.3.7 or laterv sinatra 1.0 or laterv yajl-ruby 0.7.8 or later

You can install the API in the virtual machine of the web service, which has all thenecessary prerequisites.

Installing the Iaas APIIf you have a yum repository that includes SmartCloud Provisioning packages inthe machine, for example in the web service virtual machine, the installation of theAPI is very simple.

About this task

Run:yum install isaac-iaasapis


After the installation, the API is located in /iaas/iaasapis.

Configuring the Iaas APIThis topic describes how to configure the API after it has been installed.

About this task

You must first copy /iaas/iaasapis/config/hslt.yml.sample to/iaas/iaasapis/config/hslt.yml:cd /iaas/iaasapiscd configcp hslt.yml.sample hslt.yml

The configuration file defines all the required information for the API, such as theweb service endpoint, the admin private key, and more. The following is anexample:admin:

admin_private_key: "/iaas/keys/admin.private.key"admin_access_id: "admin"

service:service_endpoint: "http://webservice:5678/query/"service_public_key: "/iaas/keys/vm_service.public.key"

logging:path: "/var/log/iaas"level: "INFO"max_log_file_num: 5log_file_size: 10

where:

admin_private_keyDefines the admin private key file location.

admin_access_idDefines the access ID of the administrator. The access ID does not have tobe admin-only, but the access must be with admin role.

service_endpointDefines the web service end point.

service_public_keyDefines the service public key file location.

path Defines the application log file directory. The file name is iaas_apis.log.

level Defines the logging level. You can use either DEBUG, INFO, WARN,ERROR, or FATAL.

max_log_file_numDefines the max log files number.

log_file_sizeDefines the maximum file size of one log file in megabytes. For example, ifconfigured with max_log_file_num equal 5 and log_file_size equal 10, thefile size of the IaaS API is at most 50 MB. The log is somethinglike:/var/log/iaas/iaas_apis.log,/var/log/iaas/iaas_apis.log.0,/var/log/iaas/iaas_apis.log.1 and so forth.


Starting, stopping, and browsing the API

About this task

To start he API:cd /iaas/iaasapis./re-run.sh

To stop the API:cd /iaas/iaasapis./re-run.sh stop

To browse the API:http://<isaac_iaasapis_ip_address>:8090/


Appendix. Support information

If you have a problem with your IBM software, you want to resolve it quickly. Thissection describes the following options for obtaining support for IBM softwareproducts:v “Using IBM Support Assistant”v “Obtaining fixes”v “Receiving weekly support updates” on page 56v “Contacting IBM Software Support” on page 56

Using IBM Support AssistantThe IBM Support Assistant is a free, stand-alone application that you can install onany workstation. You can then enhance the application by installingproduct-specific plug-in modules for the IBM products you use.

The IBM Support Assistant saves you time searching product, support, andeducational resources.

The product-specific plug-in modules provide you with the following resources:v Support linksv Education links

For more information, see the IBM Support Assistant Web site athttp://www.ibm.com/software/support/isa/.

If your product does not use IBM Support Assistant, use the links to support topicsin your information center. In the navigation frame, check the links for resourceslisted in the ibm.com® and related resources section where you can search thefollowing resources:v Support and assistance (includes search capability of IBM technotes and IBM

downloads for interim fixes and workarounds)v Training and certificationv IBM developerWorks®

v IBM Redbooks®

v General product information

If you cannot find the solution to your problem in the information center, searchthe following Internet resources for the latest information that might help youresolve your problem:v Forums and newsgroupsv Google.com

Obtaining fixesA product fix might be available to resolve your problem. To determine what fixesare available for your IBM software product, follow these steps:1. Go to the IBM Support & download Web page at http://www.ibm.com/

support/us/.


http://www-306.ibm.com/software/support/isa

http://www.ibm.com/support/us/

http://www.ibm.com/support/us/

2. Click Downloads and drivers in the Support & downloads section.3. Select one software category from the Category list.4. Select one product from the Sub-category list.5. Type more search terms in the Search within results if you want to refine your

search.6. Click Search.7. From the list of downloads returned by your search, click the name of a fix to

read the description of the fix and to optionally download the fix.

For more information about the types of fixes that are available, see the IBMSoftware Support Handbook at http://techsupport.services.ibm.com/guides/handbook.html

Receiving weekly support updatesTo receive weekly e-mail notifications about fixes and other news about IBMproducts, follow these steps:1. Go to the IBM Software Support Web site at www.ibm.com/software/support.2. Click My support in the upper right corner of the page.3. If you have already registered for My support, sign in and skip to the next

step. If you have not registered, click register now. Complete the registrationform using your e-mail address as your IBM ID and click Submit.

4. Click Edit profile.5. In the Products list, select Software. A second list is displayed.6. In the second list, select a product segment, for example, Application servers.

A third list is displayed.7. In the third list, select a product sub-segment, for example, Distributed

Application & Web Servers. A list of applicable products is displayed.8. Select the products for which you want to receive updates, for example, IBM

HTTP Server and WebSphere® Application Server.9. Click Add products.

10. After selecting all products that are of interest to you, click Subscribe to emailon the Edit profile tab.

11. Select Please send these documents by weekly email.12. Update your e-mail address as needed.13. In the Documents list, select Software.14. Select the types of documents that you want to receive information about.15. Click Update.

If you experience problems with the My support feature, you can obtain help inone of the following ways:

OnlineSend an e-mail message to [email protected], describing your problem.

By phoneCall 1-800-IBM-4You (1-800-426-4968).

Contacting IBM Software SupportIBM Software Support provides assistance with product defects.


http://techsupport.services.ibm.com/guides/handbook.html


http://www.ibm.com/software/support

Before contacting IBM Software Support, your company must have an active IBMsoftware maintenance contract, and you must be authorized to submit problems toIBM. The type of software maintenance contract that you need depends on thetype of product you have:v For IBM distributed software products (including, but not limited to, Tivoli®,

Lotus®, Rational® products, and DB2® and WebSphere products that run onWindows or UNIX operating systems), enroll in Passport Advantage® in one ofthe following ways:– Online: Go to the Passport Advantage Web site at

www.lotus.com/services/passport.nsf/WebDocs/Passport_Advantage_Home

and click How to Enroll.– By phone: For the phone number to call in your country, go to the IBM

Software Support Web site at techsupport.services.ibm.com/guides/contacts.html and click the name of your geographic region.

v For customers with Subscription and Support (S & S) contracts, go to theSoftware Service Request Web site at https://techsupport.services.ibm.com/ssr/login

v For customers with IBMLink, CATIA, Linux, OS/390®, iSeries®, pSeries®,zSeries®, and other support agreements, go to the IBM Support Line Web site athttp://www.ibm.com/services/us/index.wss/so/its/a1000030/dt006

v For IBM eServer™ software products (including, but not limited to, DB2 andWebSphere products that run in zSeries, pSeries, and iSeries environments), youcan purchase a software maintenance agreement by working directly with anIBM sales representative or an IBM Business Partner. For more informationabout support for eServer software products, go to the IBM Technical SupportAdvantage Web page at www.ibm.com/servers/eserver/techsupport.html.

If you are not sure what type of software maintenance contract you need, call1-800-IBMSERV (1-800-426-7378) in the United States or, from other countries, go tothe contacts page of the IBM Software Support Handbook on the Web(techsupport.services.ibm.com/guides/contacts.html) and click the name of yourgeographic region for phone numbers of people who provide support for yourlocation.

To contact IBM Software Support, follow these steps:1. “Determine the business impact of your problem”2. “Describe your problem and gather background information” on page 583. “Submitting problems” on page 58

Determine the business impact of your problemWhen you report a problem to IBM, you are asked to supply a severity level.Therefore, you need to understand and assess the business impact of the problemyou are reporting. Use the following criteria:

Severity 1 Critical business impact: You are unable to use the program,resulting in a critical impact on operations. This conditionrequires an immediate solution.

Severity 2 Significant business impact: The program is usable but isseverely limited.

Severity 3 Some business impact: The program is usable with lesssignificant features (not critical to operations) unavailable.

Appendix. Support information 57

http://www.lotus.com/services/passport.nsf/WebDocs/Passport_Advantage_Home

http://techsupport.services.ibm.com/guides/contacts.html


https://techsupport.services.ibm.com/ssr/login

https://techsupport.services.ibm.com/ssr/login

http://www.ibm.com/services/us/index.wss/so/its/a1000030/dt006

http://www.ibm.com/servers/eserver/techsupport.html


Severity 4 Minimal business impact: The problem causes little impact onoperations, or a reasonable circumvention to the problem hasbeen implemented.

Describe your problem and gather background informationWhen explaining a problem to IBM, be as specific as possible. Include all relevantbackground information so that IBM Software Support specialists can help yousolve the problem efficiently. To save time, know the answers to these questions:v What software versions were you running when the problem occurred?v Do you have logs, traces, and messages that are related to the problem

symptoms? IBM Software Support is likely to ask for this information.v Can you re-create the problem? If so, what steps where performed to re-create

the problem?v Did you make any changes to the system? For example, did you make changes

to the hardware, operating system, networking software, and so on?v Are you currently using a workaround for the problem? If so, be prepared to

explain the workaround when you report the problem.

Submitting problemsYou can submit your problem to IBM Software Support in one of two ways:

OnlineClick Report problems on the IBM Software Support site athttp://www.ibm.com/software/support/probsub.html. Type yourinformation into the appropriate problem submission form.

By phoneFor the phone number to call in your country, go to the contacts page ofthe IBM Software Support Handbook at techsupport.services.ibm.com/guides/contacts.html and click the name of your geographic region.

If the problem you submit is for a software defect or for missing or inaccuratedocumentation, IBM Software Support creates an Authorized Program AnalysisReport (APAR). The APAR describes the problem in detail. Whenever possible,IBM Software Support provides a workaround for you to implement until theAPAR is resolved and a fix is delivered. IBM publishes resolved APARs on theIBM product support Web pages daily, so that other users who experience thesame problem can benefit from the same resolutions.


http://www.ibm.com/software/support/probsub.html



Notices

This information was developed for products and services offered in the U.S.A.IBM may not offer the products, services, or features discussed in this document inother countries. Consult your local IBM representative for information on theproducts and services currently available in your area. Any reference to an IBMproduct, program, or service is not intended to state or imply that only that IBMproduct, program, or service may be used. Any functionally equivalent product,program, or service that does not infringe any IBM intellectual property right maybe used instead. However, it is the user's responsibility to evaluate and verify theoperation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matterdescribed in this document. The furnishing of this document does not give youany license to these patents. You can send license inquiries, in writing, to:

IBM Director of LicensingIBM CorporationNorth Castle DriveArmonk, NY 10504-1785, U.S.A.

For license inquiries regarding double-byte (DBCS) information, contact the IBMIntellectual Property Department in your country or send inquiries, in writing, to:

Intellectual Property LicensingLegal and Intellectual Property LawIBM Japan Ltd.1623-14, Shimotsuruma, Yamato-shiKanagawa 242-8502 Japan

The following paragraph does not apply to the United Kingdom or any othercountry where such provisions are inconsistent with local law:

INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THISPUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHEREXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIEDWARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESSFOR A PARTICULAR PURPOSE.

Some states do not allow disclaimer of express or implied warranties in certaintransactions, therefore, this statement might not apply to you.

This information could include technical inaccuracies or typographical errors.Changes are periodically made to the information herein; these changes will beincorporated in new editions of the publication. IBM may make improvementsand/or changes in the product(s) and/or the program(s) described in thispublication at any time without notice.

Any references in this information to non-IBM Web sites are provided forconvenience only and do not in any manner serve as an endorsement of those Websites. The materials at those Web sites are not part of the materials for this IBMproduct and use of those Web sites is at your own risk.


IBM may use or distribute any of the information you supply in any way itbelieves appropriate without incurring any obligation to you.

Licensees of this program who wish to have information about it for the purposeof enabling: (i) the exchange of information between independently createdprograms and other programs (including this one) and (ii) the mutual use of theinformation which has been exchanged, should contact:

IBM Corporation2Z4A/10111400 Burnet RoadAustin, TX 78758 U.S.A.

Such information may be available, subject to appropriate terms and conditions,including in some cases, payment of a fee.

The licensed program described in this document and all licensed materialavailable for it are provided by IBM under terms of the IBM Customer Agreement,IBM International Program License Agreement or any equivalent agreementbetween us.

Information concerning non-IBM products was obtained from the suppliers ofthose products, their published announcements or other publicly available sources.IBM has not tested those products and cannot confirm the accuracy ofperformance, compatibility or any other claims related to non-IBM products.Questions on the capabilities of non-IBM products should be addressed to thesuppliers of those products.

This information contains examples of data and reports used in daily businessoperations. To illustrate them as completely as possible, the examples include thenames of individuals, companies, brands, and products. All of these names arefictitious and any similarity to the names and addresses used by an actual businessenterprise is entirely coincidental.

COPYRIGHT LICENSE:

This information contains sample application programs in source language, whichillustrate programming techniques on various operating platforms. You may copy,modify, and distribute these sample programs in any form without payment toIBM, for the purposes of developing, using, marketing or distributing applicationprograms conforming to the application programming interface for the operatingplatform for which the sample programs are written. These examples have notbeen thoroughly tested under all conditions. IBM, therefore, cannot guarantee orimply reliability, serviceability, or function of these programs. You may copy,modify, and distribute these sample programs in any form without payment toIBM for the purposes of developing, using, marketing, or distributing applicationprograms conforming to IBM's application programming interfaces.

TrademarksIBM, the IBM logo, and ibm.com are trademarks or registered trademarks ofInternational Business Machines Corporation in the United States, other countries,or both. If these and other IBM trademarked terms are marked on their firstoccurrence in this information with a trademark symbol (® or ™), these symbolsindicate U.S. registered or common law trademarks owned by IBM at the time thisinformation was published. Such trademarks may also be registered or common


law trademarks in other countries. A current list of IBM trademarks is available onthe Web at "Copyright and trademark information" at http://www.ibm.com/legal/copytrade.shtml.

Microsoft, Windows, Windows NT, and the Windows logo are trademarks ofMicrosoft Corporation in the U.S. and other countries.

UNIX is a registered trademark of The Open Group in the United States and othercountries.

Linux is a trademark of Linus Torvalds in the United States, other countries, orboth.

Java and all Java-based trademarks and logos are trademarks or registeredtrademarks of Sun Microsystems, Inc. in the U.S., and other countries.

Other company, product, and service names may be trademarks or service marksof others.

Notices 61

http://www.ibm.com/legal/copytrade.shtml

http://www.ibm.com/legal/copytrade.shtml

Index

Aarchitecture 1

CCompute Node 1Compute Nodes

installation 11customer support

see Software Support 56

Eeducation 55

Ffixes, obtaining 55

IIBM Redbooks 55IBM support assistant 55Internet, searching to find software problem resolution 55

Kkernel services

installation 21

Nnodes 1

Pproblem determination

describing problem for IBM Software Support 58determining business impact for IBM Software Support 57submitting problem to IBM Software Support 58

problem resolution 55product

architecture 1product nodes 1PXE Server 1

setup with script 9

RRedbooks, IBM 55

Ssoftware support 55Software Support

contacting 56

Software Support (continued)describing problem for IBM Software Support 58determining business impact for IBM Software Support 57submitting problem to IBM Software Support 58

Storage Nodesinstallation 11

support 55support assistant 55support information 55

ZZookeeper Server 1


��

Product Number: 5725-C88

Installing

Documents

Transcript of Installing