Migration Guide - documentation.suse.com · HPE Helion OpenStack 8 dropped support for internally...

Migration Guide

Publication Date: April 17, 2020

Contents

1 Migration from HPE Helion OpenStack 5.0.x to HPE Helion OpenStack5.SCP 2

2 Migration from HPE Helion OpenStack 5.SCP to Ardana.Newton 20

3 Migration from Ardana.Newton to Ardana.Pike 23

4 Applying Product Updates 29

1 HPE Helion OpenStack 8

Important: Use Latest DocumentationAlways use the latest online version of the documentation for the migration process.

1 Migration from HPE Helion OpenStack 5.0.x to HPEHelion OpenStack 5.SCP

The migration procedure is based on the following assumptions:

Skilled customer sta should be readily available to assist in the migration, with expertisein the areas of cloud administration, network administration, storage administration, andknowledge of data models and configuration settings used by the Cloud Lifecycle Manager.Ideally, systems involved in the upgrade should be physically accessible in case any oper-ations become necessary that cannot be administered via their remote network interfaces.The customer is responsible for backing up all the control plane and deployer nodes to anexternal data store before starting the migration.

Important: No Migration RollbackWe do not support rollback of migration after the process has been started.

Metering, monitoring, and logging (MML) data will not be migrated (a clean installationwill be done using a new MML back-end included with HPE Helion OpenStack 5.SCP).

Upgrading will require re-imaging the operating system on the physical disks containingthe root partition of your controller nodes. Existing data on these disks, which may includeaudit logs and Swift objects, may be lost as a result. Ensure that your model les and anyother important les on the deployer are copied to a safe location. (Note that the primaryOpenStack MySQL database will undergo a backup/restore as part of the upgrade processto ensure that its contents are preserved.)

Review your existing partitioning model to understand how your data will be affected byre-imaging. Any data that is not on a disk separate from the primary root partition (i.e.Swift objects or audit logs) will be lost during re-imaging. Note that the control planenodes must be booted from physical drives and must not have any attached SAN disks.

2

Migration from HPE Helion OpenStack 5.0.x to HPE Helion OpenStack 5.SCP HPE Helion

OpenStack 8

All Compute Nodes are running supported Linux distributions, and any prior hLinux Com-pute Nodes have been removed from the cloud. (For details about migrating instances fromhLinux to SLES or RHEL Compute Nodes, please review the following HOS 5 documenta-tion: Release Notes, Installation Guide, and Operations Guide.)

SUSE Linux Enterprise Server 12 SP3 for SLES Compute Nodes

RHEL 7.3 for RHEL Compute Nodes

There are some limitations to migrating instances. For details about migrating instancesfrom hLinux to SLES or RHEL Compute Nodes, please review the following HOS 5 docu-mentation: Release Notes, Installation Guide and Operations Guide.

HPE Helion OpenStack 8 dropped support for internally deployed storage such as Cephand VSA. Data must be moved out of existing internal storage backends to an external SEScluster before performing an upgrade.

The cloud data model does not contain services which are not included in HPE HelionOpenStack 8, such as Ceph and VSA.

In order to migrate ESX compute VMs as part of this process, existing ESX hosts must becapable of migrating VMs between them, as rebalancing workloads will be necessary whileESX hosts are individually removed and restored as members of the clouds.

Any customizations to virtual environments used by OpenStack components will be lostduring the upgrade process. These may include third party extensions, regardless of origin,that have not been included in HPE Helion OpenStack 8.

No significant issues or alarms are actively being reported by Monasca and OperationsConsole.

Preparation Phase

1. Because all Freezer content will be obsolete upon migrating to the new platform, it isrecommended to stop Freezer jobs and eliminate historic backups.

(old-deployer)> source ~/backup.osrc(old-deployer)> for id in $(freezer job-list -f value | awk '{print $1}') do freezer job-stop $iddone(old-deployer)> for container in $(openstack container list -f value); do for backup in $(openstack object list $container -f value); do

3


OpenStack 8

openstack object delete $container $backup echo "Deleting $backup" donedone

2. If your cloud includes VMware ESX compute clusters, save the EON service activationtemplate for future reference, as the EON service will no longer exist once migration iscomplete.

(old-deployer)> source ~/service.osrc(old-deployer)> eon get-activation-template \ --filename ~/activationtemplate.json --type esxcluster

3. Obtain the HPE Helion OpenStack 5.SCP installation ISO and use the mount command tomake its contents available on the existing deployer:

(old-deployer)> mkdir hos-5.scp(old-deployer)> sudo mount HELION5SCP_ISO_FILE hos-5.scp(old-deployer)> tar -xzvf hos-5.scp/hos/hlm-migration-tools.tar.gz(old-deployer)> sudo umount hos-5.scp

4. Run the validation playbook to ensure that the current cloud load and configuration iscompatible with the migration process:

(old-deployer)> cd hlm-migration-tools/ansible(old-deployer)> ansible-playbook \ -i ~/scratch/ansible/next/hos/ansible/hosts/verb_hosts \ pre-migration-validations.yml

The validation playbook checks that:

The HPE Helion OpenStack version number is either 5.0.1 or 5.0.3

HPE Helion OpenStack 8-compatible Linux distributions on Compute Nodes

System clocks are synchronized across cloud nodes

HA capacity of Control Nodes (L3/DHCP agent)

Configuration and capacity of Control Nodes (Swift, Cinder)

Data model compatibility with HPE Helion OpenStack 8 (no vsa, ceph, eon-*, orcmc-service)

4


OpenStack 8

Warning: Check Validation ResultsReview the contents of /var/log/pre-migration-validations.log immediately to dis-cover if any errors were identified while running the validation playbook. Any iden-tified issues must be resolved before proceeding with the migration.

5. Set up SMT repository hosting on a SLES server external to your cloud so that operatingsystem and software updates can be applied during migration without waiting for addi-tional download steps.

6. Identify a candidate seed Control Node in the primary control plane. This node must meetthe following requirements:

Networking connectivity is equivalent to that of the existing deployer.

It is a member of the database cluster (running the MySQL service).

It is not actively hosting any singleton OpenStack services ( cinder-volume or no-va-consoleauth ). Test for empty output with the following command:

(each controller)> ps -ef | egrep 'cinder-volume|nova-consoleauth'

7. Consider the effect of control plane downtime on existing DHCP leases:

Default DHCP lease time is 48 hours as defined in the ~/helion/hos/

ansible/roles/neutron-common/templates/neutron.conf.j2 le by thedhcp_lease_duration parameter.

DHCP leases are administered by Neutron and dnsmasq. This functionality will notbe available during the control plane downtime.

DHCP clients typically renew their leases either after 50% of the lease time or uponreboot.

Normally, this leaves a 24-hour window for control plane downtime at any pointbefore workload's DHCP leases run out and cannot be renewed. While this is typ-ically enough for performing a migration, increasing the DHCP lease time to 144hours (518400 seconds) is recommended to ensure that lease expiration will not bea concern that affects your migration, even if unexpected delays in the process areencountered.

5


OpenStack 8

To view the upcoming timeouts of DHCP leases:

1. Determine which servers are running the DHCP agents

(old-deployer)> neutron agent-list | awk '/DHCP/{print $7}'mig-cp1-neut-m2-mgmtmig-cp1-neut-m1-mgmt

2. From the old deployer, SSH to each host listed above, and display the leasetimes for active VMs.

(old-deployer)> ssh mig-cp1-neut-m2-mgmt (mig-cp1-neut-m2)> for i in $(cat /var/run/neutron/dhcp/*/leases | awk '{print $1}'); do date -d @$i done(mig-cp1-neut-m2)> exit...

To increase the DHCP lease time before migrating, open the roles/neutron-com-mon/templates/neutron.conf.j2 le from the ~/helion/hos/ansible directo-ry, and edit the dhcp_lease_duration parameter. Then, apply the change with:

(old-deployer)> git commit -a -m "DHCP lease adjustment"(old-deployer)> ansible-playbook -i hosts/localhost config-processor-run.yml(old-deployer)> ansible-playbook -i hosts/localhost ready-deployment.yml(old-deployer)> cd ~/scratch/ansible/next/hos/ansible(old-deployer)> ansible-playbook -i hosts/verb_hosts neutron-reconfigure.yml

This will reset the duration for new leases, but the existing IP assignments to VMswill wait until their original DHCP lease expires before being refreshed with theupdated lease duration. Thus, these steps should be performed at least 2/3rds of thelease time prior to the planned start of migration.Continue to monitor the upcoming DHCP lease expirations as described above untilall existing leases have been refreshed to a lengthier period, and then continue withthe migration.

6


OpenStack 8

8. Prepare the cloud data model for migration:

In the ~/helion/my_cloud/definition/data/disks_*.yml les used by yourserver role that includes monitoring services, replace the existing Vertica partitiondefinition with two Cassandra partitions:

- name: cassandra_db size: (original size used by vertica partition, minus 1%) mount: /var/cassandra/data fstype: ext4 mkfs-opts: -O large_file consumer: name: vertica- name: cassandra_log size: 1% mount: /var/cassandra/commitlog fstype: ext4 mkfs-opts: -O large_file

Edit ~/helion/my_cloud/definition/data/control_plane.yml and identify thecluster definition that already includes an entry for mysql in its service-components.

Ensure that lifecycle-manager is included in the same cluster's service-compo-nents list. If it was necessary to add this as a new entry, it will also need to be re-moved from its original position in the deployer.

Update the servers.yml entry for all nodes in the control plane to specify dis-tro-id: sles12sp3-x86_64 in each one. (Otherwise, they will be re-imaged withhLinux.)

Note: VMware ESX NodesAll ESX proxy and OVSvApp nodes should be included in the control planeupdates to the new distro-id as described above.

7


OpenStack 8

If your environment includes a 3PAR storage backend, edit the le ~/he-

lion/my_cloud/config/cinder/cinder.conf.j2 and update your driver to itsnew name (from HP3PARFCDriver to HPE3PARFCDriver , or from HP3PARISCSI-Driver to HPE3PARISCSIDriver .)

Commit your changes to the Git model:

(old-deployer)> git commit -a -m "Prepare for HOS 5.1 installation"

9. If the intended seed node also currently hosts Neutron L3 routing services, evacuate therouters from the node so that it can be re-imaged without interruption to tenant networks.

In Distributed Virtual Router (DVR) enabled clouds, responsibility for network rout-ing is shared across the compute nodes, and downtime of the seed node as a con-troller will have no adverse effect on workload network traffic. To see if DVR is en-abled in your cloud, run the following command and check the entries in the "Dis-tributed" column.

(old-deployer)> source ~/service.osrc(old-deployer)> neutron router-list

If you observe non-distributed routers, then the next step will ensure that any re-sponsibility for those routers' function is migrated away from the seed node to othernodes in the controller cluster, so that routing remains available even while the seednode is taken offline. (This step is not necessary if all your routers are distributed.)

(old-deployer)> cd ~/scratch/ansible/next/hos/ansible(old-deployer)> cp -r ~/hlm-migration-tools/ansible/* ./(old-deployer)> ansible-playbook -i hosts/verb_hosts \ neutron-router-evacuate.yml --limit=SEED_HOST

SEED_HOST represents the name of the seed node, for example, hos-cp1-dbmq-m3 .

8


OpenStack 8

Important: Host SelectionFor correct behavior, you must use the --limit argument to select the targetseed host. This will output a success or error message at the end of its run.

10. Re-image the seed controller as a SLES-based system:

Display the current network connections of the seed controller (before re-imaging)for reference and save it locally.

(seed)> ip a

Reinstall the operating system on the seed node's primary disk using an ISO, cobbler(from the existing deployer), or other provisioner. Note that support for provisioningwith cobbler will be available only if the seed node is configured to use Legacy BootMode; a setting of UEFI Boot Mode will require manual installation.

If using cobbler, begin the seed node re-imaging by running:

(old-deployer)> cd ~/helion/hos/ansible(old-deployer)> sudo cobbler system remove --name SEED_NODE_ID(old-deployer)> ansible-playbook -i hosts/localhost cobbler-deploy.yml(old-deployer)> ansible-playbook -i hosts/localhost bm-reimage.yml -e nodelist=SEED_NODE_ID

Replace SEED_NODE_ID with the ID of the seed node from the servers.yml le.Cobbler will initialize the seed node's networking to use the IP address of your orig-inal deployer as the default gateway. This will negatively impact the network's op-eration later on in the migration, and should be updated after the system is upand running. On the new seed node, edit or create the le /etc/sysconfig/net-work/routes and change the address in the line starting with "default" to point toyour environment's actual default gateway. It has the format:

default 192.168.1.254 - -

9


OpenStack 8

You may be able to copy this from the value specified in that le on your existingdeployer. Then, restart networking with:

(seed)> sudo systemctl restart network

Otherwise, if installing manually or using a proprietary tool for provisioning, refor-mat the primary partition (preserving the overall partitioning scheme based on thedata model) and install the SUSE Linux Enterprise Server 12 SP3 operating system.Follow the instructions as described in Book “Installing with Cloud Lifecycle Manager”,

Chapter 3 “Installing the Cloud Lifecycle Manager server”, but when creating the user ac-count and group, use the name stack instead of ardana .It is very important to avoid changing partitioning layouts or reformatting any vol-umes other than the primary partition during the installation. This is necessary topreserve any Swift object data that may be present on the server.

11. Transfer configuration and les from your existing cloud to the new seed node

The disk volume on the new seed may be smaller than the corresponding volumeon the old deployer. It may be necessary to rstremove extraneous les from yourdeployer (such as old qcow2 or ISO images), then transfer configuration and lesfrom the old deployer to the new seed node:

(old-deployer)> rsync -avP ~ SEED_IP:/home(old-deployer)> scp -r /tmp/hel* SEED_IP:/tmp

If cobbler will be used from the new seed for provisioning the remaining cloud Con-trol Nodes:

Ensure that the seed node is able to reach IPMI interfaces referenced in theservers.yml le:

(seed)> for s in `grep "ilo-ip" ~/helion/my_cloud/definition/data/servers.yml | grep -v "#" | cut -d ":" -f 2 | tr -d " " ` do ping -q -c 2 $s

10


OpenStack 8

done

If additional routes are required for the seed to reach the IPMI interfaces, addthese network routes to the seed. For example:

(seed)> sudo ip route add IPMI_NETWORK via IPMI_GATEWAY_FROM_SEED

Kill the DHCP services running on the old deployer:

(old-deployer)> sudo pkill dhcpd

12. On the new seed node, remove the package repository that was configured when its op-erating system was rst installed. Then, mount the operating system image to make itavailable to the HPE Helion OpenStack installer.

Important: ISO filenamesFor correct behavior, the filenames of the images described below must matchexactly their names in the user's home directory.

If your cloud includes RHEL Compute Nodes, ensure that ~/rhel7.iso was eithertransferred from the original deployer or is downloaded to the user's home directory.Ensure that ~/sles12sp3sdk.iso was either transferred from the original deployeror is downloaded to the user's home directory.Ensure that ~/sles12sp3.iso was either transferred from the original deployer oris downloaded to the home directory on the seed node and named sles12sp3.iso .Then, mount it:

(seed)> sudo mkdir -p /media/cdrom(seed)> sudo mount /home/stack/sles12sp3.iso /media/cdrom

If you installed the seed node using cobbler from the original deployer, replace theprimary package repository on the new seed node with a link to the content deliveredon the ISO media instead of a remote network location.

(seed)> sudo zypper rr SLES12-SP3-12.3-0(seed)> sudo zypper ar file:/media/cdrom SLES12-SP3-12.3-0

11


OpenStack 8

(seed)> sudo zypper ref

13. Install HPE Helion OpenStack 5.SCP on the seed node by downloading and unpacking HPEHelion OpenStack 5.SCP installation content:

(seed)> cd ~(seed)> mkdir -p hos-5.scp(seed)> sudo mount HELION5SCP_ISO_FILE hos-5.scp(seed)> tar -xf hos-5.scp/hos/hos-5.1.0-*.tar(seed)> sudo mkdir -p /var/log/apache2(seed)> cd ~/hos-5.1.0(seed)> ./hos-init.bash

14. Update repository references of Compute Node as appropriate to point to the seed node.

On each SLES compute node, remove all the repositories except for SLES-OS andSLES-SDK . Use zypper lr , to list them and zypper rr <REPO_NAME> to removethem. If your compute nodes were kept up to date using the SUSE Updates repositoryvia an SMT server, they will get future updates via the SMT server running on oraccessible by the future deployer.

On each RHEL compute node, edit the repositories located in either /etc/yum.re-pos.d or /etc/yum/repos.d to redirect any IP addresses from the old deployerto point to the new seed node. (These references are likely to be found in the lescobbler-config.repo or rhel73.repo .)Then, refresh the yum cache by running:

(rhel-compute)> sudo yum clean all

15. If VMware ESX compute hosts are present in the cloud, encrypt your ESX login credentialsfor embedding in the data model. This will require use of the HOS_USER_PASSWORD_EN-CRYPT_KEY environment variable and the hosencrypt.py script, which is also used forencrypting the following other data model elements:

Ironic OneView password

Cinder back-end passwords (optional)

Barbican master key for simple_crypto_plugin backend (optional)

HPE DCN Nuage VSD password

12


OpenStack 8

The use of hosencrypt.py for encryption is required for VMware ESX credentials (unlikesome of the items above, which may optionally be stored in plaintext.)If you have used hosencrypt.py previously for any of the items above, then you havealready selected an encryption key for use with your cloud. This encryption key mustbe exported into the environment variable HOS_USER_PASSWORD_ENCRYPT_KEY when thepassword is encrypted, and again before running playbooks such as site.yml . If this isyour rst time using such encryption, choose a new, unique encryption key and proceed:

(seed)> export HOS_USER_PASSWORD_ENCRYPT_KEY=ENCRYPTIONKEY(seed)> ./hosencrypt.pyunencrypted value? <VCENTERPASSWORD>@hos@U2FsdGVkX180/6dJIPPt49epntxAOFP0MtpjWsOaYzc=

Capture the resulting encrypted value and paste it into the password eld of ~/he-lion/my_cloud/definition/data/pass_through.yml , replacing the value previouslyset there by EON. If you have questions about the structure or content in pass_through.yml,comments regarding its use are provided with the HPE Helion OpenStack 5.SCP ex-ample data models such as in ~/helion/examples/entry-scale-esx-kvm-vsa/da-ta/pass_through.yml .

16. In your cloud's data model, edit the control plane definition and add a new data modelentry for nova-placement-api into the cluster which already includes nova-api .

(seed)> cd ~/helion/my_cloud/definition/data(seed)> vi control_plane.yml(seed)> git commit -a -m "Adding nova-placement-api"

17. Prepare for running the migration playbook:

(seed)> cd ~/helion/hos/ansible(seed)> ansible-playbook -i hosts/localhost config-processor-run.yml(seed)> ansible-playbook -i hosts/localhost ready-deployment.yml

18. Initialize ansible host facts and ensure that the existing cloud is fully addressable by thenew deployer:

(seed)> cd ~/scratch/ansible/next/hos/ansible(seed)> ansible-playbook -i hosts/verb_hosts hlm-refresh-facts.yml

13


OpenStack 8

19. Note: OpenStack Clouds OnlyThe following step is only necessary on OpenStack clouds and should not be usedin standalone Swift deployments.

Flush the Keystone token cache by running the following command on any node hostingthe Keystone API service. (This may be the seed node in an entry-scale environment, or adedicated controller node in a mid-scale environment.)

(controller)> sudo /opt/stack/service/keystone/venv/bin/keystone-manage token_flush

Then, prune the primary database to remove data that is no longer relevant:

(seed)> cd ~/scratch/ansible/next/hos/ansible(seed)> ansible-playbook -i hosts/verb_hosts percona-prune.yml

PROCEDURE 1: MIGRATION PHASE

Warning: DowntimeThe cloud will not be available until all of the following steps are completed.

1. If your cloud's data model is password-protected, use the following method to save thepassword in a local le before proceeding:

(seed)> touch ~/.vault_pass.txt(seed)> chmod 600 ~/.vault_pass.txt(seed)> vi ~/.vault_pass.txt

Ensure that the contents of the le are set to include only the plaintext vault password.Then, after saving it, run export ANSIBLE_VAULT_PASSWORD_FILE=~/.vault_pass.txtin preparation for upcoming steps.

2. Stop or pause OpenStack services on the control plane (excluding the seed that has beenfreshly imaged):

(seed)> cd ~/scratch/ansible/next/hos/ansible(seed)> ./hlm-quiesce.sh --limit '!SEED_HOST'

The SEED_HOST name must exactly match the host name that is referenced in hosts/verb_hosts , such as ardana-cp1-c1-m1 .

14


OpenStack 8

3. Create a database backup by retrieving MySQL database contents from a running databasenode in the database cluster.

(seed)> cd ~/scratch/ansible/next/hos/ansible(seed)> ansible-playbook -i hosts/verb_hosts percona-vertica-removal-cleanup.yml(seed)> ansible-playbook -i hosts/verb_hosts percona-export.yml \ -e dbcontent=~/mysql_dump.sql

4. Re-image the remaining servers in the control plane.

Re-image the remaining servers in the control plane. If the original deployer is astandalone deployer, then it can be preserved until later, to assist with recovery (ifneeded). If the deployer function is combined with a controller, then ensure that theprevious section's steps to transfer configuration and les using rsync has completedso that no important data is lost.Reinstall the operating systems on the control plane primary disks using an ISO,cobbler, or other provisioner. If using cobbler, you will need to create the grub configles for all the SLES nodes using the following commands:

(seed)> cd ~/helion/hos/ansible(seed)> ansible-playbook -i hosts/localhost cobbler-deploy.yml(seed)> ansible-playbook -i hosts/localhost prepare-sles-grub2.yml \ -e nodelist=NODE_IDs(seed)> ansible-playbook -i hosts/localhost bm-reimage.yml \-e nodelist=NODE_IDs

NoteDo not reimage the Compute Nodes, an original standalone deployer, or seed.Identify the set of remaining control plane nodes from the full list of systemsaddressable by cobbler, as shown by cobbler system list , and providethem as a comma-separated list in the nodelist parameter.

If installing the operating system manually (which can be done by booting from theHPE Helion OpenStack 5.SCP installation ISO) or using a proprietary tool for provi-sioning, reformat the primary partition (preserving the overall partitioning schemebased on the data model) and install the SUSE Linux Enterprise Server 12 SP3 oper-ating system. Follow the instructions as described in Book “Installing with Cloud Life-

cycle Manager”, Chapter 3 “Installing the Cloud Lifecycle Manager server”, but when creat-ing the user account and group, use the name stack instead of ardana .

15


OpenStack 8

ESX (nova-proxy) nodes must also be rebuilt at this time, and must be done using theabove-mentioned manual process (not using cobbler). For these virtual machines,reinstall the operating system using the SUSE Linux Enterprise Server 12 SP3 instal-lation ISO. Note that, when installing, you must select a partitioning method usinglogical volumes and establish a primary logical volume which is reasonably smallin size (i.e. 50GB). It will be automatically expanded according to your data modelduring upcoming steps in the migration process. Again, the proper system user ac-count name and group for these VMs is stack .

In a mid-scale cloud topology, some data les may have been stored on the non-primary drive(s) in the system. If this was the case, the wipe_disk.yml playbookshould be used (after the nodes have been rebuilt) to purge the old data.Make a note of the output of the following command:

(seed)> cd ~/scratch/ansible/next/hos/ansible(seed)> ansible-playbook -i hosts/verb_hosts hlm-refresh-facts.yml

Specify your own list of nodes matching the identifiers shown in the results of theprevious command. However, do not run the wipe_disk.yml playbook on any Com-pute Nodes or nodes containing Swift object storage.

(seed)> ansible-playbook -i hosts/verb_hosts wipe_disks.yml \ --limit dbmq-m1,dbmq-m2,dbmq-m3,mtrmon-m1,mtrmon-m2,mtrmon-m3

5. The repos should now be set up on all servers. Verify that this is the case with the following:

(seed)> cd ~/scratch/ansible/next/hos/ansible(seed)> ansible resources -a "zypper ref" --become

All issues with the repositories must be resolved before continuing. All servers exceptRHEL Compute Nodes should complete successfully. All SLES servers should reference therepo on port 79 on the new seed.

6. Create the log directory for the configuration processor, and then run the HPE HelionOpenStack 5.SCP installation. Enable database import from the snapshot that was capturedfrom your cloud by using the dbcontent ag:

(seed)> mkdir -p /var/log/configuration_processor(seed)> cd ~/scratch/ansible/next/hos/ansible

16


OpenStack 8

(seed)> ansible-playbook -i hosts/verb_hosts -vvv site.yml -e dbcontent=~/mysql_dump.sql

7. This will be a time-consuming installation, and should be allowed to complete withoutinterruption. Once it is done, the cloud should again be operational, with the exception ofmonitoring components that will remain offline until the upgrade to HPE Helion OpenS-tack 8 is fully complete.

8. If OVSvApp nodes are present in the environment, they should now be re-imaged with anoperating system installed from the SUSE Linux Enterprise Server 12 SP3 ISO. Follow thedirections above regarding ESX (nova-proxy) nodes exactly, doing the same for OVSvAppnodes.

NoteNote that, while a VMware ESX host's OVSvApp node is offline, other running VMson the same host will experience degraded network connectivity. To ensure maxi-mum uptime, migrate all other VMs away from a host before reinstalling its OVS-vApp node. Once the node is fully online again, the affected VMs can be returnedto their original host.

After re-imaging an OVSvApp node, install HPE Helion OpenStack 5.SCP components onit from your seed using:

(seed)> ansible-playbook -i hosts/verb_hosts site.yml \ -l 'OVSVAPP_HOSTS'

More than one OVSvApp host can be installed at once by separating their names usingcommas. Note that each host should be identified by its Ansible-friendly name, matchingthe way it is referred to in the hosts/verb_hosts le.

PROCEDURE 2: CLEANING UP

Note: No DowntimeThese steps do not cause service downtime and can be performed while the cloudis up and running.

17


OpenStack 8

1. Deployer nodes offer limited disk space, and it is appropriate at this time to transfer the ~/mysql_dump.sql le to an external location outside the cloud (where it can be availablein case of any recovery needs) and remove it from the seed node:

(seed)> rm ~/mysql_dump.sql

2. If desired, transfer the deployer role in the HPE Helion OpenStack 5.SCP cloud to a dif-ferent node, such as one that originally served as a dedicated deployer.

If the deployer will be moving to an entirely different cluster (for example, Data-base cluster to Controller cluster or standalone Deployer cluster), additional softwarepackages need to be installed.

Edit ~/helion/my_cloud/definition/data/control_plane.yml and movethe lifecycle-manager from its current location into the new desired cluster'sservice-components list. Commit the change:

(seed)> git commit -a -m "Enable lifecycle-manager for new deployer"

Execute these steps to prepare for running playbooks:

(seed)> cd ~/helion/hos/ansible(seed)> ansible-playbook -i hosts/localhost config-processor-run.yml(seed)> ansible-playbook -i hosts/localhost ready-deployment.yml

At this point, it is appropriate to re-image a standalone deployer that wouldhave been left intact through the upgrade steps to this point, to ensure that theSLES operating system is loaded.

(seed)> cd ~/helion/hos/ansible(seed)> ansible-playbook -i hosts/localhost bm-reimage.yml \ -e nodelist=ORIGINAL_DEPLOYER

Install the lifecycle-manager services on the new deployer and removecloud-wide references to the seed:

(seed)> cd ~/scratch/ansible/next/hos/ansible(seed)> ansible-playbook -i hosts/verb_hosts site.yml -l NEW_DEPLOYER_HOST(seed)> ansible resources -i hosts/verb_hosts -m shell \ -a "zypper rr SLES12-SP3-12.3-0 || true" --become

18


OpenStack 8

NEW_DEPLOYER_HOST must match the node's identity in the le hosts/ver-b_hosts (for example, ardana-cp1-c0-m1 )

Stop the deployer-related services and repositories running on the current seed:

(seed)> sudo systemctl stop dhcpd(seed)> sudo systemctl disable dhcpd(seed)> sudo systemctl stop cobblerd(seed)> sudo systemctl disable cobblerd(seed)> sudo sed -i /sles12sp3.iso/d /etc/fstab(seed)> sudo umount /media/cdrom

If you used the cobbler mechanism from your seed node earlier to re-image the nodewhich will now be serving as your new deployer, its zypper repos should be changedto use local sources instead of retrieving content from the seed node via the network:

(new-deployer)> sudo zypper rr SLES-OS(new-deployer)> sudo zypper ar file:/media/cdrom SLES12-SP3-12.3-0

The old repos on other SLES systems will automatically be reset to the new deployerlater when the osconfig-run.yml playbook is run.

Unmount any ISOs that are no longer necessary on the seed node:

(seed)> sudo umount /home/stack/hos5.scp

Copy the user's home directory and certificates from the seed to the new deployer:

(seed)> rsync -avP ~/ NEW_DEPLOYER_IP:/home/stack(seed)> scp -r /tmp/hel* NEW_DEPLOYER_IP:/tmp

Connect to the new deployer and initialize it for use:

(new-deployer)> cd ~(new-deployer)> rm -rf .ansible_fact_cache(new-deployer)> sudo mkdir -p /media/cdrom(new-deployer)> sudo mount /home/stack/sles12sp3.iso /media/cdrom(new-deployer)> sudo mount HELION5SCP_ISO_FILE hos-5.scp(new deployer)> tar -xf hos-5.scp/hos/hos-5.1.0-*.tar(new deployer)> cd ~/hos-5.1.0(new deployer)> ./hos-init.bash(new deployer)> cd ~/helion/hos/ansible(new deployer)> ansible-playbook -i hosts/localhost cobbler-deploy.yml(new-deployer)> ansible-playbook -i hosts/localhost config-processor-run.yml

19


OpenStack 8

(new-deployer)> ansible-playbook -i hosts/localhost ready-deployment.yml

On each RHEL Compute Node, edit the repositories located in either /etc/yum.re-pos.d or /etc/yum/repos.d to redirect any IP addresses from the seed node topoint to the new deployer. (These references are likely to be found in the les cob-bler-config.repo or rhel73.repo .)Then, refresh the yum cache by running:

(rhel-compute)> sudo yum clean all

Use the new deployer to inform all other cloud nodes of its new role.

(new-deployer)> cd ~/scratch/ansible/next/hos/ansible(new-deployer)> ansible-playbook -i hosts/verb_hosts osconfig-run.yml

2 Migration from HPE Helion OpenStack 5.SCP toArdana.Newton

The following procedure describes the process of migrating from HPE Helion OpenStack 5.SCPto Ardana.Newton.

From this point forward, all commands should be executed on the new deployer node (whichwill be the same as the earlier seed node unless this role was explicitly transferred using theinstructions at the end of the prior section).

PROCEDURE 3: PREPARATION

1. Convert the helion Git repository to an openstack Git repository for use by Ardana:

(stack)> cd ~(stack)> cp -r helion openstack(stack)> cd openstack(stack)> git checkout site

2. If your environment included a SES storage backend, remove all references to it fromyour data model. This may be done by using git revert COMMIT-ID to revert a specificcommit including the SES backend definition, or git checkout hos -- FILENAME torestore an entire le to its original version (followed by git diff --staged to inspect

20 Migration from HPE Helion OpenStack 5.SCP to Ardana.Newton HPE Helion OpenStack 8

the difference between les and ensure that you ar satisfied with the final condition ofthe configuration le.) Committing these edits will be necessary for the following les inthe ~/openstack git repository:

hos/ansible/roles/_CND-CMN/templates/cinder.conf.j2

hos/ansible/roles/NOV-CMP-KVM/templates/hypervisor.conf.j2

hos/ansible/roles/GLA-API/templates/glance-api.conf.j2 (if the SES back-end was additionally configured for use with Glance)

3. Save a record of all other manual changes that have been previously committed to thehos/ansible directory, as you will need to reapply them later:

(stack)> git diff hos -- hos/ansible > /tmp/hos.diff

Forcibly undo all the changes identified:

(stack)> rm -rf hos/ansible(stack)> git checkout hos -- hos/ansible

4. Reset the codebase to make it ready for Ardana, then transform the rest of the repository:

(stack)> git commit -a -m "Resetting codebase in prep for Ardana"(stack)> git branch -m hos ardana(stack)> git checkout ardana(stack)> sed -i 's/hos/ardana/' .gitignore(stack)> git commit -a -m "Prep for Ardana"(stack)> git mv hos ardana(stack)> git mv hos_extensions ardana_extensions(stack)> for i in $(find -L my_cloud/config -xtype l) do ln -sfn $(readlink $i | sed 's/\/hos\//\/ardana\//') $idone(stack)> git commit -a -m "Conversion to Ardana"(stack)> git checkout cp-persistent(stack)> sed -i 's/component: helion-ca/component: ardana-ca/' \ my_cloud/persistent_state/private_data_control-plane-1.yml(stack)> sed -i 's/helion_internal_ca/ardana_internal_ca/' \ my_cloud/persistent_state/private_data_control-plane-1.yml(stack)> git commit -a -m "Conversion to Ardana"(stack)> git checkout site(stack)> git merge ardana -m "Conversion to Ardana"


Any merge conflicts that arise through this procedure can be addressed in the mannerdescribed in Book “Installing with Cloud Lifecycle Manager”, Chapter 10 “Using Git for Configu-

ration Management”, Section 10.3 “Resolving Git merge conflicts”.

5. If the le /tmp/hos.diff contains changes that were saved prior to the repo conversionabove, reinstate them in the ardana/ansible directory.

(stack)> if [ -s /tmp/hos.diff ]; then sed 's/ $[ab]\?[\/]\?$hos\/ansible/ \1ardana\/ansible/g' /tmp/hos.diff | git applyfi

If merge conflicts are reported, some les may need to be edited manually to reconcile theupdates to le paths and directory structure that make the codebase suitable for upgradingto Ardana.Newton. Further information about merge operations can be found at Book

“Installing with Cloud Lifecycle Manager”, Chapter 10 “Using Git for Configuration Management”,

Section 10.3 “Resolving Git merge conflicts”.

(stack)> git add -A(stack)> git commit -a -m "Reinstated hos playbook customizations (batch)"

6. Remove and archive the HPE Helion OpenStack 5.SCP-related content:

(stack)> mv ~/scratch ~/scratch.scp(stack)> sudo rm /etc/apache2/sites-enabled/deployer_venv_server.conf

7. Prepare the ardana_packager working directory to include existing SUSE repositories:

(stack)> sudo mkdir -p /opt/ardana_packager/ardana/sles12/zypper(stack)> sudo ln -s /opt/hlm_packager/hlm /opt/ardana_packager/hlm(stack)> sudo ln -s /opt/hlm_packager/hlm/sles12/zypper/OS \ /opt/ardana_packager/ardana/sles12/zypper/OS

8. Install Ardana.Newton content onto the deployer by mounting the installation media andrunning the following commands:

(stack)> cd ~(stack)> sudo mkdir -p /media/ardana-0.35.0(stack)> sudo mount ~/HELION5CLM_ISO_FILE /media/ardana-0.35.0(stack)> tar -xf /media/ardana-0.35.0/ardana/ardana-0.35.0*.tar(stack)> cd ardana-0.35.0(stack)> ./ardana-init.bash


If merge conflicts are reported, some les may need to be edited manually to reconcileupdates being introduced to the playbooks by the Ardana.Newton codebase. Use cautionwhen evaluating these merge conflicts, and ensure that the only changes given priorityover the incoming playbooks are those which are specifically relevant to your cloud'sconfiguration and should be retained into the future. Further information about mergeoperations can be found at Book “Installing with Cloud Lifecycle Manager”, Chapter 10 “Using

Git for Configuration Management”, Section 10.3 “Resolving Git merge conflicts”.

9. Install Ardana.Newton:

(stack)> cd ~/openstack/ardana/ansible(stack)> ansible-playbook -i hosts/localhost config-processor-run.yml(stack)> ansible-playbook -i hosts/localhost ready-deployment.yml

(stack)> cd ~/scratch.scp/ansible/next/hos/ansible/(stack)> ansible-playbook -i hosts/verb_hosts osconfig-iptables-rename.yml(stack)> ansible -i hosts/verb_hosts resources --become \ -a "sed -i 's/helion/ardana/' /etc/iproute2/rt_tables"

Because the OpenStack service versions are comparable between HPE Helion OpenStack5.0.x and Ardana.Newton, it is not necessary to run the entire deploy/upgrade playbookfor Ardana.Newton at this time.

3 Migration from Ardana.Newton to Ardana.Pike1. Create the ardana user throughout all nodes in the cloud:

(stack)> cd ~/scratch/ansible/next/ardana/ansible(stack)> ansible-playbook -i hosts/verb_hosts ardana-user-create.yml

2. Mount the HPE Helion OpenStack 8 installation media on the host (after copying the ISOle into the user's home directory)

(stack)> sudo mkdir -p /media/cloud(stack)> echo /home/stack/HPE-HELION-OPENSTACK-8-x86_64-GM-DVD1.iso \ /media/cloud iso9660 loop 0 0 | sudo tee -a /etc/fstab(stack)> sudo mount -a

3. Add the contents of the ISO as a Zypper repository:

(stack)> sudo zypper ar -G -f file:/media/cloud Cloud

23 Migration from Ardana.Newton to Ardana.Pike HPE Helion OpenStack 8

(stack)> sudo zypper refresh

4. Begin the installation:

(stack)> sudo zypper -n in patterns-cloud-ardana

5. Before continuing the installation, add a password of your choosing to the new ardanauser:

(stack)> sudo passwd ardana

6. Then, exit your current session on the deployer and open a new session as the ardanauser, logging in with the password that you just established. From this point on, the ar-dana user will be used for all operations, and the stack user should be considered dep-recated.To ensure that the stack user does not continue to be used, locking its account with thefollowing command is highly recommended:

ardana > sudo usermod -L stackardana > sudo rm -rf ~stack/.ssh/authorized_keysardana > sudo mv ~stack/scratch ~stack/scratch.newtonardana > sudo mv ~stack/helion ~stack/helion.newton

7. Continue the installation as the ardana user:

ardana > ARDANA_INIT_AUTO=1 /usr/bin/ardana-init

If merge conflicts are reported, some les may need to be edited manually to reconcile thedifference between playbook versions with respect to manual changes previously madefor your cloud. Use caution when evaluating these merge conflicts, and ensure that theonly changes given priority over the incoming playbooks are those which are specificallyrelevant to your cloud's configuration and should be retained into the future. Furtherinformation about merge operations can be found at Book “Installing with Cloud Lifecycle

Manager”, Chapter 10 “Using Git for Configuration Management”, Section 10.3 “Resolving Git merge

conflicts”

8. The installation process creates the directory /srv/www/suse-12.3/x86_64/repos/PTFand adds it as a software repository. To enable a successful migration, you must addrequired Program Temporary Fix (PTF) RPM packages to this repository manually. Thesepackages are provided to you by SUSE via an active L3 support contract.


To add the PTF RPMs manually, the RPMs should be copied into the /srv/www/

suse-12.3/x86_64/repos/PTF directory. For example, if the downloaded RPMs are lo-cated in /tmp/PTF :

ardana > sudo cp /tmp/PTF/*.rpm /srv/www/suse-12.3/x86_64/repos/PTF

9. Apply the PTF packages obtained in the previous step and confirm that they install cor-rectly:

ardana > cd ~/openstack/ardana/ansibleardana > ansible-playbook -i hosts/localhost deployer-init.ymlardana > sudo zypper refresh PTFardana > sudo zypper dup --from PTFardana > sudo zypper in ardana-sesardana > rpm -qa | grep PTF

Ensure that all migration RPMs starting with the prefix "ardana" available via your SUSEsupport contract are present in the resulting list. If any are missing, repeat the aboveprocess or contact technical support before continuing.

10. Apply the updates received via PTF packages to your active cloud:

ardana > /usr/bin/ardana-init

11. Set the default Glance store to hlm-default-store so that your existing image contentremains accessible following the upgrade.

ardana > cd ~/openstack/ardana/ansibleardana > sed -i 's/ardana-default-store/hlm-default-store/' \ roles/GLA-API/defaults/main.yml

Update certificate references to use the new "ardana" name:

ardana > sed -i 's/cert-file: helion/cert-file: ardana/' \ ../../my_cloud/definition/data/network_groups.yml \ ../../my_cloud/definition/data/control_plane.yml

Adjust data model to install cassandra instead of vertica . Manually replace verticawith cassandra in the service-components list, and remove any entry for ops-console-mon-itor:

ardana > vi ../../my_cloud/definition/data/control_plane.yml


In les where a cassandra_db partition exists, manually update its consumer name fromvertica to cassandra and add an identical consumer block to the cassandra_logpartition definition with:

ardana > vi ../../my_cloud/definition/data/disks_*

For example:

- name: cassandra_db size: 20% mount: /var/cassandra/data fstype: ext4 mkfs-opts: -O large_file consumer: name: cassandra

- name: cassandra_log size: 1% mount: /var/cassandra/commitlog fstype: ext4 mkfs-opts: -O large_file consumer: name: cassandra

If your environment includes a SES backend, add configuration les to enable it based onthe Book “Installing with Cloud Lifecycle Manager”, Chapter 23 “Integrations”, Section 23.3 “SUSE

Enterprise Storage Integration” instructions. (However, the site.yml and ardana-recon-figure.yml playbooks should not be used at this time.)Save the updates to the data model:

ardana > git commit -a -m 'Updated data model'

12. Build the ~/scratch directory for the upgrade:

ardana > ansible-playbook -i hosts/localhost config-processor-run.ymlardana > ansible-playbook -i hosts/localhost ready-deployment.yml

13. Prepare the servers for Pike infrastructure:

ardana > cd ~/scratch/ansible/next/ardana/ansibleardana > ansible-playbook -i hosts/verb_hosts FND-AP2-vhosts-transition.ymlardana > sudo rename helion ardana /tmp/helion_*ardana > sudo rename helion ardana /tmp/ardana_*/helion*


14. Set up and verify package repositories for installation:

ardana > sudo ln -s /media/cloud /srv/www/suse-12.3/x86_64/repos/Cloudardana > sudo ln -s /opt/hlm_packager/hlm/sles12/zypper/extras/sles12sp3-ceph-updates \ /srv/www/suse-12.3/x86_64/repos/sles12sp3-ceph-updatesardana > ansible-playbook -i hosts/verb_hosts ardana-replace-legacy-repos.yml

Check that the repos have been properly set up, and that refreshing them succeeds forall SLES cloud nodes:

ardana > ansible resources -m shell -a "zypper ref" --become

15. If your environment includes RHEL Compute Nodes, prepare your deployer for provision-ing OpenStack services on them by following the steps described in Book “Installing with

Cloud Lifecycle Manager”, Chapter 22 “Installing RHEL Compute”, Section 22.4 “Using RHEL as a

Compute Node”.

NoteIf you opt to install an SMT server on your new deployer for mirroring the CentOSrepo, you must rst run the command sudo ln -s /var/lib/mysql/mysql.sock /var/run/mysql/mysql.sock to enable SMT access to the database. (This access isenabled permanently upon completion of the upgrade, but must be manually setup if needed at this time.)

If the CA Management procedure is used to set up a root CA certificate in sup-port of the deployer SMT server, you must immediately follow it by running su-do ln -s /etc/ssl/ca-bundle.pem /etc/ssl/certs/ca-certificates.crtbefore proceeding with the software upgrade.

16. Run the HPE Helion OpenStack 8 upgrade playbook:

ardana > ansible-playbook -i hosts/verb_hosts -vvv ardana-upgrade-from-legacy.yml

Note that the ardana-upgrade-from-legacy.yml playbook is not guaranteed to be idem-potent, and re-running it upon a failure may produce new errors. If this playbook fails,please contact customer support and be prepared to share details of the failure from thecontents of ~/.ansible/ansible.log to determine appropriate next steps.


17. Now that the upgrade has completed, it will be necessary to reboot the control plane nodesin your cloud for changes to take effect. At this time, reboot only the control plane nodes(not compute nodes) in your cloud per the instructions in Book “Operations Guide”, Chapter 13

“System Maintenance”, Section 13.1 “Planned System Maintenance”, Section 13.1.1 “Whole Cloud

Maintenance”, Section 13.1.1.1 “Bringing Down Your Cloud: Services Down Method”.

18. If your cloud includes OpenStack-provisioned load balancers, the following step is neces-sary to update the Amphora load balancer image used by the Octavia service:

ardana > ansible-playbook -i hosts/verb_hosts -vvv service-guest-image \ -e service_package=$(ls /srv/www/suse-12.3/x86_64/repos/PTF/openstack-octavia-amphora-image*.rpm)

19. After updating the Amphora image with the above playbook, the following steps are rec-ommended to confirm that load balancer functionality works as expected. (This will ensurethat your workloads remain available through the upcoming reboots of compute nodes.)

Confirm using neutron lbaas-loadbalancer-list that any existing load balancesare online and reporting with state ACTIVE .

Create a new load balancer with neutron lbaas-loadbalancer-create migra-tion_test OCTAVIA-MGMT-NET-ID . (The ID of OCTAVIA-MGMT-NET is availablefrom the list reported by openstack network list .)

Run neutron lbaas-loadbalancer-list to ensure that the new load balancerappears and becomes ACTIVE , then try using nova delete to remove its associatedVM and ensure that it is recreated by the failover mechanism.

20. You may now proceed to reboot the compute nodes in your cloud for all changes to fullytake effect and re-enable normal behavior. Continue to refer to the steps described in Book

“Operations Guide”, Chapter 13 “System Maintenance”, Section 13.1 “Planned System Maintenance”,

Section 13.1.1 “Whole Cloud Maintenance”, Section 13.1.1.1 “Bringing Down Your Cloud: Services

Down Method”.

21. At this time, the PTF and other temporary packages can be removed:

ardana > cd ~/scratch/ansible/next/ardana/ansibleardana > ansible -i hosts/verb_hosts resources --become -m shell \ -a "zypper rr sles12sp3-ceph-updates || true"ardana > sudo rm /srv/www/suse-12.3/x86_64/repos/sles12sp3-ceph-updatesardana > sudo rm -rf /srv/www/suse-12.3/x86_64/repos/PTF/*.rpmardana > cd ~/openstack/ardana/ansible


ardana > ansible-playbook -i hosts/localhost deployer-init.ymlardana > sudo zypper refresh PTFardana > sudo zypper dup --from Cloud

NoteWhen running sudo zypper dup --from Cloud , warnings will appear aboutpackages being downgraded. This is expected and should be acknowledged as youproceed.

22. Determine how to handle modified policy les which restrict the admin user's privilegesmore tightly than in previous releases:With this upgrade, the policy.json les have been aligned with OpenStack Pike. Whenrunning OpenStack commands an operator may see messages saying Policy doesn'tallow ... such as:

Policy doesn't allow volume_extension:services:index to be performed

In this case the operator can be given permission by adding a role to the user using theopenstack role add command, for example:

ardana > openstack role add --user USER_NAME \ --project PROJECT_NAME cinder_admin

Role assignments should be carefully considered. In an environment which requires sep-aration of duties, each operator should be assigned only the roles they require to performtheir duties and should use their own user account to perform their administrative tasks.For more information see the Book “Security Guide”, Chapter 4 “HPE Helion OpenStack®: Service

Admin Role Segregation in the Identity Service”.

4 Applying Product Updates

1. Before applying updates to your cloud, you should replace the repositories configured oneach node during the migration with links to an SMT server that mirrors the latest versionsof your software. To begin this process, register your deployer node and install the HPEHelion OpenStack extension as described in Book “Installing with Cloud Lifecycle Manager”,

Chapter 3 “Installing the Cloud Lifecycle Manager server”, Section 3.5.2 “Installing the HPE Helion

OpenStack Extension”.

29 Applying Product Updates HPE Helion OpenStack 8

The easiest way to provide the required repositories on the Cloud Lifecycle Manager Serveris to configure it as an SMT server as described in Book “Installing with Cloud Lifecycle

Manager”, Chapter 4 “Installing and Setting Up an SMT Server on the Cloud Lifecycle Manager server

(Optional)”. Alternatives to setting up an SMT server are described in Book “Installing with

Cloud Lifecycle Manager”, Chapter 5 “Software Repository Setup”.

2. Next, use the following commands to replace the original ISO repositories with the latestpackage sources provided by SUSE:

ardana > cd ~/scratch/ansible/next/ardana/ansibleardana > sudo rm /opt/hlm_packager/hlm/sles12/zypper/OS /srv/www/suse-12.3/x86_64/repos/Cloudardana > ansible resources -i hosts/verb_hosts -m shell \ -a "zypper rr Cloud; zypper rr SLES-OS || true" --becomeardana > ansible-playbook -i hosts/verb_hosts _osconfig-setup-repos.yml

3. Ensure that the "Cloud" repository has been replaced by the following repositories on allSLES nodes:

SLES12-SP3-Pool

SLES12-SP3-Updates

HPE-Helion-OpenStack-8-Pool

HPE-Helion-OpenStack-8-Update

4. Finally, apply the latest versions of software packages using the process described in Book

“Operations Guide”, Chapter 13 “System Maintenance”, Section 13.3 “Cloud Lifecycle Manager Main-

tenance Update Procedure”.

30 Applying Product Updates HPE Helion OpenStack 8

Migration Guide - documentation.suse.com · HPE Helion OpenStack 8 dropped support for internally...

Documents

Transcript of Migration Guide - documentation.suse.com · HPE Helion OpenStack 8 dropped support for internally...