AWS Storage Tiers for Enterprise Workloads - Best Practices (STG301) | AWS re:Invent 2013
Moving AWS workloads to OpenStack
Transcript of Moving AWS workloads to OpenStack
![Page 1: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/1.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
www.mirantis.com
Moving AWS
Workloads to OpenStack
You're not trapped, honest.
![Page 2: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/2.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Who am I?
● Nicholas Chase
● Long-time programmer
● Author of a bunch of books
● Head of technical content at
Mirantis
● Editor-in-Chief, OpenStack:Now
![Page 3: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/3.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Special thanks to...
Darin Sorrentino
Cloud Solutions Architect
![Page 4: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/4.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
What we're going to talk about
● OpenStack for AWS users -- an overview
● The easiest way to move a workload
● Using orchestration tools
● Giving your pet a new home (or, Doing it the hard way)
![Page 5: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/5.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
OpenStack for AWS users
A quick introduction
![Page 6: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/6.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
OpenStack for AWS users
● An open source cloud platform
● An open source version of AWS -- sort of
● No one-to-one correspondence for everything
● Different API
![Page 7: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/7.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
OpenStack for AWS users
Components
● EC2 = Nova
● EC2 container service = Magnum, Kubernetes (via
Murano)
● S3 = Swift
● RDS = Trove
● Identity and Access Management = Keystone
● CloudWatch = Ceilometer
● CloudFormation = Heat
![Page 8: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/8.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
OpenStack for AWS users
Components (con't)
● SQS = Zaqar
● SWF = Mistral
● DyamoDB = MagnetoDB
● VPC = VPN as a Service (in Neutron)
● AWS management console = Horizon
![Page 9: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/9.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
OpenStack for AWS users
![Page 10: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/10.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
The easiest way to move a
workload
![Page 11: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/11.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
The easiest way to move a workload
Don't move it at all.
![Page 12: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/12.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
The easiest way to move a workload
Aspects of a cloud application architecture
● Horizontally scalable
● Inherently stateless
● Microservices-based
● Fault-tolerant
![Page 13: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/13.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
The easiest way to move a workload
● [[DIAGRAM SHOWING MOVING FROM ONE CLOUD
TO ANOTHER.]]
![Page 14: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/14.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Using orchestration tools
![Page 15: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/15.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Using orchestration tools
Orchestration is...
● The layer of your application that deals with the cloud
itself, as opposed to the actual workload
● Sharable/versionable as code
● Often targeted at hybrid cloud environments
You've got multiple options here.
![Page 16: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/16.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Using orchestration tools
Cloud-specific orchestration tools:
● AWS -> CloudFormation
● OpenStack -> Heat (originally based on CloudFormation
& retains some backwards compatibility with
CloudFormation templates)
![Page 17: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/17.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Using orchestration tools
Hybrid cloud orchestration tools:
● StackStorm
● Cloudify
● Ericsson
● Apcera
● More every day, it seems...
![Page 18: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/18.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Using orchestration tools
StackStorm
● Workflow engine in OpenStack (Mistral)
● Open source (https://github.com/StackStorm)
![Page 19: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/19.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Using orchestration tools
StackStorm
![Page 20: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/20.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Using orchestration tools
Cloudify (by Gigaspaces)
● Workflow engine in OpenStack (Mistral)
● Open source (https://github.com/StackStorm)
![Page 21: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/21.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Using orchestration tools
Cloudify
![Page 22: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/22.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Using orchestration tools
Apcera
![Page 23: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/23.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home
Doing it the hard way
![Page 24: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/24.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Windows
The general procedure
● Install VirtIO drivers in the AWS VM
● Prepare the registry on the VM
● Create a VMDK image
● Convert the VMDK to qcow2
● Upload it to OpenStack
![Page 25: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/25.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Windows
● Open the firewall for ICMP
netsh advfirewall firewall add rule name="All ICMP V4"
protocol=icmpv4:any,any dir=in action=allow
![Page 26: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/26.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Windows
● Enable Remote Desktop access (on by default)
● Allow Remote Desktop access through the firewall
![Page 27: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/27.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Windows
Install VirtIO drivers
● You have Administrator Privileges
● In this case, you can install the VirtIO drivers and make the
registry changes in the Windows VM while it is running on AWS
● You don’t have Administrator Privileges
● If you don’t have administrator rights, you will need to leverage
something like guestfish to inject the drivers and registry
settings into the image after converting it to QCOW2
![Page 28: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/28.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Windows
Obtain the latest VirtIO drivers:
● Download the ISO from
http://alt.fedoraproject.org/pub/alt/virtio-
win/latest/images/bin/
If you have Administrator privileges:
● Install Daemon Tools (or some other emulator) to mount
the ISO
● Make sure to install SPTD
![Page 29: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/29.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Windows
Installation with Administrative Privileges
● Install the drivers according to directions at
https://access.redhat.com/documentation/en-
US/Red_Hat_Enterprise_Linux/7/html/Virtualization_De
ployment_and_Administration_Guide/sect-
KVM_Para_virtualized_virtio_Drivers-
Installing_the_drivers_on_an_installed_Windows_guest
_virtual_machine.html
● Driver type is viostor
![Page 30: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/30.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Windows
Administrative Privileges cont’d
● Import the registry configuration
● Full instructions at
https://docs.mirantis.com/openstack/express/latest/pdf/Applicati
onOnBoardingGuideMirantisOpenStackExpress.pdf
● Add a second Administrators account
● Add user
● Add to Administrator's group
![Page 31: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/31.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Windows
● Attach a new volume to the VM
● Use the AWS console
● Download the VMware vCenter Converter from
https://my.vmware.com/group/vmware/evalcenter?p=co
nverter
![Page 32: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/32.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Windows
Using the converter
● Download the VMware vCenter Converter from
https://my.vmware.com/group/vmware/evalcenter?p=co
nverter
● Create a new image using the "Powered On Machine"
as the source
● Choose VMware Workstation as the product
![Page 33: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/33.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Windows
● SAVE THE IMAGE TO THE NEWLY ATTACHED
VOLUME.
DO NOT USE THE SYSTEM DISK.
![Page 34: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/34.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Windows
Convert the resulting VMDK
● Copy the file to a Linux box
● Install qemu-img
● Convert the VMDK to a qcow2 image:
qemu-img convert -f vmdk -O qcow2 VM.vmdk VM.qcow2
![Page 35: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/35.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Windows
If you did NOT have Administrator privileges to install
VirtIO, you will need to do so now utilizing guestfish:
● Tell guestfish where to find the qemu-kvm library:
export LIBGUESTFS_QEMU=$(rpm -ql qemu-kvm | grep qemu-kvm$)
● Copy the VirtIO files to the linux machine (see the guide
for full details)
![Page 36: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/36.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Windows
VirtIO installation (without Administrator Privileges)
● Create a script to inject the files into the image:
#!/bin/bash guestfish --rw -a $1 -i <<_EOF_ upload
Drivers/Inf/BALLOON.CAT 'win:\windows\inf\BALLOON.CAT'
...
_EOF_
● Run the script:./inject.sh ./vm.qcow2
![Page 37: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/37.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Windows
VirtIO installation (without Administrator Privileges)
● Create a file for registry injection:
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\CriticalDevi
ceDataba se\pci#ven_1af4&dev_1001&subsys_00000000]
...
● Inject the file:virt-win-reg --merge {Disk name}.qcow2 {file name} Example:
virt-win-reg --merge VM.qcow2 virtio.reg
![Page 38: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/38.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Windows
● Upload the image to OpenStack
● Use Horizon or the CLI
● For large images, utilize the CLI to avoid browser timeout
frustrations
● Launch a VM based on that image
![Page 39: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/39.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Linux
The general procedure
● Note the existing packages and startup services/scripts
● Create a new VM
● Install what's missing
![Page 40: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/40.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Linux
● Get the installed package list
rpm -qa --queryformat "%{NAME}\n" > /tmp/packages.txt
● Copy /tmp/packages.txt off of the AWS instance to a
local machine
![Page 41: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/41.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Linux
● Get a list of filesystems and their utilization:
df -h >/tmp/filesystems.txt
● Copy /tmp/filesystems.txt off of the AWS instance to a
local machine
![Page 42: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/42.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Linux
● Get a list of services started at boot time:
chkconfig --list | grep :on | awk '{print $1}'
>/tmp/chkconfig.txt
● Copy /tmp/chkconfig.txt off of the AWS instance to a
local machine
![Page 43: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/43.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Linux
● Backup the startup scripts in case you have any auto-
start services that are not installed via packages, such
as Weblogic
tar -zcvf /tmp/init.tgz /etc/init.d/*
● Copy /tmp/init.tgz off of the AWS instance to a local
machine
![Page 44: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/44.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Linux
● Create a new linux image (we’ll use CentOS as our
example distro to migrate)
● Create new ssh key and change permission:
nova keypair-add centos > centos.pem
chmod 600 centos.pem
● Boot centos instance via command:
nova boot --name centos-6.5 --image centos-6.5_64 --flavor
m1.small --key_name centos.pem
![Page 45: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/45.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Linux
● Get the IP address:
nova list
● Login to instance via command:
ssh -i centos.pem [email protected]
![Page 46: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/46.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Giving your pet a new home -- Linux
● Copy files package.txt and chkconfig.txt from your PC to
the /tmp directory on the new instance in OpenStack.
● Install missing packages and autorun services
sudo -s
yum update -y
while read line; do yum -y install "$line"; done <
/tmp/package.txt
while read line; do chkconfig "$line" on; done <
/tmp/chkconfig.txt
![Page 47: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/47.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Q&A
Download the slides from http://openstacknow.com/aws-to-openstack/
![Page 48: Moving AWS workloads to OpenStack](https://reader031.fdocuments.in/reader031/viewer/2022032218/55ac23df1a28ab4d298b46d6/html5/thumbnails/48.jpg)
Copyright © 2015 Mirantis, Inc. All rights reserved
Thank you
for your time