OPENSTACK

STORAGE ARCHITECTURE

Dan RadezSr. Software Engineer, Red Hatdradez@redhat.comirc: radez

Red Hat's Community OpenStack

openstack.redhat.com

Storage Components

● Database

● Glance

● Nova

● Cinder

● Swift

Component Architecture

Database :: MySQL / MariaDB

Store Options

● Local Storage (no HA)

● Shared Storage

● Replicated Storage

Database :: MySQL / MariaDB

Local Storage

DB Node

Database :: MySQL / MariaDB

Shared Storage

Storage

DB Node* DB Node

iSCSI or NFS

PacemakerActive / Passive Failover

PacemakerActive / Passive Failover

FIP: 10.10.10.123

* MySQL service is running

Database :: MySQL / MariaDB

Shared Storage

Storage

DB Node DB Node*

iSCSI or NFS

X

PacemakerActive / Passive Failover

PacemakerActive / Passive Failover

FIP: 10.10.10.123

* MySQL service is running

Database :: MySQL / MariaDB

Replicated Storage

DB Node* DB Node*

Master / Slave Replication

PacemakerActive / Passive Failover

PacemakerActive / Passive Failover

FIP: 10.10.10.123

Master Slave

* MySQL service is running

Database :: MySQL / MariaDB

Replicated Storage

DB Node DB Node*XMaster / Slave Replication

PacemakerActive / Passive Failover

PacemakerActive / Passive Failover

Master

FIP: 10.10.10.123

* MySQL service is running

Database :: MySQL / MariaDB

Pacemaker Disclaimer

3 node minimumfor quorum

Glance: Image Management

Glance: Image Management

Store Backends

● Local (mounted)

● Swift

● Cinder

● more...

/usr/lib/python2.7/site-packages/glance/store

Glance: Image Management

Use GlusterFS to back Glance# yum install -y glusterfs-fuse

# mkdir /srv/openstack

# vim /etc/fstabgluster1:/openstack /srv/openstack glusterfs defaults,_netdev 0 0

# mount -a# mkdir -p /srv/openstack/glance/images/# chown -R glance:glance /srv/openstack/glance/

Glance: Image Management

Configure OpenStack# vim /etc/glance/glance-api.conf filesystem_store_datadir=/srv/openstack/glance/images/

# vim /etc/glance/glance-cache.conf filesystem_store_datadir=/srv/openstack/glance/images/

# grep "/var/lib/glance" /etc/glance/*glance-api.conf:#scrubber_datadir=/var/lib/glance/scrubberglance-api.conf:#image_cache_dir=/var/lib/glance/image-cache/glance-cache.conf:#image_cache_dir=/var/lib/glance/image-cache/glance-scrubber.conf:#scrubber_datadir=/var/lib/glance/scrubber

# service openstack-glance-api restart# service openstack-glance-registry restart

Nova: Instance Management

Nova: Instance Management

Use GlusterFS to back Nova# yum install -y glusterfs-fuse

# mkdir /srv/openstack# vim /etc/fstabgluster1:/openstack /srv/openstack glusterfs defaults,_netdev 0 0

# mount -a# mkdir -p /srv/openstack/instances# chown -R nova:nova /srv/openstack/instances

# vim /etc/nova/nova.confinstances_path=/srv/openstack/instances

# service openstack-nova-compute restart

Cinder: Block Storage

Cinder: Block Storage

Drivers

● LVM

● NFS

● GlusterFS

● more...

/usr/lib/python2.7/site-packages/cinder/volume/drivers

Cinder: Block Storage

Example: Multiple Storage Types

Dependencies

# yum install -y glusterfs-fuse

Configuration

# vim /etc/cinder/shares.confgluster1:openstackgluster2:openstack

(gluster_host.domain.com:gluster_volume_name)

Cinder: Block Storage

Configuration

#vim /etc/cinder/cinder.confenabled_backends=my_lvm,my_glusterfs

[my_lvm]volume_group = cinder-volumesvolume_driver = cinder.volume.drivers.lvm.LVMISCSIDrivervolume_backend_name = LVM

[my_glusterfs]volume_driver = cinder.volume.drivers.glusterfs.GlusterfsDriverglusterfs_shares_config = /etc/cinder/shares.confglusterfs_sparsed_volumes = falsevolume_backend_name = GLUSTER

Cinder: Block Storage

Restart cinder & verify

# service openstack-cinder-scheduler restart

# service openstack-cinder-volume restart

# mount | grep cindergluster1:openstack on /var/lib/cinder/... type fuse.glusterfsgluster2:openstack on /var/lib/cinder/... type fuse.glusterfs

Cinder: Block Storage

Create cinder volume types

# cinder type-create lvm

# cinder type-key lvm set volume_backend_name=LVM

# cinder type-create glusterfs

# cinder type-key glusterfs set volume_backend_name=GLUSTER

Create cinder volume using type

# cinder create --volume-type glusterfs --display-name test 1

Cinder: Block Storage

Compute configuration (experimental)

# vim /etc/nova/nova.confdebug=Falseqemu_allowed_storage_drivers=gluster

# service openstack-nova-compute restart

Swift: Object Storage

Swift: Object Storage

Storage Backends

● Swift Storage

● GlusterFS

● more...

Review

Resources

● RDO: openstack.redhat.com

● https://access.redhat.com/knowledge/docs/en-US/Red_Hat_OpenStack

● OpenStack.org

● TryStack.org

● Puppetlabs.com

● Djangoproject.com

● http://radez.fedorapeople.org/TokyoMeetup_slides.pdf