Understanding OpenStack Deployments

Chris Hoge @hogepodge

!

Interop Engineer OpenStack Foundation

Who is this guy?

!

What is OpenStack?

• Identity - Keystone

• Network - Neutron

• Compute - Nova

• Block Storage - Cinder

• Image Service - Glance

• Dashboard - Horizon

• Object Storage - Swift

• Telemetry - Ceilometer

• Orchestration - Heat

• Database - Trove

• Map Reduce - Sahara

• More and growing…

What does Puppet have to do with this?

Control

A Cartoon View of OpenStack Architecture

Database

Message Queue

Keystone

Neutron API

Nova API

Glance API

Cinder API

Horizon

Network

Neutron AgentsNeutron AgentsNeutron Agents

Nova Scheduler

Glance Registry

Cinder Scheduler

Compute

Nova Compute

Network Agent

Compute

Nova Compute

Network Agent

Compute

Nova Compute

Network Agent

Compute

Nova Compute

Network Agent

Storage

Cinder Storage

Storage

Cinder Storage

Storage

Cinder Storage

Storage

Cinder Storage

NetworkNeutron AgentsNeutron AgentsNeutron Agents

ControlDatabase

Message QueueKeystone

Neutron API

Nova API

Glance API

Cinder API

Horizon

Nova Scheduler

Glance Registry

Cinder Scheduler

ComputeNova

Network

ComputeNova

Network

ComputeNova

Network

ComputeNova

Compute

Network Agent

StorageCinder StorageCinder StorageCinder Storage

Cinder Storage

API Network External Network

Administrative Network

Data Network

“Le Grand Tour” of the puppet-openstack Modules

• 1:1 correspondence with OpenStack projects.

• Community developed in StackForge!

• Builds on Canonical and Red Hat packaging.

• Tracks the major 6-month OpenStack releases.

• Modules available on Puppet Forge.

OpenStack Composition Modules• stackforge/packstack

• theforeman/staypuft

• stackforge/fuel-library

• stackforge/puppet-openstack_builder

• enovance/puppet-openstack-cloud

• puppetlabs/puppetlabs-openstack

!class openstack::profile::base { ! # make sure the parameters are initialized include ::openstack ! # everyone also needs to be on the same clock class { '::ntp': } ! # all nodes need the OpenStack repository class { '::openstack::resources::repo': } ! # database connectors class { '::openstack::resources::connectors': } ! $management_network = $::openstack::config::network_management $management_address = ip_for_network($management_network) $controller_management_address = $::openstack::config::controller_address_management $storage_management_address = $::openstack::config::storage_address_management $management_matches = ($management_address == $controller_management_address) $storage_management_matches = ($management_address == $storage_management_address) ! $api_network = $::openstack::config::network_api $api_address = ip_for_network($api_network) $controller_api_address = $::openstack::config::controller_address_api $storage_api_address = $::openstack::config::storage_address_api ! $api_matches = ($api_address == $controller_api_address) $storage_api_matches = ($api_address == $storage_api_address) ! $is_controller = ($management_matches and $api_matches) $is_storage = ($storage_management_matches and $storage_api_matches) }

NetworkNeutron AgentsNeutron AgentsNeutron Agents

ControlDatabase

Message QueueKeystone

Neutron API

Nova API

Glance API

Cinder API

Horizon

Nova Scheduler

Glance Registry

Cinder Scheduler

ComputeNova

Network

ComputeNova

Network

ComputeNova

Network

ComputeNova

Compute

Network Agent

StorageCinder StorageCinder StorageCinder Storage

Cinder Storage

API Network External Network

Administrative Network

Data Network

# The profile to install rabbitmq !class openstack::profile::rabbitmq { ! $management_address = $::openstack::config::controller_address_management ! class { '::nova::rabbitmq': userid => $::openstack::config::rabbitmq_user, password => $::openstack::config::rabbitmq_password, cluster_disk_nodes => [$management_address], rabbitmq_class => '::rabbitmq', } ! if $::osfamily == 'RedHat' { package { 'erlang': ensure => installed, before => Package['rabbitmq-server'], require => Yumrepo['erlang-solutions'], } } }

# The profile to install an OpenStack specific mysql server !class openstack::profile::mysql { ! class { '::mysql::server': root_password => $::openstack::config::mysql_root_password, restart => true, override_options => { 'mysqld' => { 'bind_address' => $::openstack::config::controller_address_management, 'default-storage-engine' => 'innodb', } } } !! class { '::mysql::bindings': python_enable => true, ruby_enable => true, } ! class { 'mysql::server::account_security': } !}

define openstack::resources::database () { class { "::${title}::db::mysql": user => $title, password => $::openstack::config::mysql_service_password, dbname => $title, allowed_hosts => $::openstack::config::mysql_allowed_hosts, mysql_module => '2.2', require => Anchor['database-service'], } }

NetworkNeutron AgentsNeutron AgentsNeutron Agents

ControlDatabase

Message QueueKeystone

Neutron API

Nova API

Glance API

Cinder API

Horizon

Nova Scheduler

Glance Registry

Cinder Scheduler

ComputeNova

Network

ComputeNova

Network

ComputeNova

Network

ComputeNova

Compute

Network Agent

StorageCinder StorageCinder StorageCinder Storage

Cinder Storage

API Network External Network

Administrative Network

Data Network

# The profile to install the Keystone service class openstack::profile::keystone { openstack::resources::controller { 'keystone': } openstack::resources::database { 'keystone': } openstack::resources::firewall { 'Keystone API': port => '5000', } ! include ::openstack::common::keystone ! class { 'keystone::endpoint': public_address => $::openstack::config::controller_address_api, admin_address => $::openstack::config::controller_address_management, internal_address => $::openstack::config::controller_address_management, region => $::openstack::config::region, } ! $tenants = $::openstack::config::keystone_tenants $users = $::openstack::config::keystone_users create_resources('openstack::resources::tenant', $tenants) create_resources('openstack::resources::user', $users) }

define openstack::resources::user ( $password, $tenant, $email, $admin = false, $enabled = true, ) { keystone_user { "$name": ensure => present, enabled => $enabled, password => $password, tenant => $tenant, email => $email, } ! if $admin == true { keystone_user_role { "$name@$tenant": roles => ['_member_', 'admin'], ensure => present, } } else { keystone_user_role { "$name@$tenant": roles => ['_member_'], ensure => present, } } }

class openstack::common::keystone { if $::openstack::profile::base::is_controller { $admin_bind_host = '0.0.0.0' } else { $admin_bind_host = $::openstack::config::controller_address_management } ! class { '::keystone': admin_token => $::openstack::config::keystone_admin_token, sql_connection => $::openstack::resources::connectors::keystone, verbose => $::openstack::config::verbose, debug => $::openstack::config::debug, enabled => $::openstack::profile::base::is_controller, admin_bind_host => $admin_bind_host, mysql_module => '2.2', } ! class { '::keystone::roles::admin': email => $::openstack::config::keystone_admin_email, password => $::openstack::config::keystone_admin_password, admin_tenant => 'admin', } }

# The profile to set up the Nova controller (several services) class openstack::profile::nova::api { openstack::resources::controller { 'nova': } openstack::resources::database { 'nova': } openstack::resources::firewall { 'Nova API': port => '8774', } openstack::resources::firewall { 'Nova Metadata': port => '8775', } openstack::resources::firewall { 'Nova EC2': port => '8773', } openstack::resources::firewall { 'Nova S3': port => '3333', } openstack::resources::firewall { 'Nova novnc': port => '6080', } ! class { '::nova::keystone::auth': password => $::openstack::config::nova_password, public_address => $::openstack::config::controller_address_api, admin_address => $::openstack::config::controller_address_management, internal_address => $::openstack::config::controller_address_management, region => $::openstack::config::region, cinder => true, } ! include ::openstack::common::nova }

class openstack::common::nova ($is_compute = false) { $is_controller = $::openstack::profile::base::is_controller $management_network = $::openstack::config::network_management $management_address = ip_for_network($management_network) $storage_management_address = $::openstack::config::storage_address_management $controller_management_address = $::openstack::config::controller_address_management ! class { '::nova': sql_connection => $::openstack::resources::connectors::nova, glance_api_servers => "http://${storage_management_address}:9292", memcached_servers => ["${controller_management_address}:11211"], rabbit_hosts => [$controller_management_address], rabbit_userid => $::openstack::config::rabbitmq_user, rabbit_password => $::openstack::config::rabbitmq_password, debug => $::openstack::config::debug, verbose => $::openstack::config::verbose, mysql_module => '2.2', } ! nova_config { 'DEFAULT/default_floating_pool': value => 'public' } ! class { '::nova::api': admin_password => $::openstack::config::nova_password, auth_host => $controller_management_address, enabled => $is_controller, neutron_metadata_proxy_shared_secret => $::openstack::config::neutron_shared_secret, } ! class { '::nova::vncproxy': host => $::openstack::config::controller_address_api, enabled => $is_controller, }

! class { [ 'nova::scheduler', 'nova::objectstore', 'nova::cert', 'nova::consoleauth', 'nova::conductor' ]: enabled => $is_controller, } ! class { '::nova::compute': enabled => $is_compute, vnc_enabled => true, vncserver_proxyclient_address => $management_address, vncproxy_host => $::openstack::config::controller_address_api, } ! class { '::nova::compute::neutron': } ! class { '::nova::network::neutron': neutron_admin_password => $::openstack::config::neutron_password, neutron_region_name => $::openstack::config::region, neutron_admin_auth_url => "http://${controller_management_address}:35357/v2.0", neutron_url => "http://${controller_management_address}:9696", vif_plugging_is_fatal => false, vif_plugging_timeout => '0', }

NetworkNeutron AgentsNeutron AgentsNeutron Agents

ControlDatabase

Message QueueKeystone

Neutron API

Nova API

Glance API

Cinder API

Horizon

Nova Scheduler

Glance Registry

Cinder Scheduler

ComputeNova

Network

ComputeNova

Network

ComputeNova

Network

ComputeNova

Compute

Network Agent

StorageCinder StorageCinder StorageCinder Storage

Cinder Storage

API Network External Network

Administrative Network

Data Network

# The puppet module to set up a Nova Compute node !class openstack::profile::nova::compute { $management_network = $::openstack::config::network_management $management_address = ip_for_network($management_network) ! class { 'openstack::common::nova': is_compute => true, } ! class { '::nova::compute::libvirt': libvirt_type => $::openstack::config::nova_libvirt_type, vncserver_listen => $management_address, } ! file { '/etc/libvirt/qemu.conf': ensure => present, source => 'puppet:///modules/openstack/qemu.conf', mode => '0644', notify => Service['libvirt'], } ! Package['libvirt'] -> File['/etc/libvirt/qemu.conf'] }

class openstack::profile::neutron::agent { include ::openstack::common::neutron include ::openstack::common::ovs } !

class openstack::common::neutron { $controller_management_address = $::openstack::config::controller_address_management $data_network = $::openstack::config::network_data $data_address = ip_for_network($data_network) # neutron auth depends upon a keystone configuration include ::openstack::common::keystone ! class { '::neutron': rabbit_host => $controller_management_address, core_plugin => 'neutron.plugins.ml2.plugin.Ml2Plugin', allow_overlapping_ips => true, rabbit_user => $::openstack::config::rabbitmq_user, rabbit_password => $::openstack::config::rabbitmq_password, debug => $::openstack::config::debug, verbose => $::openstack::config::verbose, service_plugins => ['neutron.services.l3_router.l3_router_plugin.L3RouterPlugin', 'neutron.services.loadbalancer.plugin.LoadBalancerPlugin', 'neutron.services.vpn.plugin.VPNDriverPlugin', 'neutron.services.firewall.fwaas_plugin.FirewallPlugin', 'neutron.services.metering.metering_plugin.MeteringPlugin'], } ! class { '::neutron::keystone::auth': password => $::openstack::config::neutron_password, public_address => $::openstack::config::controller_address_api, admin_address => $::openstack::config::controller_address_management, internal_address => $::openstack::config::controller_address_management, region => $::openstack::config::region, } ! class { '::neutron::server': auth_host => $::openstack::config::controller_address_management, auth_password => $::openstack::config::neutron_password, database_connection => $::openstack::resources::connectors::neutron, enabled => $::openstack::profile::base::is_controller, sync_db => $::openstack::profile::base::is_controller, mysql_module => '2.2', } ! class { '::neutron::server::notifications': nova_url => "http://${controller_management_address}:8774/v2/", nova_admin_auth_url => "http://${controller_management_address}:35357/v2.0/", nova_admin_password => $::openstack::config::nova_password, nova_region_name => $::openstack::config::region, } }

class openstack::common::ovs { $data_network = $::openstack::config::network_data $data_address = ip_for_network($data_network) $enable_tunneling = $::openstack::config::neutron_tunneling # true $tunnel_types = $::openstack::config::neutron_tunnel_types #['gre'] $tenant_network_type = $::openstack::config::neutron_tenant_network_type # ['gre'] $type_drivers = $::openstack::config::neutron_type_drivers # ['gre'] $mechanism_drivers = $::openstack::config::neutron_mechanism_drivers # ['openvswitch'] $tunnel_id_ranges = $::openstack::config::neutron_tunnel_id_ranges # ['1:1000'] ! class { '::neutron::agents::ml2::ovs': enable_tunneling => $enable_tunneling, local_ip => $data_address, enabled => true, tunnel_types => $tunnel_types, } ! class { '::neutron::plugins::ml2': type_drivers => $type_drivers, tenant_network_types => $tenant_network_type, mechanism_drivers => $mechanism_drivers, tunnel_id_ranges => $tunnel_id_ranges } }

How You Can Get Involved• Sign up to be an OpenStack Contributor!

https://wiki.openstack.org/wiki/How_To_Contribute

• Review the Code!

https://review.openstack.org

• Write the Code!

https://wiki.openstack.org/wiki/Gerrit_Workflow

• Hang out with the Amazing Devs!

#puppet-openstack on Freenode

• Share and learn!

puppet-openstack mailing list on Google Groups.

Questions?Thank you!