Production Plone on OpenStack Cloud

Copyright © 2016 Mirantis, Inc. All rights reserved

software.mirantis.com

Production Ploneon OpenStack Cloud

Learn how to automate scalable multi-node deployments of Plone for OpenStack Cloud and see how easy it is to have this deployment tweaked to your taste


About me

● 4 Years in OpenStack Community● Now Development Manager for Murano● Started as regular developer

Sergey Melikyan

[email protected]@sergmelikyanfb.me/sergmelikyan #running #sightseeing

#hiking #motorcycles #rock


Production OpenStack Plone Deployments

How to deploy Plone on OpenStack

● Introduction to the Cloud and OpenStack● Basics of automation using built-in OpenStack tools● How to automate scalable multi-node deployments● Extend a cloud-ready Plone for your use-cases● Everything else you want to learn today and I can help


Introduction to the Cloud and OpenStack

why your head is not in the clouds yet?


Definition: Cloud

As described by the National Institute of Standards and Technology (NIST):

“Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction”


Essential Characteristics of the Cloud*

● Provisioning of computing capabilities (compute, network, storage) without human interaction

On-demand Self-service

● Services are available over the network and accessed through standard mechanisms

Broad Network Access

● Capabilities can be elastically provisioned and released to scale rapidly

Rapid Elasticity

● Computing resources are pooled to serve multiple consumers using a multi-tenant model

● Customer generally has no knowledge over the exact location of resources

Resource Pooling

● Resource usage can be monitored, controlled, and reported

Measured Service

*According to NIST (National Institute of Standards and Technology)


SPI Model

● Software as a Service (SaaS)● Provides consumers with an application

running on a cloud infrastructure

● Platform as a Service (PaaS)● Provides consumers with the ability to

deploy an application to the cloud utilizing tools and platform supported by the provider

● Infrastructure as a Service (IaaS)● Provides consumers with the ability to

provision fundamental computing resources and deploy arbitrary software

Con

sum

er F

lexi

bil

ity Level o

f Ab

straction

Data CenterHardware, servers, networking

Provision CPU, RAM, VM

INFRASTRUCTURE AS A SERVICE (IAAS)

Remote login to install applications

PLATFORM AS A SERVICE (PAAS)

Browser or thin client access

SOFTWARE AS A SERVICE (SAAS)


Cloud Deployment Models*

*According to NIST (National Institute of Standards and Technology)

● Cloud services are only available to members of a single organization.

● Cloud infrastructure can be hosted by the same organization or a third-party.

Private Cloud

● Cloud services are offered to members of a community or organization.

● Cloud infrastructure is hosted by members of the same organization.

Community Cloud

● Cloud services are offered to the general public.● Cloud infrastructure is hosted by anyone.

Public Cloud

● Composition of two or more distinct cloud infrastructure, such as a Private+Public cloud combination.

Hybrid Cloud


Private Cloud


Public Cloud Providers


Hybrid


Definition: OpenStack

As described by the OpenStack Foundation:

“Aims to produce the ubiquitous Open Source Cloud Computing platform that will meet the needs of public and private clouds regardless of size, by being simple to implement and massively scalable.”


History

Founded by Rackspace and NASA in 2010Mission: to enable any organization regardless of size to create and offer cloud computing services running on standardized hardware


Deployments by Release


Openstack Foundation

Created in 2012 to provide shared resources to help achieve the OpenStack Mission by Protecting, Empowering, and Promoting OpenStack software and the community around it, including users, developers and the entire ecosystem.


Governance


OpenStack Layer


Core Services


Optional Services


Sample Configuration - Web Hosting


Basics of Automation

… using built-in OpenStack tools


Goal

● Install & configure development environment● Provision Plone via Heat● Provision Plone via Murano

Develop automated Plone deployment


Not going to happen...

What I am not going to do today:● Teach you how to develop software on python● Explain what is version control systems, specifically git● Teach you how to use linux or virtualbox

… but what I should do if I don’t know how to … ?


Let me google that for you...


Development Environment

or... tools matter!


● OS: Ubuntu or CentOS● OpenStack: DevStack● Dependencies:

● git, python, pip, tox, virtualenv● OpenStack System Dependencies● IDE or text editor

Tools


Installing DevStack

● Requirements● 4Gb RAM● 2 CPU● 32 Gb Storage

● Recommendation● Use VirtualBox

https://etherpad.openstack.org/p/murano-development-environment


OpenStack System Dependencies

Ubuntu● python-dev● libxml2-dev● libxslt-dev● libffi-dev● libpq-dev● python-openssl● mysql-client

CentOS● python-devel● libxml2-devel● libxslt-devel● libffi-devel● postgresql-devel● pyOpenSSL● mysql


Tools: pip

pip is a package management system used to install and manage software packages written in Python. Many packages can be found in the Python Package Index (PyPI).

$ curl -O https://bootstrap.pypa.io/get-pip.py

$ sudo python -W ignore get-pip.py --force-reinstall

$ rm get-pip.py

http://pypi.python.org


Tools: virtualenv

virtualenv solves a very specific problem: it allows multiple Python projects that have different (and often conflicting) requirements, to coexist on the same computer.

● Installed as dependency for tox● Managed through tox

https://virtualenv.pypa.io/


Tools: tox

tox is a generic virtualenv management and test command line tool you can use for: checking your package installs correctly with different Python versions and interpreters or running your tests in each of the environments, configuring your test tool of choice.

$ sudo pip install tox

https://tox.readthedocs.org/


or short overview of how Plone looks like

Plone Architecture & Deployment


More: http://docs.plone.org/manage/deploying/stack.html

Architecture

Internet Zone / Plone ZODB/Blobs


Requirements

● OS: Ubuntu (or anything else...)● Recommended Flavor

● 2Gb RAM● 40 Gb Storage

● Software Prerequisites:● Python 2.7 (dev), built with support for expat (xml.parsers.expat),

zlib and ssl.● libz (dev), libjpeg (dev), readline (dev), libexpat (dev), libssl or

openssl (dev), libxml2 >= 2.7.8 (dev), libxslt >= 1.1.26 (dev).


Unified Plone UNIX Installer

● The Universal UNIX Installer lets you choose an installation mode:● a standalone mode - single Zope web application server will be

installed. ● or a ZEO cluster mode - ZEO Server and Zope instances will be

installed.

http://docs.plone.org/manage/installing/installation.html


Provision Plone via Heat

or ...heating the cloud!


Heat - OpenStack Orchestration

To orchestrate composite cloud applications using a declarative template format through an OpenStack-native

REST API.

Orchestration for Openstack


Orchestration vs Configuration Management

● Orchestration is a sub-category of automation, concerned with coordination of multiple component:● For example, servers, networks, volumes, etc..● Orchestration is a “higher form” of automation ● Not just simple or lower-level tasks, but multilayer applications

● Configuration Management is automation of server configuration:● Typically a declarative model, based on “fact” discovery of the server● Abstracts out the underlying implementation detail of service deployment

● Both are needed to fully automate cloud application deployment


Heat Capabilities

● Describes the infrastructure for a cloud application – stack (deployment):● OpenStack resources: for example, servers, networks, volumes, etc..● Relationships between resources: for example, this volume is connected to this server● In a text file in the special format – template (blueprint)

● Manages that infrastructure:● Automatically changes the infrastructure when the template is modified and re-applied● Deletes infrastructure when the stack is deleted

● Integrates with software configuration management tools such as Puppet and Chef:● For example: can create VM with puppet server and install puppet clients on VMs● Can pass parameters to cloud-init, etc.


Heat Orchestration Template

● Declarative definition of resources, based on YAML● Stack = deployed template, instance of the template● Structure

● Parameters● Resources● Outputs

Full Specification:http://docs.openstack.org/developer/heat/template_guide/


Example

heat_template_version: 2013-05-23

description: Simple template to deploy a single compute instance

resources:

my_instance:

type: OS::Nova::Server

properties:

key_name: my_key

image: cirros-0.3.0-i386-uec

flavor: m1.small


CloudInit

● An open-source package from Ubuntu that is the industry standard for bootstrapping cloud servers (initialization on first boot)

● Available on various Linux distributions such as Ubuntu Cloud Images and the official Ubuntu images available on EC2.

● Some of the things it configures are:● setting a default locale● setting hostname● resizing boot disk to that specified in boot flavor● adding ssh keys to user's .ssh/authorized_keys so they can log in● setting up ephemeral mount points


CloudInit: Example

resources:

database:

type: OS::Nova::Server

properties:

...

user_data: |

#!/bin/bash -v

yum -y install mariadb mariadb-server

systemctl start mariadb.service


Developing Template

● Uses “Unified Plone UNIX Installer”● Single node deployment● Allows to configure

● Size of the VM (RAM, Storage)● Setup password for Plone● Choose port on which Plone is going to be running


Template

http://q.melikyan.me/plone-via-heat-template


Links

● Project Links:● https://launchpad.net/heat ● https://wiki.openstack.org/wiki/Heat ● http://docs.openstack.org/developer/heat/

● MailingList:● [email protected] with [Heat] in the subject


Provision Plone via Murano

or ...going for a good stuff!


Murano

Introduce an application catalog to OpenStack, enabling application developers and cloud administrators to publish various cloud-ready applications in a browsable categorized

catalog.

Application Catalog for OpenStack


Catalog


Benefits

Developer Tooling Application Catalog Cloud Orchestration

Onboarding Developers Onboarding Workloads Self-Service Deployment

IT Admins End Users

Build better cloud apps faster Deploy without assistance from ITManage what end users can deploy


Capabilities

● Supports both Windows and Linux● Complete application lifecycle management● Integration with configuration management tools

● Ansible, Chef & Puppet● Pluggable app definition languages

● Heat Templates● MuranoPL


Demo

or ...demo is worth a thousand words!


Past

● Windows DataCenter as a Service● Fixed building blocks: AD, IIS, MS SQL, Exchange, Sharepoint● Complicated software composition● Only Windows software provisioning

Bringing Windows to OpenStack


LayersLayers

catalog

app interoperability layer

orchestration

configuration management

glare murano-api + dashboard

murano pl murano-engine

murano-engine

heat heat-translator third-partymurano pl

murano-agent

bash puppet powershellchef


Apps Interoperability

MySQL

host: instance

deploy()createDatabase()createUser()getConnectionS().destroy()

WordPress

webserver: httpddb: mysql

deploy().destroy()

WordPress: deploy() → … webserver.deploy() … db.createDatabase(<name>) db.createUser(<name>, <pass>) db.assignUser(<dname>, <uname>) ...

● Murano PL: imperative OOP DSL● Sandboxed, built on top of Python

● Everything is an object● Dependencies on interface● Decoupling

Apps form an ecosystem where complex environments can be constructed from

independent building blocks


YAQL

YAQL: Yet Another Query Language● Embeddable and extensible query language● Allows performing complex queries against arbitrary objects● Used by Murano, Mistral, Heat and Fuel pip install yaql

$ yaql

yaql> @load my_file.json

yaql> $.customers.groupBy($.sex)

yaql> $.customers.where($.orders.len() >= 1 or name = "John")

Find more about YAQL at https://github.com/openstack/yaql or play at http://yaqluator.com/


Developing Plone App

or … getting your hands dirty!


https://github.com/sergmelikyan/plone-training

Source Code


Hello World: Step 1

Adding manifest.yaml$ mkdir HelloWorld

$ vi HelloWorld/manifest.yaml

FullName: com.yourdomain.HelloWorld

Type: Application

Checkout sources for step #1 - git checkout step1


Hello World: Step 2

Adding HelloWorld.yaml$ mkdir Classes

$ vi Classes/HelloWorld.yaml

Name: com.yourdomain.HelloWorld

Extends: io.murano.Application

Methods:

deploy:

Body:

- $reporter: $this.find('io.murano.Environment').reporter

- $reporter.report($this, "Hello, World!")



Hello World: Step 2

Updating manifest.yaml$ vi HelloWorld/manifest.yaml

...

Classes:

com.yourdomain.HelloWorld: HelloWorld.yaml



HelloWorld: Pack & Upload

Pack & Upload$ zip -r hello_world.zip *$ murano package-import ./hello_world.zip


HelloWorld: Deploy

Deploy$ vi input.js

[{

"op": "add", "path": "/-",

"value": {

"?": {

"name": "Demo",

"type": "com.yourdomain.HelloWorld",

"id": "42"

}

}

}]http://q.melikyan.me/input-json


HelloWorld: Deploy

Deploy$ murano environment-create TestHello

$ murano environment-session-create <environment-id>

$ murano environment-apps-edit --session-id <session-id> <env-id> ./input.json

$ murano environment-deploy --session-id <session-id> <env-id>

$ murano environment-show <env-id>


Hello World: Step 3

Adding User Input$ vi Classes/HelloWorld.yaml

Properties:

username:

Usage: In

Contract: $.string().notNull()

Methods:

deploy:

Body:

- $reporter: $this.find('io.murano.Environment').reporter

- $reporter.report($this, "Hello, " + $this.username + "!")



HelloWorld: re-upload the package

Pack & Reupload$ zip -r hello_world.zip *$ murano package-import ./hello_world.zip


HelloWorld: Deploy again

Deploy with an updated input$ vi input.js

[{

"op": "add", "path": "/-",

"value": {

"?": {

"name": "Demo",

"type": "com.yourdomain.HelloWorld",

"id": "42"

},

"username": "Alice"

}

}]

http://q.melikyan.me/input-json-updated


HelloWorld: Deploy again

Deploy$ murano environment-create TestHello2

$ murano environment-session-create <environment-id>

$ murano environment-apps-edit --session-id <session-id> <env-id> ./input.json

$ murano environment-deploy --session-id <session-id> <env-id>

$ murano environment-show <env-id>


Hello World: Step 4

Adding UI wizard$ mkdir UI$ vi UI/UI.yaml

Application:

?:

type: com.yourdomain.HelloWorld

username: Alice



Hello World: Step 4 (continued)

Application:

?:

type: com.yourdomain.HelloWorld

username: $.step1.username

Forms:

- step1:

fields:

- name: username

type: string

description: Username of the user to say 'hello' to

required: true



Hello World: Use UI Wizard


Hello World: Use UI Wizard (continued)


Hello World: Use UI Wizard (ending)


Plone: Goals

● Provision a virtual machine in OpenStack (VM);● Configure VMs network connectivity and security;● Download a distribution of Plone from Internet to the

virtual machine;● Install the distribution and configure some of its

parameters with user input.


Plone: Preparation

Preparation$ mkdir PloneApp$ vi PloneApp/manifest.yaml

FullName: com.yourdomain.Plone

Name: Plone CMS

Description: Simple Plone Deployment

Type: Application

Author: John Doe



Plone: Preparation (continued)

Add Plone.yaml$ mkdir PloneApp/Classes$ vi PloneApp/Classes/Plone.yaml

Namespaces:

=: com.yourdomain

std: io.murano

apps: io.murano.applications

Name: Plone

Extends: apps:SingleServerApplication



Plone: Preparation (ends)


Updating manifest.yaml$ vi PloneApp/manifest.yaml

...

Classes:

com.yourdomain.Plone: Plone.yaml

Require:

io.murano.applications:


Plone: Properties

Adding properties$ vi PloneApp/Classes/Plone.yaml

Properties:

deploymentPath:

Contract: $.string()

Default: '/opt/plone'

adminPassword:

Contract: $.string().notNull()

listeningPort:

Contract: $.int()

Default: 8080


Plone: Properties

Adding UI.yaml server:

?:

type: io.murano.resources.LinuxMuranoInstance

name: $.instanceConfiguration.hostname

image: $.instanceConfiguration.image

flavor: $.instanceConfiguration.flavor

assignFloatingIp: $.instanceConfiguration.assignFloatingIp



Plone: Configuration Management

● Use “Unified Plone UNIX Installer” ● Use almost same script as we did for Heat template

http://q.melikyan.me/install-plone-sh


Plone: Configuration


Add install-plone.sh$ mkdir PloneApp/Resources

<download and put install-plone.sh in the directory above>


Plone: Deploy


Adding logic$ vi PloneApp/Classes/Plone.yaml

...

Methods:

onInstallServer:

...

Body:

- $file: sys:Resources.string('install-plone.sh').replace({"$1" => $this.deploymentPath, "$2" => $this.adminPassword, "$3" => $this.listeningPort})

- conf:Linux.runCommand($server.agent, $file)


Bonus: Logo & Security Rules

● Please checkout step10 and take a look● Try to analyze what we are doing and why● Explain



Ready to use applications!

DBaaS Containers

PaaS

CI/CD

CMS Web/ App Servers


How to get them?

http://apps.openstack.org


Screencasts

● Openstack Application Catalog. Quick introduction● HA & Autoscaling for Applications● Docker Integration● Support for Heat based application● Composing Murano Application Package

Please, find more screencasts published on wiki: https://wiki.openstack.org/wiki/Murano/Screencasts

http://youtu.be/Auk7P4cZBoQ

http://youtu.be/Auk7P4cZBoQ

http://www.youtube.com/watch?v=OvPpJd0EOFw

http://www.youtube.com/watch?v=OvPpJd0EOFw

http://youtu.be/P51ZUIqS4n4

http://youtu.be/P51ZUIqS4n4

http://www.youtube.com/watch?v=oRD3ihwa9u4

http://www.youtube.com/watch?v=oRD3ihwa9u4

http://youtu.be/_zGo-MtGu78

http://youtu.be/_zGo-MtGu78

https://wiki.openstack.org/wiki/Murano/Screencasts


Links

● Project Links:● https://launchpad.net/murano ● https://wiki.openstack.org/wiki/Murano ● http://docs.openstack.org/developer/murano/

● MailingList:● [email protected] with [Murano] in the subject

Weekly IRC meeting is held on Tuesdays at 17:00 UTC on #openstack-meeting-alt at Freenode.


Survey

ploneconf.sixfeetup.com


Credits

● Alexander Tivelkov & Evgeniy Mashkin● for amazing articles and developing Plone app for Murano

● David Flanders & Philip Bauer● for opportunity being here and present this training


Thank you

for your time

Production Plone on OpenStack Cloud

Software

Transcript of Production Plone on OpenStack Cloud