Compass first meetup
23
Shuo Yang Principle Cloud Architect Huawei Technologies.
description
Give a high level description of Compass project's background, progress, trying to listen to people's feedback
Transcript of Compass first meetup
Shuo Yang
Principle Cloud Architect
Huawei Technologies.
Compass View – A Short Preview
Why We Started It
Compass Deploy – A Review
1
2
3
Forward Looking – Open, DRY, Collaboration 4
Live CD / GRUB
CPU
CPU Disk
Disk
Switch
NIC
/dev eth0, lo /proc
Linux
Gnome System Monitor
OpenStack
Cinder Neutron Nova
CPU
Server Disk
Storage
Switch
Switch
Compass Deploy
Compass View
Open Cloud OS (OpenStack) Compass Deployment Compass View
Pioneer, a great web app
Great for starters from the OpenStack startup Mirantis
Tools for OpenStack Deployment
Crowbar
Fuel
Attractive concept to OpenStack
folks
A great tool for simple
OpenStack, developer oriented
TripleO (“under the cloud”)
DevStack
A General System to Deploy Distributed Systems
Not Limited to OpenStack
Extensibility as a Primary Design Goal
Automated OpenStack Deployment First, Ceph Deployment Now
Help us Build More Target Systems (e.g., Spark, Mesos…)
Compass Timeline
Think Big, Started Small
OpenStack Ceph
2013 2014
Zero Touch Deployment
Rack and Stack
Blueprint
Robot
Compass Deploy
AI
From 10,000 Feet: What Compass Deploy Solves
Switch 1
Server 1
Host OS
Switch 2 Switch k
Server 2
Host OS
Server n
Host OS
IPMI
SNMP / CLI / SSH…
Plus a lot of scripts: Boilerplate Target system
description
Cobbler / Razor / …
Chef/Ansible/ Docker + Fleet / …
Cumulus / …
KVM
Nova-agent
ESXi LXC
…
Neutron-agent
OVS …
Linux Bridge
Block-storage
Ceph
Gluster
…
ONL / Pica8 / …
OVS Indigo
Ansible Chef
NSX / Floodlight/ RYU
/ ODL / …
Compute Storage Networking
Server
Host OS
Server
Host OS
Server
Host OS
Switch
NOS
Switch
NOS Switch
NOS
Router
NOS
Compass SDNController Compass in Action
1. Network Bootstrap/Installation 2. Network Configuration 3. Server/Storage Bootstrap/Installation
Compass View – A Short Preview
Why We Started It
Compass Deploy – A Review
1
2
3
Forward Looking – Open, DRY, Collaboration 4
Current UI
DB
Cobbler plugin
Razor plugin
Chef plugin Ansible plugin
Package Deployment Hardware Management
Switch Plugin
OS Provisioning
Message bus (RabbitMQ)
Restful API Server
3rd Party ISV UI
Rest Client
Compass -- Open Source
Compass Deploy – Building Blocks
Server Plugin
Machines – physical machine
Switches –network for machines
Adapters –systems to be deployed
Clusters – clusters after the deployment
Hosts – machines in a cluster
…
A Programmable Framework
Mapping Fundamental Concepts during Software Deployment into Restful Resources
Time
(1) Find all available machines: /machines?switch=0e8756dc
(2) Select machines for my system to be deployed on to: {1, 2, 3, 4}
(3) Select an adapter, which describes the result system: /?adapter=OpenStack
(4) Find components for result systems {“controller”, “compute”}
(5) Tell Compass a plan {1 -> “controller”; [2,3,4] -> “compute”} (or specify ‘?auto=true’)
(6) Poll the deployment progress
Focus on Answering the Real Questions
Distributed System
Hardware
HP
OS
Ubuntu
CentOS
Windows Server?
OpenStack Hadoop
Ceph
OCP?
ESXi ?
Huawei
Extensible in Use Cases
OS Provisioning Tools
Cobbler
Razor
Chef Puppet
Ansible
Salt Package Management Tools
Extensible in Tool Chain
Compass View – A Short Preview
Why We Started It
Compass Deploy – A Review
1
2
3
Forward Looking – Open, DRY, Collaboration 4
Search
switch
server server server
agents
Agents:
agents agents
agents
Config Server
Data Path
Probe Proxy/Distiller
Probe Proxy/Distiller:
TSDB (RRD)
History API (perf, utilization)
Event API (alerts, etc)
API Wrappers -a Restful API for Monitoring
Compass View
Storage Engine
Control Path
ElasticSearch
switch
server server server
logstash
Agents:
logstash logstash
logstash
Config Server
Data Path
Redis
Probe Proxy/Distiller:
Restful API
Compass View – Log Pipeline
Storage Engine
Control Path
Logstash
ElasticSearchRestful API
Cassandra
switch
server server server
Agents:
Collectd
Collectd
Config Server
Data Path
KariosDB TCP Server Probe Proxy/Distiller:
Restful API
Compass View – Monitoring Pipeline
Storage Engine
Control Path
Wrap around KariosDB API
Collectd Collectd
Compass View – A Short Preview
Why We Started It
Compass Deploy – A Review
1
2
3
Forward Looking – Open, DRY, Collaboration 4
Chef-Server
Compass
Cobbler-server
Policy Environment Vars (e.g., exclusive; default) OpenStack Cookbook
snippets kickstart files
Roles in the Compass World
App writer
Lib writer
Restful Reqs
100% Python for core engine, AngularJS for frontend Several Dogfood Clusters, Customer clusters up to 200 nodes Compass Wiki Page: https://wiki.openstack.org/wiki/Compass
Toward a Self-Driving Infrastructure – Experience of Running a 200 Physical Server BigData Cloud (Monday, November 3 • 11:40 - 12:20)
Panel: Open Source OpenStack Provisioning Tools: What, Why, and How? (Monday, November 3 • 16:20 - 17:00)
Software Defined Infrastructure: Provisioning and Managing an OpenStack Cloud (Tuesday, November 4 • 11:15 - 11:55)
Open Source under Apache 2.0
Related Talks at OpenStack Summit Paris’14
Contact Us: [email protected]
Project Website: www.syscompass.org
Project LauchPad site: https://launchpad.net/compass
Report bugs; Propose a blueprint; Fix a bug
Please send email to: [email protected]
Collaborate with Us
And Yes, We Are Also Hiring
