Cloud Technologies for Microsoft Computational Biology Tools
Building a Microsoft cloud with open technologies
-
Upload
alessandro-pilotti -
Category
Software
-
view
412 -
download
1
Transcript of Building a Microsoft cloud with open technologies
ENGINEERING WORKSHOP
Compute Engineering Workshop
Building a Microsoft cloud with open technologies
Alessandro Pilotti
Cloudbase Solutions
CEO
ENGINEERING WORKSHOP
OCS + OpenStack design goals
▪ Scalability
▪ Full automation
▪ Easy & fast deployment
▪ Integrate with OCS design
▪ Support for multiple compute node options:
▪ Hyper-V, KVM, etc
ENGINEERING WORKSHOP
Open Compute Server v2 and clouds
▪ “Cattle” vs “Pets”
▪
▪ OCS servers focuses mostly on cattle
▪ Limited BMC / IPMI
▪ The server is the unit of failure
ENGINEERING WORKSHOP
OpenStack
▪ OpenStack focuses on “cattle” as well
▪ Perfect fit for OCS
▪ Question becomes:
▪ How to easily automate OpenStack node provisioning and orchestration
ENGINEERING WORKSHOP
OpenStack components
▪ Compute - Nova (Hyper-V, KVM, etc)
▪ Networking - Neutron (Open vSwitch, Hyper-V SDN)
▪ Identity - Keystone (integrated with Active Directory)
▪ Template - Glance
▪ Block Storage - Cinder (SMB3 and iSCSI)
▪ Object Storage - Swift
▪ Telemetry - Ceilometer
▪ Dashboard- Horizon
ENGINEERING WORKSHOP
Hyper-V
▪ Microsoft’s hypervisor
▪ Used in multiple scenarions, including:
▪ Windows, Azure, Xbox one
▪ 100% free SKU available
▪ No limitations
▪ Current version: 2012 R2
▪ Supported guest OSs: Windows, Linux, BSD
ENGINEERING WORKSHOP
Microsoft OCS - Chassis Manager
▪ Shared management host
▪ TPM for SecureBoot
▪ REST API
▪ Power on / off
▪ Open Source:
▪ https://github.com/MSOpenTech/ChassisManager
ENGINEERING WORKSHOP
Bare metal options▪ TripleO / Ironic
▪ Part of OpenStack core (WiP for OCS support)
▪ Ubuntu MAAS + Juju
▪ Good integrated support, great enterprise choice
▪ OCS support included
ENGINEERING WORKSHOP
MAAS (Metal as a Service)▪ Takes care of the bare metal deployment for all nodes
▪ enlisting
▪ commissioning
▪ deployment (via Juju in our case)
▪ MAC addresses are discovered automatically by MAAS
▪ Rules can be applied to determine the usage based on the MAC
▪ Only the nodes BMC initial address and credentials are needed
▪ MAAS creates automatically a dedicated BMC user
ENGINEERING WORKSHOP
Power adapters
▪ MAAS can use a wide range of BMCs to power on/off
▪ IPMI
▪ Intel vPro / AMT
▪ Cisco UCS
▪ HP Moonshot
▪ MSFT OCS - Chassis Manager
▪ etc
ENGINEERING WORKSHOP
Deployment steps1. Deploy MAAS node
▪ E.g. via v-magine, a portable standalone executable
▪ No need for a pre-configured jump box
2. Enlist / Commission bare metal nodes
3. Deploy OpenStack charm
▪ VMs on the MAAS node
▪ all bare metal compute nodes
▪ storage nodes
ENGINEERING WORKSHOP
Juju
▪ OpenStack deployment managed by Juju charm bundle
▪ Charms are the rough equivalent of a Puppet manifest
▪ Main advantages over Puppet, Chef, Salt, etc:
▪ Relationships between charms
▪ Easy integration with different providers: MAAS, OpenStack, etc
▪ Great CLI and GUI
ENGINEERING WORKSHOP
MAAS and Juju Windows support
▪ Windows / Hyper-V porting and support provided by
▪ Cloudbase Solutions
▪ Including Juju charms for:
▪ OpenStack Hyper-V, Active Directory, IIS, SQL Server, SharePoint, Exchange, Lync
ENGINEERING WORKSHOP
Networking with Hyper-V SDN
▪ NVGRE
▪ Hardware offloading supported by Mellanox on OCS
▪ Overlay allows minimal physical network config
▪ VLAN
▪ Most performant option with less that 4096 networks
▪ Still, simple trunk mode switch requirement
▪ Full interoperability in OpenStack with OVS
▪ No need for additional components
ENGINEERING WORKSHOP
Open vSwitch
▪ OVSDB
▪ VXLAN
▪ Hyper-V vswitch forwarding extension
▪ Full interoperability with other hypervisors
▪ Support for:
▪ NSX
▪ OpenDaylight
▪ See: http://www.youtube.com/watch?v=Sdu1Ej6A6C0
▪ Same CLI on Windows and Linux
ENGINEERING WORKSHOP
OpenDaylight + Hyper-V▪ OpenDaylight is an open platform for network programmability to enable SDN and NFV for networks at any side and scale
▪ Integrates well with Open vSwitch (OVS) and OpenStack
▪ Since OVS has been ported to Hyper-V, it is now possible to include Hyper-V in OpenDaylight + OpenStack scenarios
▪ http://www.youtube.com/watch?v=Sdu1Ej6A6C0
ENGINEERING WORKSHOP
Block storage
▪ Hyper-V supports SMB3, iSCSI and FC
▪ iSCSI
▪ Simplest interoperable option
▪ SMB3
▪ Recommended, supported on Hyper-V and KVM
ENGINEERING WORKSHOP
Orchestration
▪ On top of the OpenStack deployment or on bare metal
▪ Juju or Heat
▪ Most common Windows and Linux support
▪ Cloudbase provides:
▪ AD, IIS, SQL Server, SharePoint, Exchange, Lync
ENGINEERING WORKSHOP
How to build Windows cloud images?
▪ https://github.com/cloudbase/windows-openstack-imaging-tools
▪ Need to add all required drivers
▪ Images can be periodically updated adding Windows updates
ENGINEERING WORKSHOP
Validating the deployment
▪ OpenStack is using state of the art CI testing
▪ Microsoft and Cloudbase run the Nova/Neutron/Cinder CI
▪ Every CI run includes almost 2000 compute Tempest tests
▪ After an OpenStack deployment, we can run a Tempest run to validate it
ENGINEERING WORKSHOP
Cloudbase-Init
▪ 100% Python
▪ Windows service
▪ Supports:
▪ OpenStack
▪ EC2
▪ CloudStack
▪ OpenNebula
▪ MAAS
ENGINEERING WORKSHOP
Cloudbase-Init actions (examples)
▪ Create user / set password / add to group(s)
▪ Set host password
▪ Set SSH public keys
▪ Extend volumes
▪ User data (including multi-part)
▪ License activation / KMS
ENGINEERING WORKSHOP
Windows remote management
▪ Passwords are complicated to manage
▪ PowerShell remoting / WinRM
▪ Nova and Cloudbase-Init allow client generated passwords
▪ Error prone
▪ Solution: X509 certificate authentication
ENGINEERING WORKSHOP
Windows SSH equivalent?
▪ WinRM
▪ Command execution over HTTP/HTTPS
▪ PowerShell remoting is based on it
▪ Supports Basic, Kerberos and X509 authentication
▪ Linux example:
▪ wsmancmd.py -H 10.228.163.11 -s -a certificate -c ~/.ssl/winrm_client_cert.pem -k ~/.ssl/winrm_client_cert.key “dir c:\”