USING OPENSTACK TO INTEGRATE NON-OPENSTACK SERVICE · USING OPENSTACK TO INTEGRATE NON-OPENSTACK...

USING OPENSTACK TO INTEGRATE NON-OPENSTACK SERVICEJUNHO YOON, ANDREW LIU, JACK NING

AGENDA• INTRODUCTION• MOTIVATIONS• INTEGRATE AUTHN/AUTHZ• INTEGRATE PLATFORM UI• INTEGRATE PLATFORM COMMUNICATION• CONTINOUS DEPLOYMENT WITH CUSTOMIZATION

Introduction

JUNHO YOONSenior developer of

NAVER

ANDREW LIUSenior developer of

NAVER China

JACK NINGSenior developer of

NAVER China

Introduction

• Established in 1999, South Korea• Handle more than half of internet search market in Korea • Have more than 8000 employees• Some apps have more than 100m users

Introduction

• Have a own IDC and a public cloud service https://www.ncloud.com• However NOT OpenStack based

PASTA - IN-HOUSE PAAS

1000+ projects / 800+ daily user10+ integrated platforms so far

PASTA – Architecture

PASTA-web

keystone

horizoncinder

platforms

ceph

Nova

Shipdock(In-house docker

cluster)

Company SSO

platformsplatforms

Experimental

Today’s Topic

Users

ADENDA• INTRODUCTION• MOTIVATIONS• INTEGRATE AUTHN/AUTHZ• INTEGRATE PLATFORM UI• INTEGRATE PLATFORM COMMUNICATION• CONTINOUS DEPLOYMENT WITH CUSTOMIZATION

• About 40 platforms – It’s impossible even to remember URL

• No single entrance/catalog• No resource utilization• No common user experience• Reinvent wheel

Motivations – Too many platforms

Motivations - authz / authn

• Each platforms had its own authz/authn• Takes too much time for first access• Has different permission set• Requires even different user id/password sometimes

PlatformA PlatformB

Common problems in big company

AWS comes to our sight• What does AWS provide

• Integrated UI/UX - consistency• Organized services catalog• Separated PaaS UI with the main UI• Centralized user management - AWS IAM

• We decide to make our platforms as a PaaS like AWS

Component which enables PaaS

PAAS

INTEGRATED CONSOLE DYNAMIC RESOURCE PROVISIONING

• Consistent UX• Integrated Authz/Authn• Seamless integration b/w

platforms

• Resource Provisioning on demand

• Docker Cluster?https://www.slideshare.net/deview/221-docker-orchestration

• Make new from scratch?• Start from opensouce or

commercial system? • Or OpenStack …

Composable Infrastructure

Decide to adopt openstack

ADENDA• INTRODUCTION• MOTIVATIONS• INTEGRATE AUTHN/AUTHZ• INTEGRATE PLATFORM UI• MAKE PLATFORMS INTEROPERATE• PACKAGE/DEPLOY WITH CUSTOMIZATION

Keystone • Authn/Authz in OpenStack• Feature

• Configurable auth/identity backend• Easy to extend by Adding plugin for Authz/Authn• Abundant API interface

Token Backend

CatalogBackend

AssignmentsBackend

IdentityBackend

OpenStackServices

Keystone API

PolicyBackend Credentials

Backend

KeystoneProblem we are facing:• Need to integrate into our existing SSO• Need to identify not logged-in user as well• Want to avoid to save user’s ID/PW in our DB

HORIZON

KEYSTONE OPENSTACK COMPONENT

1) ID/PWPROJECT 2) issue X-AUTH-TOKEN

3) connect to ENDPOINTwith X-AUTH-TOKEN

4) ask the X-AUTH-TOKEN info(PROJECT ID + ROLE + USER)

extended Keystone v1

KEYSTONE PLATFORMS

PASTA-WEB(IN-HOUSECONSOLE)

OAUTH2PROVIDER

1.1)verifyOAUTH-TOKEN

0)OAUTHAuth

1)ID/OAUTH-TOKENorID/PASSWORD

IN-HOUSELDAP

2)GETIDENTITY

1.2)verifyPASSWORD

LDAPIDENTITYPLUGIN

4)ACCESSWITHX-AUTH-TOKEN

5)verifyX-AUTH-TOKEN

3)ISSUEX-AUTH-TOKEN

AUTHPLUGIN

USING COMPANY’S SSO

USING COMPANY’S LDAP FOR IDENTITY

Auth Plugin

SSO HTTP API/api/Auth/tokenInfo to verify token

default identity auth

PASSWORD AUTH

AUTH PLUGIN

LOGIN

LOGIN FAILED

LOGIN DEFAULTAUTH

SSOAUTH

Success

Success

Fail

Fail

Auth Plugin• Keypoint is…

üTreat SSO token as passwordüTry default auth method first. If failed, use auth using SSO nextüExtends auth handler Keystone.auth.plugins.password.Password

• Keystone Configuration

Extended Keystone v1

KEYSTONE PLATFORMS


OAUTH2PROVIDER


0)OAUTHAuth


IN-HOUSELDAP

2)GETIDENTITY

1.2)verifyPASSWORD

LDAPPLUGIN



3)ISSUEX-AUTH-TOKEN

AUTHPLUGIN

Problem• Do not have right to save OpenStack system

users in LDAP• Deadly slow when retrieving all users.

Extended Keystone v2

KEYSTONE PLATFORMS


OAUTH2PROVIDER


0)OAUTHAuth


2)GETIDENTITY

1.2)verifyPASSWORDHYBRIDIDENTITY

PLUGIN(AUTH+IDENTITY)



3)ISSUEX-AUTH-TOKEN

PASTAAUTHHANDLER

IN-HOUSELDAP SQL

INTRODUCE HYBRID INDENTITY PLUGIN• Save new user in SQL• Read from only SQL when querying all users

Hybrid Backend Plugin

• About the auth part

• Based on keystone-hybrid-backendü Implement LDAP Indentityü extending SQL Indentity

IDENTITY AUTH

LOGIN

LOGIN FAILED

SQLAuth

LOGIN

LDAPAuth

Success

Success

Failed

Failed

Hybrid Backend Plugin• Identity

ü For API like get/update user just like the auth flow

• Why customzied for list large usersü 2000+ LDAP USERü List all user take 10~60s in horizonü No domain concept when adopting legacy platforms

• Configuration

LIST_USERS

SQL + LDAPUSERS

SQL USERS

Filter byName

Yes

No

ADENDA• INTRODUCTION• MOTIVATIONS• INTEGRATE PLATFORM AUTHN/AUTHZ• INTEGRATE PLATFORM UI• MAKE PLATFORMS INTEROPERATE• PACKAGE/DEPLOY WITH CUSTOMIZATION

Previously our platforms….• Have each own web based management console

• No consistent user experience

• Implemented using various tech set• Backend : Spring/Node.js/Golang (No python…)• Framework: Backbone.js/Angular/Vue.js/React/Jquery

Openstack - Horizon

Horizon Nova

NeutronNova UI

Cinder

Manilla Plugin

BlarBlar Plugin

Keystone

Manilla

Neutron UI

Cinder UI

Blar Blar

Openstack - Horizon • Not fit for NAVER

• Is not working very well with large user set• Seems “”little bit”” UGLY for us• Implemented with Python + Django• Need to restart and test whenever some platform’s UI upgraded

• How to evenly distribute the UI development job to each platform’s developer guaranteeing consistency?• Make UI independently developed

without forcing to use specific tech set

Micro Service Architecture

UI /Monolithic

App

Microservice

Logic + Database

Microservice Microservice Microservice Microservice

Microservice

Horizon

Micro Service Architecture - modified

UI Integrator

Microservice+UI

Logic + Database

Microservice+UI

</>

</> </> </> </>

</>

PASTA

Microservice+UI Microservice+UI Microservice+UI

Microservice+UI

Micro Service Architecture - modified

UIintegratorhandlesthispart

Eachplatformhandlesthispart

Spring Cloud – Netflix ZUULHTTP Request

“pre” filters “routing” filters “post” filters

“custom” filters “error” filters

BACKEND SERVER

Realized Runtime Flow

Pasta WEB

PlatformA

PlatformB

PlatformC

2. Accessservice-id.pasta.navercorp.com/platform-id/a.txt

6. https//{{platform-host}}/platform-id/*

OAUTH-PROVIDER

3. OAUTH

KEYSTONE(OPENSTACK)

4. Service Permission Check & Issue X-AUTH-TOKEN

ZUUL

7. User permission check using X-AUTH-TOKEN

X-AUTH-TOKEN

8. Final HTML Rendering

5. Decide where to route based on context path8. Render platform page

1. Prepare routing table

Platform Info Extension• Be able to keep each platform endpoint info

in keystone’s Service catalog and endpoints• Service(Openstack Term) = Platform (PASTA Term)

Use the platform name as a context path

Pick internal interface URL for routing

Platform Info Extension• Need extra room to store extra routing info

• Ex) Platform Icons / Display order …• Need separate DB to store these?• Use description section with JSON

Service Info Extension• Should store the project’s extra info into keystone

• Project(Openstack Term) = Service (PASTA Term)

• https://blueprints.launchpad.net/horizon/+spec/support-extra-prop-for-project-and-user

• OpenStack4J

UI Level Interoperability• Already be able to call the other platform’s REST API

• Because all platform UI share same authn/z in a user session

PLATFORM-A PLATFORM-B PLATFORM-C

$.get(“/platform-a/api/functionA”) $.get(“/platform-b/api/functionB”) $.get(“/platform-c/api/functionC”)

http://sample.pasta-host/kaleido/

Backend Interoperability• Need special way to communicate each other

• ex) Batch / Event Handler which run outside of the user session• ex) Run user’s platform interoperation code when event is triggered

• Introduce Serverless Framework (openwisk)

• Not cover detail here

OPENWISK PlatformA PlatformB

PASTA-WEB

PASTAOpenStack

Pasta web package and delivered in traditional way

How we package and deliver OpenStack?

Packaging/Deployment• Requirement

• Should guarantee 7*24 service • No downtime allowed

• Should be one-click deployment• Adopt existing infrastructure (L4, MySQL) into OpenStack

deployment• Minimize our own customization – to reduce rebase burden

• Kollaü Easy to customize by adding ansible playsü Easy to scale outü Highly configurable

Kolla – minimize customization• Kolla plugin

ü Inspired by Kolla plugin for neutronü Contributed for keystone https://review.openstack.org/#/c/401145/ü Documents: kolla plugin functionality https://docs.openstack.org/kolla/ocata/image-

building.html#plugin-functionalityü Format: [<image>-plugin-<plugin-name>]ü Install the plugin as a python module

kolla-build.conf

Kolla – plugin• Benefits

ü Minimize Customization: only 6 commitü Separated our customization from KOLLA

• Make your own pluginü Refer to plugin functionality of KOLLA image buildingü Add similar template in the dockerfile.j2ü Change the block name based on your imagesü Add plugin source in the kolla-build.conf file

Kolla – integrate into legacy• Integrate into company’s existing Database and LoadBalancer

üDisable DB and LB in Kolla globals.yml :enable_mariadb: "no"enable_haproxy: "no"

üAdd New configurations for integration

Kolla – no downtime• Requires no down time when upgrading keystone• kolla-ansible option: --limit <host>

ü Deploy keystone on limit node

• Upgrading one by oneü Ansible/site.yml: Serial attributeü Loadbalancer: support health check by a specific URL and switch traffic

automaticallyü Customized ansible plays

Start to deploy one keystone node

Remove health check url

LB switch traffic to other nodes

Stop and deploy keystone

Wait for keystone back

Added health check url

Waiting for LB switch traffic back

Q&ATHANKS

USING OPENSTACK TO INTEGRATE NON-OPENSTACK SERVICE · USING OPENSTACK TO INTEGRATE NON-OPENSTACK...

Documents

Transcript of USING OPENSTACK TO INTEGRATE NON-OPENSTACK SERVICE · USING OPENSTACK TO INTEGRATE NON-OPENSTACK...