Building a right sized, do-anything runtime using OSGi technologies: a case study - E Schnabel

Building a right-sized, do-anything runtime

using OSGi technologiesa case study

(sort of)

Erin Schnabel schnabel@us.ibm.com

@ebullientworks

Some notes on motivationThe full profile of WebSphere application server is awesome in its capabilities

It is also well-known that the full profile is not well-suited for development

We did and do listen… and were presented with a challenge: “Create a light-weight profile of WebSphere that starts in under 2 seconds… [but] Don’t break any eggs” — Ian Robinson

HistoryWebSphere Application Server (the full profile) has been around forever.

Big codebase

Big customer base

Big workloads

… Big inhibitors to massive change

Big codebase

Big customer base

Big workloads

This is not a complaint. This is a problem we are

happy to have.

Big codebase

Big customer base

Big workloads

This is not a complaint. This is a problem we are

happy to have.

But it is still a problem.

Code that has been around forever…

Doesn’t matter how good you are, or how smart you are

If your code lives long enough, and is used enough, it ends up looking like…

No matter how good you are, or how smart you are

If your code lives long enough, and is used enough, it ends up looking like… dragons

No matter how good you are, or how smart you are

If your code lives long enough, and is used enough, it ends up looking like… dragons

Especially code that has roots going back to late ‘90s…

OSGi and WAS: The first pass…

OSGi was included in WAS v6.1, in 2006

Went from lots of arbitrary jars to a few bundles

Achieved some modularity enforced by OSGi

We did not use or expose OSGi services

Compatibility constraints: WAS is the bottom of the stack

Assumptions about resource initialization and availability

Entrenched dependencies between some core elements

</background>

This is the version of the story you won’t have heard before…

What if… If we could start over, what would we want?

Developer-friendly

Simple

Dynamic

Light-weight

Composable / Flexible

Extensible

What if… If we could start over, what would we want?

Developer-friendly

Simple

Dynamic

Light-weight

Composable / Flexible

Extensible

selectable content

clear API/SPI runtime/app isolation

human usable configuration

runtimeonly

some combination of technologies

!app-centered

everything

What if…

grokable config

provisioning

What if…

grokable config

provisioning

runtimeonly

!app-centered

everything

How do we do this?

What if… runtime

!app-centeredeverything

And yet allow this?

no restarts

What if…

And for crying out loud, can we prevent THIS?!

runtime3rd partybundle

systembundle

Application

Building a kernel from scratchOSGi-based for all the reasons

First-class use of OSGi services

Must react to configuration changes

Runtime composition on-the-fly

ConfigurationSettled on XML for configuration format

Ubiquitous

Expressive

BUT, for simplicity:

single file

usable defaults

ConfigurationComposable system requires composable configuration:

Individual components own their config

No centralized repository

No externally defined global config model

ConfigurationComposable system requires composable configuration:

Individual components own their config

No centralized repository

No externally defined global config modelConfiguration Admin and Metatype #FTW!

Configuration Admin We rolled our own (sorry)

Parse and merge user configuration and bundle-provided defaults

Resolve variables

Provide configuration to consumers as required by the spec (mostly)

Metatype Equinox impl + extensions

http://www-01.ibm.com/support/knowledgecenter/was_beta_liberty/com.ibm.websphere.wlp.nd.multiplatform.doc/ae/rwlp_extensions_osgi_metatype.html

Uniform validation of user input

Define configuration and constraints in one place, it gets used everywhere else.

We favor metatype.xml for this reason

Custom namespace for additional types and validators

ibm:type — duration, location, password

pid/reference

unique, final, variable, etc.

pid/reference

human readable:1h30m converted to unit of choice used by developer tools to help

prompt for the right kind of path: file vs. url

pid/reference

type=“String”ibm:type=“password”

The value is a “SerializedProtectedString”,

which is not a String.

Developer tools display encoding options: xor or aes, etc.

pid/reference

This is some crazy stuff.

ibm:type=“pid” ibm:reference=“specific.service.pid”

Allows nested configuration elements

to define service relationships

#awesome

ProvisioningTwo phases of provisioning:

Bootstrap the kernel to get configuration

Add or remove features based on configuration update

Features as in Subsystem features (*.esa files, metadata, etc.)

Adding or removing features installs or uninstalls bundles, which adds or removes configurations, whichtriggers the creation or removal of services!

ProvisioningTwo phases of provisioning:

Bootstrap the kernel to get configuration

Add or remove features based on configuration update

Features as in Subsystem features (*.esa files, metadata, etc.)

Adding or removing features installs or uninstalls bundles, which adds or removes configurations, whichtriggers the creation or removal of services!

Dynamically respond to configuration changes at any time

without requiring a restart. !

#really

Using OSGi Services… But who in their right mind wants to manage OSGi services themselves??

Exactly. NOBODY.

BlueprintDeclarative Services

yes, there are others. We focused on these two.

Declarative ServicesWe chose DS for two main reasons:

Timing: Blueprint and Aries were just getting started

Integration with Configuration Admin and Metatype!

Config injected as one unit

activate/modified/updated methods

Service instance creation based on metatype-declared factory pid

DS target filters can be set via configuration

DS is AWESOME!DS is a central part of the Liberty runtime

CA + M + DS = “magic”We do insane things with config-derived target filters

Our runtime would not be what it is without DS in the middle of it

Service dynamics can hurt!Service dynamics are a huge hurdle for “new” developers

DI and IoC can turn even experienced brains inside out if they aren’t prepared. Thankfully, they do seem to recover.

Utilities created to “help” can have unintended consequences. Especially if cut and paste are involved.

There is definitely a “better way” to do things with DS..

Let DS do it. Really.DS is excellent at managing service dynamics.

DS is excellent at managing non-trivial service dependencies

It is very unlikely that you will be able to do better— just let DS do it. That means:

Don’t register services inside a component

Don’t manage references inside a component

IsolationWe mean this in a good way.

Liberty runtime serves two masters:

Typical Application Server paradigm(apps strictly separated from runtime) — API

Platform extender paradigm(the “app” is the runtime) — SPI

Persistent problem:how to allow apps or extensions to use their own versions of libraries that don't conflict with the runtime!?

Subsystems, Resolver Hooks, and Regions… (oh my!)

Features must explicitly declare API and SPI packages (IBM-* metadata in the feature manifest)

Isolation between API/SPI, apps/extensions/runtime is enforced in a few ways:

Subsystems (the Aries impl) for OSGi Applications (API)

Resolver hooks and/or Eclipse Regions for isolation between runtime, extensions (SPI), and containers (API).

</cleanSlate>Of course, we didn’t really get a clean slate.

Application compatibility had to be preserved. !

But that still gave us a LOT of room…

Dealing with our legacyWe did start over with our kernel

Used the new base to re-group…

Lots of code still common with full profile

Wrap/Shim: New face on old code

Patch: tweak and replace bits where necessary

Thank you!

Questions?

Building a right sized, do-anything runtime using OSGi technologies: a case study - E Schnabel

Technology

Transcript of Building a right sized, do-anything runtime using OSGi technologies: a case study - E Schnabel

OSGi with Apache Felix Karaf - Welcome | · OSGi with Apache Felix Karaf © 2010 ... grep, cat, ... © 2010 – anova r&d bvba 24/63 ... Apache Felix Gogo Shell Runtime ...

OSGi IoT Demo - OSGi Community Event 2014

Making everything better with OSGi - a happy case study in building a really nice runtime - H Cummins

Canon Schnabel

OSGi Tutorial - Knopflerfish OSGi

Distributed OSGi - OSGi DevCon - Eclipsecon 2011

FT-OSGi: Fault-Tolerance extensions for the OSGi … · FT-OSGi: Fault-Tolerance extensions for the OSGi ... 3.2 Building Blocks ... reusable and collaborative components. The OSGi

OSGi service platform indigoo.com OSGi · 2015-01-01 · OSGi service platform indigoo.com 6. OSGi service registry The service registry is the central component of OSGi. Through

Apache Sling – A REST-based Web Application … implementation: OAK (!) 21 21 22 ROA and REST . ... Flexible script search algorithm 44 . 45 OSGi . Runtime Requirements !

GlassFish OSGi - from modular runtime to hybrid applications33degree.org/pdf/AlexisMoussinePouchkineGlassFishOSGi.pdf · GlassFish OSGi - from modular runtime to hybrid applications

Enabling Java 2 Runtime Security with Eclipse Plug-ins ___ Analyzing Security Requirements for OSGi-Enabled Platforms Marco Pistoia, Ted Habeck, Larry.

OSGi and SOA - WSO2 | The Open Source Technology for ... · OSGi and SOA Understanding how OSGi ... puzzle was available as OSGi components: ... com/2008/10/osgi-bundle-start-levels.html

Build federated GraphQL schemas in Java using OSGi and Jahia · OSGi modules with support for GraphQL federation make it possible to evolve a GraphQL schema at runtime. 18. GraphQL

GlassFish OSGi - From modular runtime to hybrid applications

Beethoven Sonata Op.110 Schnabel

OSGi & Other Technologies Peter Kriens, aQute OSGi Fellow.

Will Schnabel, Silverpop/B2B Engage

Configuration and commissioning of Turnkey Production …Nov 09, 2018 · Idea – IoT Gateway Based on Java and OSGi-Framework With OSGi dynamic adding of modules at runtime possible

Cook Up a Runtime with The New OSGi Resolver - Neil Bartlett

OSGi DevCon 09 - OSGi on Scala