Cloud & OSGi - The Dawn of Composite Clouds (Now with demo videos)

Transforming the Way the World Runs Applications

Service Fabric Overview March 2011

Cloud & OSGi - The Dawn of Composite Clouds

info@paremus.comRichard Nicholson

Mythology, marketing...

Cloud Computingaddresses this

The Problem is...

Anne Thomas Manes (Gartner) – SOA Symposium: Berlin, October 2010

✗ Deployment of static software images

✔ Dynamic Assembly composites applications

Modular Systems are Maintainable Systems

Anne Thomas Manes (Gartner) – SOA Symposium: Berlin, October 2010

Introducing the Paremus Service Fabric

(1..m) ‘Systems’ may run upon a single Service Fabric

(1..n) compute resource may contribute to a Service Fabric

Fabric ‘EclipseCon’ = ∑ all ‘EclipseCon’ fibres

An OSGi™ technology based cloud runtime

Off Fabric node

Bootstrap Atlas ManagerSource Repository for Examples

POSH CLIApplication UI

Service Fabric Nodes

Service Fabric InfrastructureCached Examples

Deployed Systems

Demo EnvironmentA ‘blue’ Atlas Agent

Managing characteristics of population

rather than each individual node has a

dramatic effect on decreasing runtime

complexity -

http://adaptevolve.blogspot.com/

2008/01/complexity-part-ii-it-all-

depends-on.html

Atlas & Resource Target StateA blue !"bre#

Service Fabric $ EclipseCon

Atlas agents:host resource property: colour

nonered

fibres: \ infra; quota=[10..10]; filter=(&(Group=Red)(os.name=Darwin), \ default; quota=[100..1000]; filter=(Group=Blue) \

Demo: Bootstrapping a Service Fabric

A video of the demo is available on the next page

System Description Running System

A Model Driven Runtime

System

=System

ElementsScaling Behaviour

(Replication Handlers)

+Resource Contract

= (cost_center=engineering)

= !(os.name=Windows)

= (os.name=linux) & (CPU.speed > 3 Ghz)

= fn(z)

= fm(y)

= fl(x)

A System

A Reference to...an OSGi assembly ora WAR, or EAR or a package name or evena named VM image.

service wires (remote) Service AService BService DService EService F

OSGi Service Registry

Service Propertiesfor E

ExportFabric SystemSystem Element

Topology Manager

Remote Service AdminEssence RMI

Remote Service AdminAVRO

Discovery ProviderSLP

Discovery ProviderDDS

Service E EndpointEssence RMIE

Service Eover Essence RMI

Export Registration

OSGi - Remote Service / Remote Service Admin

Target State Runtime State

Deploy

Starting a System

Target State

"Structure#SLA$

Runtime

Entity

Monitor

Provision Delta

Planned Deltase.g. Configuration

changes

Unplanned Deltase.g. Resource

failures

The Service Fabric responds by creating the corresponding System

Starting a System

Runtime State

Target State

To change a runtime System, change its model in the required manner.

Re-Configure

Updating a System

Target State

"Structure#SLA$

Runtime

Entity

Monitor

Provision Delta

Planned Deltase.g. Configuration

changes

Unplanned Deltase.g. Resource

failures

The Service Fabric responds by modifying the running System

Updating a System

Demo: Application Agility

Each ‘fibre’ dynamically assembles and configures any local infrastructure services in response to the runtime dependencies of each hosted System Element

A self-configuring PaaS Modular & runtime dependencies

triggers policy based runtime assembly

triggers policybased runtime assembly

Robust Systems...

Reddit Went Down: Blame Amazon, the Cloud or Both?

“I am a data storage/data center specialist. It pains me to see the "cloud" illusion come crashing down on a great product like Reddit.”......

“It's no secret among most IT folks that the cloud really isn't cheaper than rolling your own infrastructure for reasons exactly like this.”

http://www.readwriteweb.com/cloud/2011/03/ipad-for-business-round-up-web.php

Financial Services Organization: “How do you stop people rebooting machines....?”

•A ‘Robust’ platform is designed to survive cascading failure.

•A ‘Robust’ platform is adaptive & self-repairing: continually attempting to ‘settle’ into the desired state.

•No central or static points of command & control

Adaptive & Recovery Oriented

What do we mean by Robust?

Whac-A-Mole :)

Whac-A-Mole the Paremus Way

Management & Monitoring

ProvisionerSystem Managers

RegistryService Advertisements

RepositoryOSGi bundles

System DescriptionsNimble Policies

WAR EAR

General artifacts

Principles:• No ‘special’ management nodes• Source so*ware repository may fail• Population changes over time (nodes come and go)• Systems change over time (upgrades / roll-backs)• Attempting to cache everything / everywhere / is doomed to

failure

Approaches:• All infra services: Dynamic Group formation / re-formation• All infra services : Dynamic leadership election• On-fabric Repository Cache: Eventual Consistency across

self-selecting set of participating members

Robust PaaS Services

CDS Behaviour - Form Group & Elect Leader

CDS Group Lead! active members

A System is Loaded

= 1= 5= 1

Artefacts are automatically cached in response

Business Group Repository

Business Division Repository

Artefacts propagated around Group

Data Flow

Lead Fails

New leader elected - Group closes around failure

New members join team

No Silver Bullets!Consistency, Availability, Partition-ability (CAP) trade-offs still apply. Use most appropriate data service for each business application.

• Unstructured data processing - Hadoop

• Key / Value - Voldemort

• Column - Cassandra

• Graph Database - Neo4j

• Relational - Derby, MySQL

• Distribution to nodes - BitTorrent

What about Application Data?

http://blog.nahurst.com/visual-guide-to-nosql-systems

Late Break Demos....

Introducing Zeus

+A video of the demo is

available on the next page

Lab49 is a strategy, design and technology consulting firm. Lab49 deliver strategic consulting and build advanced solutions for the world’s leading investment banks, hedge funds and exchanges.

“Lab49 deliver world-class solutions to some of the most sophisticated firms in financial services today. Our innovative approach stems from adopting user-centric practices enabling clients to respond quickly to today’s challenges, as well as realize new business opportunities in the modern world of trading.

SDP Foreign Exchange (FX) functionality;

‣Request for Quote (RFQ)

‣Request for Stream (RFS)

‣ Execution blotter

‣Research/News

Lab49 SDP leverages;

‣ Paremus OSGi Service Fabric

‣ Akka Open Source Agent-Actor Framework

‣ Lab49 Design and Engineering

‣HTML5/JavaScript

Demo: Single Dealer Platform

For access to demo videoplease contact

info@paremus.com

Industry Standards...

• OSGi framework - Equinox, Knopflerfish or Felix

• Paremus OSGi command shell (Posh): OSGi RFC147

• Paremus Nimble Resolver: OSGi OBR resolver (RFC112)

• Paremus implementation of OSGi Remote Services Administration

• OSGi ConfigAdmin

• Paremus implementation of OSGi Web Archive Bundle (RFC66)

• RTI implementation of OMG DDS: service discovery, configuration, management & monitoring

Industry Standards...

• Cloud / OSGi workshop organised by Peter Kriens - EclipseCon 2011

• Ongoing EEG conference calls throughout 2010 / 2011 chaired by David Bosschaert

• RFP-133 Cloud Computing - Current Status

• To get involved: contact David Bosschaert or Peter Kriens

http://dspace.mit.edu/bitstream/handle/1721.1/60085/BBFRFC66.pdf?sequence=1

Modularity is a Must Have not a Maybe!

Patterns of Modular Architecture (Kirk Knoernschild) - http://www.kirkk.com/modularity/chapters/

Diversity and Complexity - Scott E. Page

Distributed OSGi - Dr Robert DunneTuesday, 10:40, 20 minutes | Ballroom BC

Any Questions?

& drop by the Paremus stand for a chat!

OSGi DevCon BoFTuesday, 7:30 | Napa

Cloud & OSGi - The Dawn of Composite Clouds (Now with demo videos)

Technology

Transcript of Cloud & OSGi - The Dawn of Composite Clouds (Now with demo videos)

Embrace OSGi

OSGi Tutorial - Iowa State University Smart Home …smarthome.cs.iastate.edu/documents/Manuals and Tutorials/OSGi/osgi... · This tutorial introduces you to OSGi programming based

OSGi IoT Demo - OSGi Community Event 2014

OSGi Tutorial

AQute OSGi Bootcamp Workshop Exercise By Peter Kriens CEO aQute OSGi Technology Officer and OSGi Fellow.

OSGI overview

OSGi and SOA - WSO2 | The Open Source Technology for ... · OSGi and SOA Understanding how OSGi ... puzzle was available as OSGi components: ... com/2008/10/osgi-bundle-start-levels.html

osgi specificaiton

OSGi Community Event 2010 - Experiences with OSGi in Industrial Applications

OSGi Community Event 2010 - OSGi ME - An OSGi Profile for Embedded Devices

OSGi and Private Clouds

OSGi Tutorial - Knopflerfish OSGi

OSGi - ssw.jku.at

AQute Inside OSGi By Peter Kriens CEO aQute OSGi Technology Officer and OSGi Fellow.

Distributed OSGi - OSGi DevCon - Eclipsecon 2011

OSGi & Other Technologies Peter Kriens, aQute OSGi Fellow.

FT-OSGi: Fault Tolerant Extensions to the OSGi Service ...

Cloud and OSGi at eBay - OSGi Cloud Workshop March 2012

Security in OSGi applications: Robust OSGi Platforms, secure Bundles

OSGi fundamentals