NetBioSIG2012 mikesmoot

45
Architecture for Extension Michael Smoot Ph.D. Cytoscape Chief Architect ISMB NetBIO SIG July 13, 2012 Thursday, July 19, 2012

description

Cytoscape is a popular, open source desktop application for visualizing and analyzing biological networks. Cytoscape 2.X consists of a core application that provides a visualization and analysis capabilities along with an API for extending Cytoscape’s functionality through “plugins.” Scientists and other Cytoscape users benefit from the analytical depth provided by the plugins, while plugin authors benefit from the core Cytoscape functionality and the framework for distributing and advertising plugins. This mutually beneficial relationship has resulted in over 150 plugins (http://cytoscape.org/plugins.html) along with dozens of publications about the plugins themselves.Cytoscape 3.0 represents an attempt to refactor Cytoscape to make app writing (plugins will be renamed “apps” in 3.0) simpler while at the same time providing more stability, power, and flexibility to the system as a whole. First and foremost, Cytoscape 3.0 has been modularized with the API cleanly separated from the implementation. This modularity is being facilitated and enforced with OSGi (http://osgi.org), a popular Java modularization framework. OSGi’s micro service architecture allows private implementation code to remain private by registering micro services, which rely only on the public API. This means that any app only has the opportunity to depend on the public API, which will hopefully clarify and simplify what is needed to write an app. We have also begun using the Semantic Versioning standard (http://semver.org) for Cytoscape code to make clear how and when a public API may change. This will all go towards helping the Cytoscape core maintain backwards compatibility, which will greatly increase app stability. All “apps” in 3.0 can be written as OSGi bundles, just like the Cytoscape core modules. This means that apps will now have the opportunity to register their own public API, eliminating the distinction between core and app and resulting in a much more powerful and flexible system. While the architecture of Cytoscape 3.0 relies on OSGi, very little code does. While not without risk, we believe that Cytoscape 3.0 will enable a new generation of Cytoscape apps as well as much greater opportunity for collaboration with different systems. This talk will elaborate on the new Cytoscape architecture including its benefits, challenges, and risks.

Transcript of NetBioSIG2012 mikesmoot

Page 1: NetBioSIG2012 mikesmoot

Architecture for Extension

Michael Smoot Ph.D.Cytoscape Chief Architect

ISMB NetBIO SIG July 13, 2012

Thursday, July 19, 2012

Page 2: NetBioSIG2012 mikesmoot

2

Desktop Application written in Java

Thursday, July 19, 2012

Page 3: NetBioSIG2012 mikesmoot

3Biological Network Visualization + AnalysisThursday, July 19, 2012

Page 4: NetBioSIG2012 mikesmoot

4

Free Software - LGPL

Thursday, July 19, 2012

Page 5: NetBioSIG2012 mikesmoot

5~5000 downloads per monthThursday, July 19, 2012

Page 6: NetBioSIG2012 mikesmoot

6

Thursday, July 19, 2012

Page 7: NetBioSIG2012 mikesmoot

7Photo credit: Samuel L. Livingston

Cytoscape’s most important feature?

Our Plugin App Architecture

Thursday, July 19, 2012

Page 8: NetBioSIG2012 mikesmoot

8

Builds a community of stakeholders

Apps allow for customization

Primary analysis mechanism

Thursday, July 19, 2012

Page 9: NetBioSIG2012 mikesmoot

9

AllegroMCODE APCluster APID2NET BioQualiPlugin BLAST2SimilarityGraph BNMatch CABIN CalculateNodeDegree CentiScaPe ChemViz clusterExplorerPlugin clusterMaker ClusterONE ClusterViz COMA CommFinder CyClus3D CyOog cytoHubba Cytomcl DualLayout dynamicXpr EnhancedSearch EnrichmentMap ExprEssence GraMoFoNe GraphletCounter HiderSlider jActiveModules mcl-new MCODE MINE NeMo NetAtlas NetCirChro netMatch NetworkAnalyzer NetworkEvolution OmicsAnalyzer OmicsViz OrthoNets PanGIA PerturbationAnalyzer PinnacleZ RandomNetworks RDFScsape Reactome FIs RemainingDegreeDistribution ReOrientPlugin ShortestPath Plugin SimTrek structureViz TransClust VennDiagramGenerator VistaClaraPlugin WordCloud BiNoM bioCycPlugin BiogridPlugin BiomartClient BioNetBuilder BisoGenet ConsensusPathDBplugin CoryneRegNetLoader CyThesaurus-ID-Mapping CytoSQL DroID Genoscape GPML-Plugin GraphMLReader IntActWSClient iRefScape KGMLReader MetScape MiMIplugin NCBIClient NCBIEntrezGeneUserInterface Pathintegrator PhosphositePlus Web Service Client Module PICRClient PSICQUICUniversalClient ReConn SessionForWeb SFLDLoader StringWSClient Superpathways-Plugin 3DScape AgilentLiteratureSearch Cytoprophet DisGeNET DomainGraph ExpressionCorrelation GeneMANIA MetaNetter M O N E T B i N G O B u b b l e R o u t e r C l u e G O C o m m o n A t t r i b u t e s DisplayNetworkFromFlatFile FluxViz FunNetViz HyperEdgeEditor PiNGO CyGoose C y t o s c a p e R P C G r o o v y S c r i p t i n g E n g i n e J a v a S c r i p t E n g i n e M i S i n k P y t h o n S c r i p t i n g E n g i n e R u b y S c r i p t i n g E n g i n e S c r i p t E n g i n e M a n a g e r addParentNeighbors AdvancedNetworkMerge batchTool BiLayout commandTool coreCommands CyAnimator edgeLengthPlugin edgeLister EpiTrace FERN FM3 GoogleChartFunctions GroupTool MetaNodePlugin2 MultilevelLayoutPlugin NamedSelection NatureProtocolsWorkflow NeighborHighlight NetLink nodeCharts PhyloTree VennDiagrams

Thursday, July 19, 2012

Page 10: NetBioSIG2012 mikesmoot

10

AllegroMCODE APCluster APID2NET BioQualiPlugin BLAST2SimilarityGraph BNMatch CABIN CalculateNodeDegree CentiScaPe ChemViz clusterExplorerPlugin clusterMaker ClusterONE ClusterViz COMA CommFinder CyClus3D CyOog cytoHubba Cytomcl DualLayout dynamicXpr EnhancedSearch EnrichmentMap ExprEssence GraMoFoNe GraphletCounter HiderSlider jActiveModules mcl-new MCODE MINE NeMo NetAtlas NetCirChro netMatch NetworkAnalyzer NetworkEvolution OmicsAnalyzer OmicsViz OrthoNets PanGIA PerturbationAnalyzer PinnacleZ RandomNetworks RDFScsape Reactome FIs RemainingDegreeDistribution ReOrientPlugin ShortestPath Plugin SimTrek structureViz TransClust VennDiagramGenerator VistaClaraPlugin WordCloud BiNoM bioCycPlugin BiogridPlugin BiomartClient BioNetBuilder BisoGenet ConsensusPathDBplugin CoryneRegNetLoader CyThesaurus-ID-Mapping CytoSQL DroID Genoscape GPML-Plugin GraphMLReader IntActWSClient iRefScape KGMLReader MetScape MiMIplugin NCBIClient NCBIEntrezGeneUserInterface Pathintegrator PhosphositePlus Web Service Client Module PICRClient PSICQUICUniversalClient ReConn SessionForWeb SFLDLoader StringWSClient Superpathways-Plugin 3DScape AgilentLiteratureSearch Cytoprophet DisGeNET DomainGraph ExpressionCorrelation GeneMANIA MetaNetter M O N E T B i N G O B u b b l e R o u t e r C l u e G O C o m m o n A t t r i b u t e s DisplayNetworkFromFlatFile FluxViz FunNetViz HyperEdgeEditor PiNGO CyGoose C y t o s c a p e R P C G r o o v y S c r i p t i n g E n g i n e J a v a S c r i p t E n g i n e M i S i n k P y t h o n S c r i p t i n g E n g i n e R u b y S c r i p t i n g E n g i n e S c r i p t E n g i n e M a n a g e r addParentNeighbors AdvancedNetworkMerge batchTool BiLayout commandTool coreCommands CyAnimator edgeLengthPlugin edgeLister EpiTrace FERN FM3 GoogleChartFunctions GroupTool MetaNodePlugin2 MultilevelLayoutPlugin NamedSelection NatureProtocolsWorkflow NeighborHighlight NetLink nodeCharts PhyloTree VennDiagrams

Thursday, July 19, 2012

Page 11: NetBioSIG2012 mikesmoot

11

AllegroMCODE APCluster APID2NET BioQualiPlugin BLAST2SimilarityGraph BNMatch CABIN CalculateNodeDegree CentiScaPe ChemViz clusterExplorerPlugin clusterMaker ClusterONE ClusterViz COMA CommFinder CyClus3D CyOog cytoHubba Cytomcl DualLayout dynamicXpr EnhancedSearch EnrichmentMap ExprEssence GraMoFoNe GraphletCounter HiderSlider jActiveModules mcl-new MCODE MINE NeMo NetAtlas NetCirChro netMatch NetworkAnalyzer NetworkEvolution OmicsAnalyzer OmicsViz OrthoNets PanGIA PerturbationAnalyzer PinnacleZ RandomNetworks RDFScsape Reactome FIs RemainingDegreeDistribution ReOrientPlugin ShortestPath Plugin SimTrek structureViz TransClust VennDiagramGenerator VistaClaraPlugin WordCloud BiNoM bioCycPlugin BiogridPlugin BiomartClient BioNetBuilder BisoGenet ConsensusPathDBplugin CoryneRegNetLoader CyThesaurus-ID-Mapping CytoSQL DroID Genoscape GPML-Plugin GraphMLReader IntActWSClient iRefScape KGMLReader MetScape MiMIplugin NCBIClient NCBIEntrezGeneUserInterface Pathintegrator PhosphositePlus Web Service Client Module PICRClient PSICQUICUniversalClient ReConn SessionForWeb SFLDLoader StringWSClient Superpathways-Plugin 3DScape AgilentLiteratureSearch Cytoprophet DisGeNET DomainGraph ExpressionCorrelation GeneMANIA MetaNetter M O N E T B i N G O B u b b l e R o u t e r C l u e G O C o m m o n A t t r i b u t e s DisplayNetworkFromFlatFile FluxViz FunNetViz HyperEdgeEditor PiNGO CyGoose C y t o s c a p e R P C G r o o v y S c r i p t i n g E n g i n e J a v a S c r i p t E n g i n e M i S i n k P y t h o n S c r i p t i n g E n g i n e R u b y S c r i p t i n g E n g i n e S c r i p t E n g i n e M a n a g e r addParentNeighbors AdvancedNetworkMerge batchTool BiLayout commandTool coreCommands CyAnimator edgeLengthPlugin edgeLister EpiTrace FERN FM3 GoogleChartFunctions GroupTool MetaNodePlugin2 MultilevelLayoutPlugin NamedSelection NatureProtocolsWorkflow NeighborHighlight NetLink nodeCharts PhyloTree VennDiagrams

Thursday, July 19, 2012

Page 12: NetBioSIG2012 mikesmoot

So whatʼs the problem?

12Photo credit: TaekwonweirdoPhoto credit: Dean247

Thursday, July 19, 2012

Page 13: NetBioSIG2012 mikesmoot

13

Thursday, July 19, 2012

Page 14: NetBioSIG2012 mikesmoot

14Photo credit: TaekwonweirdoPhoto credit: Dean247

Thursday, July 19, 2012

Page 15: NetBioSIG2012 mikesmoot

15

Thursday, July 19, 2012

Page 16: NetBioSIG2012 mikesmoot

16Photo credit: Taekwonweirdo

Thursday, July 19, 2012

Page 17: NetBioSIG2012 mikesmoot

17

Thursday, July 19, 2012

Page 18: NetBioSIG2012 mikesmoot

!!! A tangled mess of inter-dependencies

!!! Every public class is part of the API

!!! Backwards compatibility is impossible

!!! Poor deprecation and versioning strategy

18

Thursday, July 19, 2012

Page 19: NetBioSIG2012 mikesmoot

Is that all?

19

Thursday, July 19, 2012

Page 20: NetBioSIG2012 mikesmoot

20

!!! Apps canʼt use conflicting libraries

!!! Apps canʼt publish their own APIs

Thursday, July 19, 2012

Page 21: NetBioSIG2012 mikesmoot

21

Hurts core developers,app writers, and users!

Thursday, July 19, 2012

Page 22: NetBioSIG2012 mikesmoot

How do we fix this?

22

Thursday, July 19, 2012

Page 23: NetBioSIG2012 mikesmoot

1. OSGi

2. Semantic Versioning

3. Maven

23

Thursday, July 19, 2012

Page 24: NetBioSIG2012 mikesmoot

Cytoscape 3.0

24

Thursday, July 19, 2012

Page 25: NetBioSIG2012 mikesmoot

Cytoscape 3.0 Milestone 5 is available here:

http://cytoscape.org

25

Thursday, July 19, 2012

Page 26: NetBioSIG2012 mikesmoot

26

OSGi− provides

runtime container

− defines module boundaries

− provides micro service architecture

Cytoscape Application Stack

Java Virtual Machine

OSGi Framework

Cytoscape API Bundle A

Cytoscape API Bundle B

Cytoscape API Bundle C

Lib 1

Lib 2

Cytoscape Impl Bundle A

Cytoscape Impl Bundle C

Cytoscape Impl Bundle B

App 1 App 2 App YApp X

Thursday, July 19, 2012

Page 27: NetBioSIG2012 mikesmoot

How does OSGi accomplish this?

27

Thursday, July 19, 2012

Page 28: NetBioSIG2012 mikesmoot

Module = Jar + Metadata

28

Thursday, July 19, 2012

Page 29: NetBioSIG2012 mikesmoot

Defines what can depend on the module.Export-Packages: org.cytocape.model

29

Defines what the module can depend on.Import-Packages: org.cytoscape.events

Module X Module YModule Z

API 1

API 2

API 3

Thursday, July 19, 2012

Page 30: NetBioSIG2012 mikesmoot

Independence

30

Module AModule C

Module B

Library X v 1.1

Module K

Module Z

Module M

Library X v 1.2

Every App gets its own Classloader!

Thursday, July 19, 2012

Page 31: NetBioSIG2012 mikesmoot

μServices

31

xyz IMPL

xyz API

abc IMPL

abc API

Your App

IMPL modules register instances of service interfaces with the Service Registry

Service Registry

Apps depend on classes and service interfaces defined in the API

Apps get on instances of services interfaces from the Service Registry usingONLY the service interfaces!

IMPL modules create instances of service interfaces defined in the API

Cytoscape +

OSGi

Thursday, July 19, 2012

Page 32: NetBioSIG2012 mikesmoot

OSGienables

Modularity

32Photo credit: Maurice Koop

Thursday, July 19, 2012

Page 33: NetBioSIG2012 mikesmoot

OSGi enforces Modularity

33Photo credit: BlackmanVision

Thursday, July 19, 2012

Page 34: NetBioSIG2012 mikesmoot

Semantic Versioning:

34

Version numbers have meaning!

Thursday, July 19, 2012

Page 35: NetBioSIG2012 mikesmoot

VERSION: major.minor.patch

patch - backwards compatible bug fixes

minor - backwards compatible new features

major - backwards INcompatible changes

35

Thursday, July 19, 2012

Page 36: NetBioSIG2012 mikesmoot

As long as we version honestly, your app will always work!

36

(At least until the next major version update)

Because versions have a functional meaning, we can reason about ranges

[3.1,4)

Thursday, July 19, 2012

Page 37: NetBioSIG2012 mikesmoot

37

Maven

Organize

Build

Release

Thursday, July 19, 2012

Page 38: NetBioSIG2012 mikesmoot

38

1. OSGi

2. Semantic Versioning

3. Maven

- To enable/enforce modularity

- To define how and when changes happen

- To organize, build, and release

Thursday, July 19, 2012

Page 39: NetBioSIG2012 mikesmoot

1. Clearly define API

2. Enforce separation of API and implementation

3. Clearly define what can change and when

4. Sensible dependencies

5. Make writing and maintaining plugins easy

39

Thursday, July 19, 2012

Page 40: NetBioSIG2012 mikesmoot

For more information visit:

http://cytoscape.org

40

Thursday, July 19, 2012

Page 41: NetBioSIG2012 mikesmoot

41

NIH GM070743-01

P01 HG005062

P41 RR031228

Thursday, July 19, 2012

Page 42: NetBioSIG2012 mikesmoot

42

Thursday, July 19, 2012

Page 43: NetBioSIG2012 mikesmoot

Thank you!

43

Thursday, July 19, 2012

Page 44: NetBioSIG2012 mikesmoot

Any Questions?

44

Thursday, July 19, 2012

Page 45: NetBioSIG2012 mikesmoot

45

Climber

No ropes!!!Risks?

Photo credit: notorious d.a.v.

Lose momentum in lengthy rewrite?

Is the new API too limiting?

Porting apps more trouble than itʼs worth?

Is OSGi just too complicated?

Thursday, July 19, 2012