workspace build - EclipseCon

m2e, an exercise in mixing oil and water

Key differences between Eclipse and Maven and what they mean for m2e

developers and users

Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0

PresentaIon overview

•  m2e goals and approach •  Look under the hood – Project import and configuraIon – Dependency resoluIon – Workspace build

•  Q&A


Prereqs

•  This is an “advanced” session •  Good m2e user experience is expected – m2e extension development experience is a plus –  Solid understanding how to use Maven

•  Maven plugin development experience is a plus •  Understanding of Eclipse plugin development concepts – OSGi framework, bundles and classloading model – Workspace and incremental builders


m2e goals and approach

•  Map Maven project configuraIon to Eclipse workspace

•  Full support for incremental workspace build •  Framework for extension execuIon, not a monolith –  Integrate with “naIve” Eclipse tools, like jdt, wtp, ajdt, adt and so on

– Delegate to Maven plugins where desirable and feasible


m2e core features •  Project registry – m2e maintains registry of all workspace Maven projects and their dependencies

– Highly opImized for speed, not for RAM – Workspace dependency resoluIon – AutomaIcally recalculates dependencies when Maven projects change

– Maven project change event listener •  Build lifecycle mapping and extensions discovery •  Project configurators and build parIcipants extension points


… and much more

•  Full-‐blown JDT integraIon •  Rich pom.xml editor with awesome Dependency Hierarchy view

•  Extensive set of extension points •  m2e extension discovery catalog with quick fixes and project import integraIon

•  Maven launch configuraIons


LOOK UNDER THE HOOD Things that work and not so much


Eclipse vs Maven

•  Eclipse JVM is a long-‐lived process l  JVM resources leaks become more prominent

l  Eclipse workspace builds run aaer each resource l  NoIon of ‘incremental’ build

l  Eclipse workspace and underlying filesystem must be kept in sync – Elaborate resource change delta tracking


PROJECT IMPORT AND CONFIGURATION


plugin goal execuIon





build lifecycle execuIon

Maven project build lifecycle •  Build lifecycle is a sequence of plugin goal execuIons –  Calculated upfront based on pom.xml

–  Executed from start to finish •  Plugin goal execuIon is the most basic unit of work in Maven –  Executable goal, a.k.a “mojo”, and corresponding configuraIon


file system

Maven Project model






Mojo execuIon •  A mojo is a Maven plain Old Java Object –  Opaque to Maven core –  Can access Maven Session, Project Model and filesystem

•  Instances are discarded aaer execuIon

•  There is no mechanism for direct mojo-‐to-‐mojo communicaIon

•  No standard mechanism to track changes between builds


Workspace project configuraIon •  Maven project build lifecycle fully defines what happens

during project commend line build –  No general mechanism to “know” what a Maven plugin goal does

•  m2e provides metadata-‐driven build lifecycle mapping –  Determine if project is compaIble with Eclipse installaIon –  Discovery of addiIonal plugins required to work on the project

•  Each plugin goal execuIon is mapped to <ignore>, <configurator> or <execute> –  <configurator> allows Ight integraIon with other Eclipse tools

•  m2e detects changes in project build lifecycle –  … but workspace configuraIon update must be triggered manually


Lifecycle mapping metadata sources

•  pom.xml file of the project •  parent, grand-‐parent and so on pom.xml files •  [m2e 1.2+] workspace preferences •  installed m2e extensions (in no parIcular order) •  [m2e 1.1+] lifecycle mapping metadata provided by maven plugin

•  default lifecycle mapping metadata shipped with m2e


m2e extension discovery

•  Catalog of available m2e extensions – Uses P2 Eclipse Marketplace code, not necessary a good thing

•  Includes m2e lifecycle mapping metadata to facilitate discovery

•  Separate catalogs for m2e 1.0 and 1.1/1.2 – Due to P2 limitaIons

•  Integrated in Maven project import wizard and mapping errors quick-‐fixes.







build lifecycle execuIon

<configurator> <configurator>

<ignore>

<execute>

<configurator>

<ignore>

m2e project configuraIon and build

m2e build lifecycle mapping


Lifecycle mapping acIons •  Lifecycle mapping affects both workspace project configuraIon and build

•  <ignore> tells m2e to completely ignore the plugin goal execuIon

•  <configurator> tells m2e to delegate to an extension –  Can parIcipate in project configuraIon and/or workspace build

•  <execute> tells m2e to execute the goal – During incremental or full workspace build – During project configuraIon


<execute> on configuraIon •  Goals contribute addiIonal project model elements –  Source roots –  ProperIes

•  Other goals or configurators require the model elements to be present –  No mechanism to configure projects directly

•  Must not generate any resources or honour empty BuildContext

<configurator>

<ignore>

<execute>

<execute>

<ignore>

Maven Project model


DEPENDENCY RESOLUTION


Dependency resoluIon •  Project dependencies are calculated from project pom.xml –  No good support for this in Eclipse –  m2e has two separate code paths for auto-‐build on and off

•  m2e tracks dependencies of all workspace projects •  When projects change, dependencies are recalculated and listeners are noIfied

•  In Maven, dependencies are always resolved to packaged arIfacts

•  In m2e, workspace dependencies are resolved to projects output folders –  Some Maven plugins are not compaIble


Akached arIfacts

•  Maven project can produce mulIple arIfacts, eg., main, test, ejb-‐client, sources, etc – No arIfact contents informaIon in Maven project metadata

– m2e can guess content of main and test jar, but not other arIfacts

•  JDT Java project dependency includes all output folders of the target project –  Impossible to depend on just main or just test aspects of other workspace projects


Dependency scopes

•  Maven disInguishes main and test dependencies – EffecIvely each project has two classpaths

•  JDT Java has single project classpath – Main classes are compiled against test classpath –  JDT team proposed splilng projects in two, which would result in even harder to navigate workspace and likely won’t work


RunIme classpath and source lookup

•  Maven runIme classpath calculaIon rules – No easy way to instrument exisIng launch configuraIons with custom classpath providers

•  m2e hijacks JDT Java ApplicaIon and Junit launch configuraIon –  ILaunchConfiguraIonListener injects m2e classpath providers, not preky but works

•  No easy way to enable for other launch configuraIon types


Dynamic source code lookup •  ApplicaIons oaen load code not listed on iniIal JVM

classpath –  OSGi bundles are loaded by framework –  WARs are loaded by web container, like Jeky –  Maven plugins are loaded by Maven core

•  It is oaen possible to locate sources of arIfacts originaIng from Maven repositories –  Embedded pom.xml –  Match jar SHA1/MD5 to Maven repository index

•  Generic javaagent/jsr45/jdwp soluIon possible, but… –  No way to plug custom ISourceLocator to exisIng launch configuraIon

–  JDT ignores custom jsr45 strata if default ‘java’ is present


WORKSPACE BUILD


Resource change events •  Eclipse workspace tracks state of all resources •  Resource change event for all(!) changes

–  Pre/post create, update, remove, markers, team, phantom… –  Grouping of event delivery is not guaranteed

•  Resource change listener –  Workspace changes “may be disallowed”

•  Incremental workspace builder –  More elaborate, long running resource change listener –  Workspace changes are allowed and other builders are noIfied

•  Both listener and builder are hard to implement correctly


Workspace build mapping

compiler :compile

antlr3 :antlr

resources :resources

resources :testResources

compiler :testCompile

build lifecycle execuIon Maven Build ParIcipant

Maven Build ParIcipant

Maven Build ParIcipant

Java Builder


Workspace build •  Different granularity of Maven plugin goals and Eclipse builders –  Can’t order workspace compile/testCompile properly

•  Single change triggers mulIple builder execuIon – All builders must keep workspace and filesystem in sync

–  Required to propagate changes from Maven builder to Java and other Eclipse builders

•  Resource deltas are not communicated among Maven build parIcipants –  This is a bug in m2e


Problem: endless build

l  Aaer making a change to a file, workspace build runs forever


Endless workspace build (1)

save

Best code change ever!

Workspace

codegen builder

jdt builder



save

Best code change ever!

Workspace

codegen builder

jdt builder



Workspace

codegen builder

jdt builder


Endless workspace build (4 == 1)

Workspace

codegen builder

jdt builder

???


Problem: missing generated resources

•  Resources “disappear” from project output folder aaer git-‐pull or similar mulI-‐resource change


Missing generated resources (1)

git-‐pull

Filesystem

resgen builder

jdt builder Workspace sync



git-‐pull

Filesystem

jdt builder Workspace no sync

change'

resgen builder



git-‐pull

Filesystem

resgen builder

jdt builder Workspace no sync

rebuild

???


m2e compaIble Maven plugins •  Builders MUST ignore unrelated changes •  Builders MUST keep workspace and filesystem in sync •  Most mojos are incompaIble unless explicitly coded to support m2e workspace

•  BuildContext API –  Available since m2e 1.0 –  Provides mojos access to resource delta, workspace refresh and error/warning markers

–  Number of drawbacks and limitaIons •  Build avoidance API –  Significant rework –  Planned but no ETA


Wrapper project configurator

•  Pseudo code if (mojo input changed) {! execute mojo;! refresh mojo output;!}!

•  Must know details of the mojo configuraIon parameters and output –  In many cases no easy way to determine


NESTED PROJECTS


Nested project support (lack of) Maven project nested modules Eclipse flat workspace


Flat workspace •  Hard to navigate workspaces, especially large – Modules are grouped together for a reason, flat workspace works against that reason

•  Same/many resources appear in mulIple places –  Confuses team providers and resources lookup dialogs

•  Switching between git branches –  Import/delete workspace project aaer each switch

•  Cross-‐module resource references •  Modules can have same names –  Special requirement for workspace root projects is parIcularly annoying


Conclusions

•  Maven and Eclipse provide significantly different runIme environments

•  Fundamental Eclipse limitaIons – Lack of nested project support – Lack of classpath scope support

•  Workspace build parIcipaIon is hard to implement correctly – Many mojos will require changes


We welcome quality patches

•  m2e development environment setup instrucIons hkp://wiki.eclipse.org/M2E_Development_Environment

•  m2e-‐dev mailing list hkps://dev.eclipse.org/mailman/lisInfo/m2e-‐dev


Q&A


BACKUP


m2e overview

•  From project homepage “The goal of the m2e project is to provide a first-‐class Apache Maven support in the Eclipse IDE”

•  Tight integraIon with Eclipse workspace and tooling – Full support for incremental workspace build –  IntegraIon with naIve Eclipse tools


m2e history •  m2e started as “m2eclipse” at codehaus in 2005 •  Moved to Sonatype SVN in 2008 and to Github in 2010

•  Proposed as Eclipse project in 2008 but actual move was delayed due to m2e dependency on prerelease version of Maven 3.0 (released on 2010-‐10-‐08)

•  3 releases since m2e moved to Eclipse •  #1 most popular download on Eclipse Marketplace as of 2012-‐09-‐22


workspace build - EclipseCon

Documents

Transcript of workspace build - EclipseCon