Java 7 Modularity:a View from the Gallery

– Neil Bartlett –

A Skills Matter “In The Brain” Talk

A Small Correction

There will probably never be a “Java 7”.

Sun always says JDK7 or OpenJDK: a product, not a specification.

Nevertheless, probably a de facto standard.

Java Timeline

1. That’s TODAY folks!

2. http://java.sun.com/products/archive/eol.policy.html

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

[2]

[1]

GAJSR 59

GAJSR 176

GAJSR 270

Java 1.4

Java 5

Java 6

Java 7?

JDK7 Features

Project Coin

JSR 292 (dynamic language support)

G1 garbage collector

Modularisation

About You

A Java developer, of course.

Building libraries or apps or both.

Probably not building your own JVM or Java compiler (shout if I’m wrong!)

Therefore...

Modularisation must be a good thing...

.. so long as you’re not forced to change anything!

You don’t care how it works inside the JVM and compiler.

Therefore...

This talk is not about OSGi vs Jigsaw for modularising the JVM.

It is about how the existence of multiple modules systems will affect you.

JDK Modularisation

Why?

Why?

JDK 1.1 = 3.5Mb

JDK 6u16 = 65.2Mb

>1800% growth ;-)

Slow start-up

Difficult to deploy with apps

Difficult to evolve APIs

API Evolution

Never let an old API just die?

Deprecation is meaningless.

Version dependencies have no upper bound.

E.g. “this software requires Java 1.4 or higher”.

Dependency Versions

Depending on whole JDK version is no longer feasible.

Would like to depend on, e.g., Swing v2 and Concurrency v1.x.

Or on Swing 1.x but NO higher. Now v2 can change incompatibly.

Dependency Versions

.NET has always done this

Allowed them to introduce generics without the Erasure mess!

Project Jigsaw

Semi-Disclaimer

I am an “OSGi person” (training courses, book, etc).

But I’m paid by Sun (JavaFX in Eclipse).

My principal allegiance is to the Java platform and community.

Overview

Announced in a blog post by Mark Reinhold, Dec 2008.

“focused narrowly upon the goal of modularizing the JDK”

Example

module M1@1.0 provides M2@2.0 { requires M3 @ 3.0; requires private M4 @ 4.0; permits M6;}

Provides

A way for a module to “alias” itself as another module.

E.g. module A requires B.

B is not present but C “provides” B.

Permits

If module A “permits” B then only B may depend on A.

If no explicit permits listed, then all are permitted.

A bit like “friends” in C++.

Requires

Creates a dependency on another module.

All types in other module will be visible.

Re-export

A requires B requires C

B’s dependency on C is re-exported by default

A can see all types in B and C.

A B C

Requires Modifiers

Private: no re-export

Optional: no error if dependency not present (but ClassNotFoundException or NoClassDefFoundError can result).

Local: module must be loaded by same class loader.

Versions

String starting with a digit or [ or (

“The Java language assigns no meaning to the version of a module”

“Vaguely Debian-like version strings, for now” – Jigsaw JavaDocs

Version Ranges

Dependencies can address a range of providers.

The range syntax is not specified/documented yet.

JSR 277 Versions

Five segments: four numeric, one text

major.minor.micro.update-qualifier

JSR 277 Ranges

Closed:

1 = exactly 1.0.0.0 only

JSR 277 Ranges

Open Range:

1+ = anything 1.0.0.0 or above

1.1+ = anything 1.1.0.0 or above

JSR 277 Ranges

Family Range:

1* = anything 1.0.0.0 or above but below 2.0.0.0

1.1* = anything 1.1.0.0 or above but below 1.2.0.0

JSR 277 Ranges

Open Range Within Family:

1.[1.1.1+] = anything 1.1.1.1 or above but below 2.0.0.0

1.1.[1.1+] = anything 1.1.1.1 or above but below 1.2.0.0

OSGi

Bundle = Module

Example

Manifest-Version: 1.0Bundle-ManifestVersion: 2Bundle-Version: 1.0Export-Package: org.foo;version=“1.0.0”Import-Package: org.bar;version=“[1,2)”Bundle-SymbolicName: mybundle

Exports

The fundamental unit of sharing is the package.

Only explicitly exported packages are available to importers.

Imports

List all the packages used by your bundle.

OSGi will “wire” them to the best provider.

Do not care which provider exports each package.

Substitution

OSGi emphasises the ability to substitute implementations.

Framework attempts to minimise the number of time a class is loaded.

Always get from others before getting from yourself!

Requires

We can use whole-module dependencies as an alternative.

Require-Bundle: org.foo

Strongly discouraged! Breaks substitution, creates high degree of fan-out.

Re-export

With Require-Bundle we can re-export dependencies

Tends to create a mess

With Import-Package, re-exporting is simply not needed.

Versions

Four segments: three numeric, one text

major.minor.micro.qualifier

e.g. 3.5.1.beta_20091030

Defined semantics, e.g. new major version = breaking change, etc.

Version Ranges

1.0.0 = anything 1.0.0 or above

[1.0.0,2.0.0) = anything 1.0.0 or above but below 2.0.0

[ and ] mean inclusive

( and ) mean exclusive

Why Jigsaw?

My Initial Reaction

Admittedly not positive...

Just kill

it now

What’s Hard in OSGi?

Focus on packages as unit of sharing makes split packages awkward.

Legacy class loading/visibility assumptions

Broken Class.forName()

The JDK

One, massive, tangled, legacy mess.

API used by everybody.

Can’t just refactor!

Tangled?

Example:

java.lang depends on java.net, java.util, java.io.

Everything depends on java.lang.

Circular dependencies make true separation impossible.

Assumptions

Some internal classes (e.g. under com.sun.*) assume that certain other classes are always visible on every class loader.

Why Jigsaw?

Jigsaw appears designed for this specific legacy modularisation task.

No proscription against split packages.

“requires local” to hack around class loader assumptions.

Why Jigsaw

I now accept Jigsaw may be the most convenient way to modularise the JDK in the short term.

Why Jigsaw is Worrying

Why Care?

As I said, you probably don’t care how modularity works inside the JVM.

But you do care how it affects your libraries and applications.

Why Care?

“...available for developers to use in their own code, and will be fully supported by Sun...” - Mark Reinhold

Sun wants you to use it in your apps!

Therefore...

Some applications will use it.

Forget about technical merits... it’s in the JRE!

I’m not here today to convince you that OSGi is better.

Therefore...

Library authors must cope with demands for their library to work in Jigsaw.

Competition?

Why resist Jigsaw?

Not just fear of competition.

Nor legacy & inertia.

If Jigsaw was better than OSGi, I would gladly switch!

Fragmentation

OSGi is Standard!

Both de facto and de jure.

Firmly established.

Maybe not so much in apps (yet)...

... but look inside your app server, ESB, build platform, IDE, CRM...

OSGi is the King of Infrastructure.

As a Result...

For library authors, compatibility with OSGi is already very important.

Many Apache libraries packaged as bundles (and of course, all Eclipse libs).

Many of Sun’s libraries also!

What about jpkg?

jpkg is a cool tool for installing Java modules using your O/S’s native packaging system.

i.e. RPM (Red Hat), apt-get (Ubuntu).

Makes for nice JavaOne demos.

What about jpkg?

Nothing about jpkg fundamentally requires Jigsaw.

Dalibor Topic (jpkg author) is in favour of jpkg supporting OSGi.

Unfortunately, no story for Windows or Mac OS yet.

JSR 294’s Promise

Overview

“Superpackages” described in a blog post by Gilad Bracha, April 2006.

Focus on Java language support for modules, rather than runtime aspects.

Complemented by JSR 277, the proposed runtime module system.

Overview (cont)JSR 277 now dead, partially replaced by Project Jigsaw.

Superpackages idea dropped.

New focus on module declaration and access modifier.

In theory can benefit any runtime module system.

Accessibility & the Missing Modifier

Accessibility Today

Top level types:

Modifier Accessibility

public Anywhere

(default) Same Package

Accessibility Today

Members (fields, methods):

Modifier Accessibility

public Anywhere

protected Same Package, subtypes

private Same Compilation Unit

(default) Same Package

Accessibility Today

Difficult to create “library internal” utility classes.

Most libraries contain >1 package.

Utilities must be public.

Therefore also usable by clients of the library.

OSGi’s Answer

Restricted list of packages exported from a module.

Only explicitly exported packages may be seen by other modules.

OSGi’s Answer

Not highly granular, only works on whole packages.

May need to move utilities to an “internal” package.

However, encourages strong separation of API from implementation.

OSGi’s Answer

Based on visibility not accessibility.

To compile accurately, we need to fiddle the classpath.

Would still be nice to support true compiler-supported “module private” access.

Compiler/Runtime Fidelity

Ever Seen These...?

AbstractMethodError

ClassCircularityError

ClassFormatError

IllegalAccessError

InstantiationError

NoSuchFieldError

NoSuchMethodError

VerifyError

Mostly Caused By:

Compiler bugs – you have my sympathy!

Partial compilation after refactoring – zero sympathy.

Except...

AbstractMethodError

ClassCircularityError

ClassFormatError

IllegalAccessError

InstantiationError

NoSuchFieldError

NoSuchMethodError

VerifyError

“Package-Private”

What could possibly go wrong??

package org.foo;

public class Bar { // Default access, a.k.a // "package-private" static void doStuff() { System.out.println("Hello"); }}

package org.foo;

public class Baz { public static void main(...) { Bar.doStuff(); }}

Are Bar and Baz in the same package?

It depends!

“Package-Private”package org.foo;

public class Bar { // Default access, a.k.a // "package-private" static void doStuff() { System.out.println("Hello"); }}

package org.foo;

public class Baz { public static void main(...) { Bar.doStuff(); }}

...is really runtime-package-private (JVM spec, §5.4.4)

“determined by the package name and defining class loader” (JVM spec, §5.3)

Accessibility depends on runtime deployment. The compiler doesn’t have a clue.

Compiler gives up and simply allows access. Result: IllegalAccessError!

“Package-Private”

OSGi’s Answer

Packages are the fundamental unit of sharing.

“Split packages” are strongly discouraged.

Therefore types in the same package are always* in the same class loader.

* OSGi sadly does not prevent us from being stupid and splitting packages if that’s what we really, really want to do. See: Eclipse.

JSR 294: The Reality

New Access Modifier

Top level types:

Modifier Accessibility

public Anywhere

module Same Module

(default) Same Package

Members (fields, methods):

Modifier Accessibility

public Anywhere

module Same Module

protected Same Package, subtypes

private Same Compilation Unit

(default) Same Package

New Access Modifier

Samplepackage org.foo;

module class Wibble { module static void doStuff() { System.out.println("Hello"); }}

package org.bar;

public class Wobble { public static void main(...) { Wibble.doStuff(); }}

Module Declaration

Specifies a new named module.

module-info.java

Examples: Jigsaw

module M1 @ 1.0 { requires M2 @ 2.0, M3 @ 3.0; provides M4 @ 4.0, M5 @ 5.0; permits M6; class com.foo.bar;}

Examples: OSGi-like

module M @ 1.2.0.beta for OSGi @ 4.2 { requires N; requires package P @ [1.0,2.0); requires X:5.0 vendor=S; classpath a.jar b.jar c.jar;}

With Annotations...

import com.mycorp.annotations.*;@Foomodule M @ 5.0u7:SPARC for osgi @ 4.2 { @Bar(“wibble”) requires N; @Quux requires package P @ [1.0,2.0); // ...}

Anatomy

module M1 @ 1.0 { requires M2 @ 2.0, M3 @ 3.0; provides M4 @ 4.0, M5 @ 5.0; permits M6; class com.foo.bar;}

New restricted keyword

Anatomy (cont)

module M1 @ 1.0 { requires M2 @ 2.0, M3 @ 3.0; provides M4 @ 4.0, M5 @ 5.0; permits M6; class com.foo.bar;}

Name = word + (optional) version

Anatomy (cont)

module M1 @ 1.0 { requires M2 @ 2.0, M3 @ 3.0; provides M4 @ 4.0, M5 @ 5.0; permits M6; class com.foo.bar;}

Directives

Versions

Version string is opaque – meaning is undefined (determined by the module system).

Also optional – default is undefined (determined by the module system).

Directives

Content is arbitrary (though some syntax rules apply).

Similar to annotations, but not imported.

Semantics are undefined (determined by the module system).

Module Membership

Which classes are members of this module?

Undefined (determined by the module system).

Here Be Dragons!

Compiler Implications

Not a lot is actually defined!

The compiler must understand these directives somehow.

Compiler cannot function without knowledge of the specific module system.

Compiler Implications

We need either:

A Jigsaw version of javac and an OSGi version of javac (plus N other javacs for other module systems).

Compiler plug-ins.

For Developers

The directives for each module system are completely different.

A module that compiles for Jigsaw won’t compile for OSGi and vice versa.

Must target a single module system.

Supporting Many

Can libraries support multiple module systems?

Perhaps... but at a high price

Compile & test everything N times.

Must offer N separate JARs for download.

Supporting Many

Two version schemes for one library?

Choosing a version number for our library will bind us to a specific module system.

A Rant about Standards

Resolving Conflict

How do standards bodies resolve conflicting requirements?

Hit the Beach

“I know! Let’s just keep both ways as options. Then implementers and users can choose for themselves, and we can hit the beach before they run out of loungers!”

Also Known As...

Agreeing to disagree.

Does not help users, it hurts them.

Too often we are forced to deal with both approaches.

Thanks a lot, standards dudes.

Simple Module System

Overview

A proposal to JSR 294 expert group by:

Peter Kriens (OSGi Alliance)

BJ Hargrave (IBM & OSGi Alliance)

Richard Hall (Sun & author of Felix)

i.e. the OSGi posse.

What Is It?

A concrete module system.

Remove the undefined “black holes” of JSR 294.

Provide as much of a module system as needed by most libraries.

~80% subset of OSGi, Jigsaw.

Focus on Users

Libraries should be offered as modules.

Do not make library authors build and ship multiple module types.

Enable “drop-in” deployability to Jigsaw and OSGi runtimes.

Heal Fragmentation

Not another new module system!

Does not address all use-cases.

An easy on-ramp to modularity.

If you need more power, move up to Jigsaw/OSGi.

Concrete

All keywords must mean something.

No arbitrary module-specific keywords.

Extensible

Allow access to extended features by specific module systems.

Extensions must be extra-lingual.

E.g. MANIFEST.MF for OSGi, annotations for Jigsaw.

Simple!

Simple visibility model

Single version number scheme

Module membership

Visibility: OSGi

Exports are limited

Import packages, or whole modules

Optional re-export of modules

Visibility: Jigsaw

Modules export everything

Whole-module imports

Re-export is the default

Module aliases (“provides”)

Module friends (“permits”)

Visibility: SMS

Modules export everything

Whole-module imports

That’s it!

Visibility: SMS

No change for Jigsaw – a subset of its behaviour.

For OSGi, very close to Require-Bundle with all packages exported.

Will require a minor change to the OSGi spec (R4.3 or R5?)

Versions: SMSOSGi’s versioning scheme is admittedly quite strict (but qualifiers can smooth over many differences).

Sun has stated the OSGi scheme does not meet Jigsaw’s requirements.

TELL US THE REQUIREMENTS.

OSGi will update its spec!

Module MembershipJigsaw:

Individual Java files can declare module membership

module com.mycorp;package com.mycorp;

import java.util.*;...

Module Membership

OSGi:

Not a compile-time concept.

Any class in the physical bundle (JAR file) is a member.

Module Membership

SMS:

Membership corresponds to the module artifact, e.g. JAR file.

One class loader per module.

Summary

A modular JDK will be a Good ThingTM.

Summary

But the JDK is “special”

The same technology does not necessarily work well for applications.

Summary

For library authors, OSGi metadata can be added and tested now.

If future compatibility with Jigsaw is a concern, keep it simple and follow SMS’s rules.

Get Involved

Support SMS

Subscribe to the JSR 294 mailing lists

Support OSGi!

Thank You