Merijn de Jonge IPA Spring Days 2005 April 1, 2005 Build-level Components.

Merijn de JongeIPASpring Days 2005April 1, 2005

Build-level Components

IPA Spring Days Merijn de Jonge 2Build-level ComponentsApril 1, 2005

Overview

•Introduction/motivation

•Build-level components

•Build-level composition

•Decoupling into build-level components

•Concluding remarks


Overview

•Introduction/motivation

•Build-level componentsBuild-level components

•Build-level compositionBuild-level composition

•Decoupling into build-level componentsDecoupling into build-level components

•Concluding remarksConcluding remarks


Mozilla


Mozilla Directory Structure

•One very large directory tree•26,000 files•2,500 subdirectories•240 file types

•One very large directory tree•26,000 files•2,500 subdirectories•240 file types


Mozilla Directory References

•1,521 directory references•Avg. fan-in/fan-out 12•Max fan-in=100•Max fan-out=42•Median fan-in=5•Median fan-out=12

•1,521 directory references•Avg. fan-in/fan-out 12•Max fan-in=100•Max fan-out=42•Median fan-in=5•Median fan-out=12


Build-level

•File (atomic entity)– Source– Documentation

•Directory tree (container) = source tree

•Build process– To build/install software– Driven by make/ANT/…

•Configuration process– To control build process– Driven by configure/configuration files/…

•Build-level interfaces– Build interface– Configuration interface


Mozilla Build Level

•1,350 Makefiles•40,000 LOC build instructions•16,000 LOC configuration•Cyclic dependencies•Two-phase build process•Centralized build/configuration knowledge•Code duplication•Component implementations scattered

•1,350 Makefiles•40,000 LOC build instructions•16,000 LOC configuration•Cyclic dependencies•Two-phase build process•Centralized build/configuration knowledge•Code duplication•Component implementations scattered


Wishes

•Independent deployment (of build-level artifacts)–No circular dependencies–Individual build processes–Individual configuration processes

•Automated composition–Standardized build interface–Standardized configuration interface–Standardized binding mechanism for build-level variability and dependencies


Idea

Apply CBSE principles to the build level

•Make build-level artefacts available for reuse by decomposing software systems into build-level components

•Make build-level component usable by assembling them into software systems using automated composition techniques


Overview

•Introduction/motivationIntroduction/motivation

•Build-level components





Component Definition (Szyperski)

A software component is

•A unit of composition …

•with contractually specified interfaces …

•and explicit context dependencies only.

•It is subject to independent deployment …

•and to third-party composition.


Standardized Build Interface

•Different build systems exist, e.g., – make, ANT, shell scripts, IDE

•Different software systems require different build actions, e.g.,

– make– make bootstrap, make, make install

make cleanmake allmake checkmake (un)installmake dist

make cleanmake allmake checkmake (un)installmake dist

•Standardized build interface to enable that software construction can be controlled via a standard set and sequence of build actions:


Standardized Configuration Interface

•Different mechanisms exist to control software construction, e.g.,–configuration files, configuration tools, Makefile editing

•Standardized configuration interface to enable uniform compile-time configuration:

configure --helpconfigure --prefix=/usrconfigure --with-aterm=/usr/libconfigure --with-optimization=trueconfigure --with-debug=false

configure --helpconfigure --prefix=/usrconfigure --with-aterm=/usr/libconfigure --with-optimization=trueconfigure --with-debug=false


Explicit Context Dependencies

•Dependencies on build-level components are declared in configuration interfaces

> configure --help … --with-aterm=DIR use ATerm Library at DIR --with-sglr=DIR use SGLR Parser at DIR …

> configure --help … --with-aterm=DIR use ATerm Library at DIR --with-sglr=DIR use SGLR Parser at DIR …


Independent Deployment

•Build-level components are deployed as packages

•A package is a versioned release of a build-level component, e.g.,

firefox-1.0.tar.gz

•Packages are published on web/ftp sites


Third-party Composition

•A configuration interface enables late-binding of dependencies

•Compositions are thus not predefined

•and can be defined by a third party

> configure --with-aterm=/usr/local/aterm --with-sglr=/usr/local/sglr --with-…=

> configure --with-aterm=/usr/local/aterm --with-sglr=/usr/local/sglr --with-…=


Overview



•Build-level composition




Architectural Description Language

•Package Definition Language (PDL)–Domain-specific language designed specially for build-level component composition

–Used for initial implementations–No explicit provides interfaces

•Koala–Re-use of existing composition language–Implementation is work in progress–Multiple provides interfaces–No version resolution


Koala Concepts for Build-level Composition

•Interface definitions–To group build-level artefacts into types

•Explicit provides interfaces–To declare the type(s) of a component

•Diversity interfaces–To declare build-level variation points

•Module as atomic entity–To declare the implementation of a component


Koala Definition for Build-level Componentcomponent Sglr{ provides ISglr p; requires IATerm aterm; IToolbusLib libtb; IErrorSupport error_support; IPtSupport pt_support; contains module m url="http://www.cwi.nl/sglr-3.13.tar.gz"; connects p = m; m = aterm; m = libtb; m = error_support; m = pt_support;}

component Sglr{ provides ISglr p; requires IATerm aterm; IToolbusLib libtb; IErrorSupport error_support; IPtSupport pt_support; contains module m url="http://www.cwi.nl/sglr-3.13.tar.gz"; connects p = m; m = aterm; m = libtb; m = error_support; m = pt_support;}


Mapping Koala to Build-level Concepts



> configure --help

…

--with-aterm=DIR use ATerm component at DIR

--with-libtb=DIR use LIBTB component at DIR

--with-error_support=DIR use ERROR_SUPPORT component at DIR

--with-pt_support=DIR use PT_SUPPORT component at DIR

…

> configure --help

…

--with-aterm=DIR use ATerm component at DIR

--with-libtb=DIR use LIBTB component at DIR

--with-error_support=DIR use ERROR_SUPPORT component at DIR

--with-pt_support=DIR use PT_SUPPORT component at DIR

…

> curl http://www.cwi.nl/sglr-3.13.tar.gz | tar xfz -> curl http://www.cwi.nl/sglr-3.13.tar.gz | tar xfz -

> configure --with-aterm=<aterm-binding> \

--with-libtb=<libtb-binding> \

--with-error_support=<error_support-binding> \

--with-pt_support=<pt_support-binding>

> configure --with-aterm=<aterm-binding> \

--with-libtb=<libtb-binding> \

--with-error_support=<error_support-binding> \

--with-pt_support=<pt_support-binding>


•Koala source tree composition

•Checkout www.program-transformation.org/Tools/KoalaCompiler

Implementation

ParsingParsing Realization

RealizationNormalizationNormalization

generic

Component

definition

PackageComponent

definition


Unbound Diversity/Requires Interfaces

•Propagate upwards to top-level configuration interface

r

interface IConf {

string foo=“some_value”;

string bar;

}

interface IConf {

string foo=“some_value”;

string bar;

}

> configure --help

--with-bar=VAL bind bar to VAL

--with-r=DIR use R component at DIR

> configure --help

--with-bar=VAL bind bar to VAL

--with-r=DIR use R component at DIR

> configure --with-foo=“some_value”> configure --with-foo=“some_value”

> configure --with-foo=“some_value” \

--with-bar=bar_binding() \

--with-r=r_binding()

> configure --with-foo=“some_value” \

--with-bar=bar_binding() \

--with-r=r_binding()

rr


Online Package Base

www.program-transformation.org/package-basewww.program-transformation.org/package-base


Overview




•Decoupling into build-level components



Three Phases of Source Tree Decoupling

1. Source tree analysis

2. Source tree transformation

3. Online package base creation

Performed successfully for graphviz (www.graphviz.org)


Source Tree Analysis

•Find components

•Find component references

•Fine tune–Add missing arrows–Remove cycles–Combine nodes with strong cohesion


•Create components

•Create component definitions

•Fine tune

Source Tree Transformation

sglr-3.13.tar.gzsglr-3.13.tar.gz


Decoupling Mozilla

•Cyclic dependencies

•Centralized build/configuration knowledge

•Cyclic dependencies

•Centralized build/configuration knowledge


Dealing with Cyclic Dependencies

•108 potential build-level components•4 cyclic-dependent clusters•37 non-cyclic components

•108 potential build-level components•4 cyclic-dependent clusters•37 non-cyclic components


Dealing with Cyclic Dependencies

•For each cluster create a composite component and a Koala component definition

•One provides interface for each sub-component

•On-demand refactoring to remove cyclic dependencies

•If all cyclic dependencies for a sub-component are removed, the component can be separated


Dealing with centralized build/configuration

•Build knowledge–Duplicate build system for each component, or …

–Generate (Automake) build process, or …–Dedicated component with build knowledge

•Configuration knowledge–Duplicate for each component,–Prune to remove irrelevant code


Overview





•Concluding remarks


Concluding Remarks

•Reusable functionality is often not independently deployable due to build-level deficiencies

•CBSE principles help to decouple source trees into build-level components

•Composition of build-level components can be automated

•Introducing CBSE principles can be (partly) automated

Merijn de Jonge IPA Spring Days 2005 April 1, 2005 Build-level Components.

Documents

Transcript of Merijn de Jonge IPA Spring Days 2005 April 1, 2005 Build-level Components.