Post on 22-Jan-2015
description
itemis AG 2009 All rights reserved
Mastering differentiated
MDSD requirements at
Deutsche Brse AG
Code Generation 2009
Heiko Behrens (itemis)
Karsten Thoms (itemis)
The project at Deutsche Brse
! Projects delivers a Global Trading System
! First (internal) user is ISE in NY
! Scalable, Distributed System
! Goes live in 2011
Structure of this talk
! 8 timesMotivation ! Applied Approach
! Inbetween: Demo of discussed ideas
! In the end: Some discoveries we made
http://www.bildtankstelle.de
Application ofCode Generation
in this project
Messaging Model
Structures
Messages Docu
HTML,
TextC++
Factories
Messaging Generator
Msg/Struct.
Adapters
Reference Data Model
Reference Data
Generator
Configuration Model
JPA Entities
Report
Java HTML
IML
Structures
Messages
Java
DAOs
Listener
Container Model
Containers
Filters
Boost
Operator
C++
XML
Model
Data
Access
Generator
XML
Transactional
Model
Queries
DAOs
C++
DB
Service
Generator
Hibernate
SQL
DDL Scripts
JAXB
XML
WSFacade
XSD
Val.List.
UnitTest
Validation
Listeners
WSFacade
DTOs
Persistence
Descriptor
IML
Datatypes
Model
UOW Exec.
Bean
Ref Data
Mngr Class
Global key
functionsDDL Scripts
SQL
UnitTest
Templates
DDL Scripts
(MySQL)
SQL
Excel Model
System Test
Generator
Python
Tests
Messages
Structures
Messages
Structures
C++ Python
Scripting Generator
Factories
Structures
Messages
.Net
.Net
Generator
OFi
Editor
Msg Client
Descriptor
Artifact sizes
Reference Impl.Sources (manual)Tests (manual)ModelTransformationGenerated
2.747 units
Actual numbers had to be replaced by
fictive units
Guess what!
Motivation:Code Generation
Why Code Generation?! Positive experience with former projects
! Prime example: mapping between messages and structures sending with a compact binary wire protocols
ComponentComponent
B1
B2
Structure B
S1
S2
SubStructure S
A2A3A1
Component
B1
B2
Structure B
S1
S2
SubStructure S
S2B2S1B1
Why openArchitectureWare?
Former times:
! Home-grown solution based on graphical modeling and proprietary tools
! Many different post-processors such as XSLT and Perl
Evaluation results of oAW for GTS
! Supports textual modeling
! Has mature tool support
! Works with different model types and output formats
UMLXpand
JavaC++Text
Different Input
XMIXML XMIXML
Same Transformation Language, Different Output ,
When in Rome do as the Romans do
Motivation: Address existing experience and perception
Modeling in UML
! Years of experience for modeling entities graphically (ER-Diagram)
! Established Tool (MagicDraw)
! Proven Technology (Profiles, Tagged Values)
! Great for overview perspectives
UML Reference Model
Textual Modeling
! Years of experience ;)
! Established tool support (grep, editor, mail, svn)
! Highly customizable ! DSL
! Inexpensive metamodel evolvement
! Great for detailed modeling
Textual Model
!""#$%&!$'!($)#*"
Motivation:Reduce Complexity of Modeling
Reducing Model Complexity
! Following Convention-over-configuration paradigm for modeling
! Only model necessary information
! Complete model according to conventions by M2M transformation
Convention over Configuration
can be derived
required model information
uml::Model modify (uml::Model this) : ... -> // Search for unnamed association ends and give them a default name this.eAllContents.typeSelect(Property) .select(x|x.association!=null).nameUnnamedEnds() -> ...
/** * This function sets a default name for an association's end property if the property * has no name and it is navigable. */ Void nameUnnamedEnds(Property this) : if !isNamed() && isNavigable() then ( ("setting default name for association end "+opposite.type.name+" -> "+type.name).info() -> setName(computeDefaultName()) ) -> this;
Convention: Navigable association ends withoutname get the name of the referred Entity. Use plural for to
many relationships.
Convention over Configuration
uml::Model modify (uml::Model this) : ... -> // Search for unnamed association ends and give them a default name this.eAllContents.typeSelect(Property) .select(x|x.association!=null).nameUnnamedEnds() -> ...
private create uml::Property createTechnicalId(RDMProfile::Entity entity): ("Creating technical id attribute for entity "+entity.name).debug() -> entity.ownedAttribute.addToFront(this) -> this.setName("id") -> this.setType(findDataType(Long) -> this.setLower(1) -> this.setUpper(1) -> this.setIsUnique(true) -> this.setVisibility(uml::VisibilityKind::^private) -> this.applyStereotype("RDMProfile::Key") -> this.setTaggedValue("RDMProfile::Key", "keyType", RDMProfile::KeyType::PK)-> this.applyStereotype("RDMProfile::Field") -> this.setTaggedValue("RDMProfile::Field", "uniqueGroup", "444")-> this;
Convention: All entities without attribute get a new property with name id of type Long, and Stereotype
is applied.
Combination of model types
Domain modelUML2platform independent
entity Market
{
relatedTo MarketAbstract with EAGER loading and cascade with PERSIST
relatedTo MarketGroup with LAZY loading and cascade with PERSIST
}
Configuration modeltextual DSLplatform dependent
entity Market
{
relatedTo MarketAbstract with EAGER loading and cascade with PERSIST
relatedTo MarketGroup with LAZY loading and cascade with PERSIST
}
public class Market implements RDSEntity, Serializable { ... @OneToMany(mappedBy = "market", cascade = CascadeType.PERSIST) @Basic(fetch = FetchType.EAGER) private List marketAbstracts = new ArrayList(); @ManyToMany(cascade = CascadeType.PERSIST, targetEntity = MarketGroup.class, mappedBy = "markets") @Basic(fetch = FetchType.LAZY) private List marketGroups = new ArrayList();
Combination of model types
Avoidable Accidents
Motivation:Detect errors early
(c) Ernest von Rosen, www.amgmedia.com
context R
DMProfile:
:EntityRel
ation
(memberEnd
.select(e|
e.upper ==
-1).size
== 2)
WARNI
NG "Many t
o many rel
ation " +
memberEnd.
type.name.
info() +
as owner o
f the rela
tion (owne
rOfAssocia
tion is no
t set)!" :
ownerO
fAssociati
on != null
;
context u
ml::Associ
ation if m
emberEnd.t
ype.select
(e|e.isAct
ive()).siz
e>1 && (me
mberEnd.se
lect(e|
e.upper ==
-1).size
== 2) && m
emberEnd.s
elect(e|e.
aggregatio
n.toString
()!="none"
WARNI
NG "Many t
o many rel
ation " +
memberEnd.
type.name.
info() + "
must be o
f type Ent
ityRelatio
n
and define
one of th
e entities
as owner
of the rel
ation (ste
reotype no
t applied)
!"
RDMPro
file::Enti
tyRelation
.isInstanc
e(this);
context P
roperty if
class.isA
ctive() &&
(associat
ion == nul
l)
ERROR
"Property
" + name
+ " in ent
ity "+clas
s.name+" r
eferences
an entity,
but is n
ot an
associatio
n!":
!RDMPr
ofile::Ent
ity.isInst
ance(type)
;
cont
ext Proper
ty if clas
s.isActive
() && asso
ciation ==
null && ^
default!=
WARNI
NG "Proper
ty "+loc()
+" has an
empty stri
ng (not nu
ll) as def
ault value
"
="";
cont
ext Proper
ty if clas
s.isActive
() && RDMP
rofile::En
tity.isIns
tance(clas
s) && asso
ciation!=
ERROR
"Class "+
type.name+
" must be
stereotype
d as Entit
y, since i
t is assoc
iated by E
ntity
"+class.na
me+" in it
s property
"+name :
RDMPro
file::Enti
ty.isInsta
ncef(type)
;
context P
roperty if
class.isA
ctive() &&
RDMProfil
e::Entity.
isInstance
(class) &&
associati
on==
ERROR
"Only Fun
ctional Da
tatypes or
Functiona
l Enumerat
ions are a
llowed as
property t
ype.
Violated f
or "+loc()
:
RDMPro
file::Func
tionalData
Type.isIns
tance(type
) ||
RDMProfile
::Function
alEnumerat
ion.isInst
ance(type)
;
Detecting errors early
! Check constraints implemented for
! Every assumption or modeling restriction
! Modeling error that lead to generator or artifact errors (e.g. Whitespace in names)
! Inter-model consistency checks
! Risk of modeling errors decreased
context uml::Association if memberEnd.type.select(e|e.isActive()).size>1 && (memberEnd.select(e|e.upper == -1).size == 2) && memberEnd.select(e|e.aggregation.toString()!="none").size == 2 WARNING "Many to many relation " + memberEnd.type.name.info() + " must be of type EntityRelation and define one of the entities" +
" as owner of the relation (stereotype not applied)!" : RDMProfile::EntityRelation.isInstance(this);
context Property if class.isActive() && (association == null) ERROR "Property " + name + " in entity "+class.name +
" references an entity, but is not an association!": !RDMProfile::Entity.isInstance(type);
context Property if class.isActive() && association == null && ^default!=null WARNING "Property "+loc() +" has an empty string (not null) as default value" : ^default.trim()!="";
Constraints with Check language
Integration
! Static Typing while developingchecks and transformations
! IDE integration for workflows invocation
! Continuous Generation (server-side)CMake / Ant / Maven
! Automated execution after SVN checkout
Demo! Graphical EMF model
! Enhanced with textual DSL (with Xtext)
! Validation
! Code Generation
! Changes textual DSL
Standing on the shoulders of giants
Motivation:Reusing generator aspects
Cartridge adaption! Using OS cartridges as
closed package
! Required model structure created through M2M transformation
! Required changes added non-invasive through Xpand/Xtend AOP support
! Redefined functionality concentrated in a small adapter cartridge project
Annotation
Template
EntityClass
Template
Mapping
Template
Attribute
Template
Class
Template
Operation
Template
Fornax Hibernate Cartridge
Fornax JavaBasic Cartridge
Aspect
Template
M2M
Transformation
Adapter
Workflow
Cartridge
Workflow
Cartridge
Workflow
Fornax Adapter Cartridge
DTO
Template
WebService
Template
UnitTest
Template
Project Cartridge
Cartridge
Workflow
uml::Model modify (uml::Model this) : this.applyProfile(getPersistenceProfile()) -> // Create real enumerations for FunctionalEnumerations. This must be done first since // we map datatypes after that this.eAllContents.typeSelect(RDMProfile::FunctionalEnumeration).adaptEnumeration() -> entities().adaptEntity() -> adaptEmbeddedKeys() -> this;
RDMProfile::Entity adaptEntity (RDMProfile::Entity entity) : debug("Adapting entity: "+ entity.name) -> entity.applyStereotype("Persistence::Entity") -> entity.setTaggedValue("Persistence::Entity", "tableName", entity.name.asTableName())-> entity.testAndAssignInheritanceStrategy()-> entity.attribute.select(a|a.association==null).adaptAttribute() -> entity.attribute.select(a|a.association!=null).adaptRelation() -> entity;
//In case the entity extends another class, assigne the persistence stereotype for //strategy on the uml::Generalization//TPC="TABLE_PER_CLASS" | J="JOINED" | ST="SINGLE_TABLE";Void testAndAssignInheritanceStrategy(RDMProfile::Entity this): if !general.isEmpty then assignInheritanceStrategy(); Void assignInheritanceStrategy(RDMProfile::Entity this): let gener = generalization.first(): gener.applyStereotype("Persistence::"+getInheritanceStrategyFromDardl().mapToStrategy()) -> if(getInheritanceStrategyFromDardl() == "SINGLE_TABLE") then ( gener.setTaggedValue("Persistence::TablePerClassHierarchy",
"discriminator_value", getDiscriminatorValue())-> gener.setTaggedValue("Persistence::TablePerClassHierarchy",
"discriminator_superClass_column", gener.general.name+"_type")-> gener.setTaggedValue("Persistence::TablePerClassHierarchy",
"discriminator_superClass_type", "INTEGER") ) ;
UML2 model modification with Xtend
around org::fornax::cartridges::uml2::javabasic::extensions::DataType::NormalizedDefaultValue(uml::Property property): internal_getNormalizedDefaultValue(property);
private String internal_getNormalizedDefaultValue (uml::Property property) : JAVA gts.ise.refdata.oaw.fornax.util.Extensions.getNormalizedDefaultValue(org.eclipse.uml2.uml.Property);
AROUND org::fornax::cartridges::uml2::javabasic::templates::Documentation::documentation FOR uml::Classifier targetDef.proceed() EXPAND JPAAnnotationsENDAROUND
DEFINE JPAAnnotations FOR Persistence::Entity @javax.persistence.Entity @javax.persistence.Table(name="asTableName()") EXPAND InheritanceAnnotations @javax.persistence.NamedQueries({ @javax.persistence.NamedQuery(name="findnames", query="SELECT e from name e") })
IF !isAbstract @javax.persistence.EntityListeners (getQualifiedPackageName("ListenerOnce").nameValidationListener.class) ENDIF EXPAND UniqueConstraintsENDDEFINE
AOP with templates and functions
Arange your ideas
Motivation:Managing Manual Code
Sucessfully applied MDSD best practices
! Separate generated from manual code
! Avoid check-in generated code
BaseClass
AbstractA AbstractB
ConcreteBSome
Manual
Code
ConcreteA
src-gen
always generated
src-once
generated once
src
manually written
BaseClass
ConcreteB
AbstractB
ConcreteCSome
Manual
Code
ConcreteA
AbstractD
ConcreteD
Adjusted Folder Layout
Design your tools
Motivation: Managing Generator
Complexity
Reference Implementation
! Manually implemented
! Far more than a prototype!
! Compilable, deployable, executable, testable
! Covers every architectural concept
! Code seperated in to-be-generated and manual code
Reference Model
! Contains every supported modeling concept
! Not necessary excerpt from the real domain, but helps for understanding
! Try to minimize the model size
Testing the reference
! Reference Implementation is tested intensively and automatic
! Unit tests and integration tests with high coverage
! Tests against generated artifacts
! Real application does not need to be tested in the same extend
generated from reference model
manually implemented
reference
EntitiesAttributesAssociationsModel metrics
Motivation: Optimizing Generator Speedand Readability
Refactorings / Profiling
! Reference Implementation/Unit-Tests ensure valid output
! Static Typing of Xpand supports development
HTML report
HTML report
standard format (gprof)
standard format (gprof)
standard format (gprof)
Most significant problems
! Repeated calls of functions
! Suboptimal algorithms/data structures
! Branches with best-case scenarios
! Profiling reduced generation timefrom 28 min downto 3 min
Discoveries
Value of Cartridges
! Project benefited from cartridges developed by OS community
! But: Largest part of the code generators are totally project dependent
Early adopters
! Using the bleeding edge of Eclipse Modeling was challenging
! But: Continuous improvement of used technologies thanks to direct feedback with OS community
! And: Significant performance and stability gains since early adoption
Learning curve
! Establishing MDSD and tools was hard for unexperienced developers at the beginning
! But: Coaching helped to fill the gap fast
! And: Now developers educate others to use MDSD tooling
Reference Implementation
! Reference Models and Reference Implementation are invaluable
! Reducing the complexity
! Enable refactoring
! But: Effort to create and maintain a real Reference Implementation quite high
Personal blogshttp://www.1160pm.net http://kthoms.wordpress.com http://www.itemis.com