Agile xml

72
1 © 2011 Zynx Health Incorporated | The information contained herein is confidential and proprietary to Zynx Health Incorporated and is intended for its authorized recipient. Unauthorized review, use, disclosure or distribution is strictly prohibited. All rights reserved.

description

 

Transcript of Agile xml

Page 1: Agile xml

1 © 2011  Zynx Health Incorporated  |  The information contained herein is confidential and proprietary to Zynx Health Incorporated and is intended for its authorized recipient. Unauthorized review, use, disclosure or distribution is strictly prohibited. All rights reserved.

Page 2: Agile xml

2 © 2011  Zynx Health Incorporated  |  The information contained herein is confidential and proprietary to Zynx Health Incorporated and is intended for its authorized recipient. Unauthorized review, use, disclosure or distribution is strictly prohibited. All rights reserved.

Agile XML

There is such a thing.

Page 3: Agile xml

Agile developers like it dynamic.

JavaScript, Ruby, NoSQL

Java,.Net,SQL

Page 4: Agile xml

STATIC STUFF IS A DRAG Strict types in static languages like C++ and

Java make code more time-consuming to write and verbose to read.

Static code is harder to unit-test (type system fights mocks).

Dynamic languages present a lower barrier to entry and the code often ends up more terse and expressive.

Dynamic flexibility means no hoops to jump through when you want to change things.

Developers prefer to ensure integrity through tests rather than type safety features.

Page 5: Agile xml

Mapping data to objects produces profuse code, mostly boilerplate but obligatory neverthelessPiles of model classes and mapping artifacts,

often containing specialized configurations for indirectly tuning queries, add friction to change.

Abstracts code away from database interactions in ways that can encourage sub-optimal behavior (e.g., the "n+1 selects" problem).

But still preferable to the drudgery and maintenance nightmare of stored procedures.

SQL is a drag because it's static

Page 6: Agile xml

Static schema is strict and very difficult and complicated to change.

Normalized relational models make complex data cumbersome to manipulate in native query language.

Query language (SQL) is virtually impossible to unit-test.

SQL is a drag because it's static

Page 7: Agile xml

Developers like NoSQL

Low barrier to entry Scales horizontally Seamless programming language

integration

Page 8: Agile xml

Developers like NoSQL

Low barrier to entry Scales horizontally Seamless programming language

integration BUT MOSTLY because it's dynamic

Easier to change Easier to scale up Easier to test

Page 9: Agile xml

But when you think XML,you don't think NoSQL.

Page 10: Agile xml

When I think XML,I think Static XML

Mapping to objects strikes again• Serialization libraries map schematic structures

to classes– Classes generated once from XML schemata and

then manually maintained (JAXB)– Manually mapped classes (XStream)

• Many manually-maintained artifacts with brittle dependencies on data

• Lots of friction to change, just like SQL ORMs

Page 11: Agile xml

When I think XML,I think Static XML

Mapping to objects strikes again• Serialization libraries map schematic structures

to classes– Classes generated once from XML schemata and

then manually maintained (JAXB)– Manually mapped classes (XStream)

• Many manually-maintained artifacts with brittle dependencies on data

• Lots of friction to change, just like SQL ORMs

STATIC

Page 12: Agile xml

When I think XML,I think Static XML

• Pedantic namespace usage complicates code, especially at the edges

• Big, complicated, repetitive and impenetrable XSLT code often a core implementation feature

• Storage often still SQL-based (and slow) due to departmental culture/policy

Page 13: Agile xml

And yet…

• Nothing about XML requires that you map it to objects– Plenty of support in programming languages for

manipulation through other means

• Namespaces can be walled off or eliminated altogether

• XSLT (and XQuery) code can be designed with modularity and expressiveness– Was your first LISP program modular?– Might your tenth one have been better?

Page 14: Agile xml

Developers don't like XML itself(even though it's a dynamic document format, just like JSON)

Wordy and complicatedo AttributesoNamespaces

oMixed text/element contentoWhitespace

Tools and languages seem arcaneo XSLT / XPatho XML Schema / RelaxNGo XQuery

Limited database optionsImpedance mismatch with JSON

Page 15: Agile xml

And yet…• Attributes, namespaces and mixed

content are all optional• Can be isolated at the edges if required for

integration with other systems• Tools and languages are very powerful,

once learned• DSLs like XSLT maximize expressiveness

and leverage for their given mission• Functional approaches are gaining

popularity generally

Page 16: Agile xml

And yet…• MarkLogic is an excellent enterprise

database• Provides ACID guarantees• Tons of query, search and other features• Scales on commodity hardware (for a

price)• eXist-db is a pretty good open source

database• JVM-based• Lots of activity lately• Support available

Page 17: Agile xml

What's More…

• XML provides a rich ecosystem• Rich transformation…

– …with XQuery in the database– …with XSLT almost everywhere– …in code with rich, embedded DSLs

• Rich query…– …with XQuery in the database– …with XPath almost everywhere– …with GPath in Groovy

Page 18: Agile xml

What's More…

• XML provides multiple, rich schema standards (unlike other document NoSQL formats)– Automated validation and even content repair

at database level– Usage is completely optional; could be used

as a barrier, or just to generate information about schema violations

Page 19: Agile xml

What's More…

• XML provides multiple facilities for data integration…– …XInclude for automated document

aggregation (including automated broken link reporting)

– …Attributes, including RDFA, for aspect-oriented tagging

– …Namespaces for more aspect-oriented integration

Page 20: Agile xml

What's More…

• Breadth and depth of XML ecosystem provides all sorts of network effect benefits– Multiple implementations of various strategies

for dealing with large data sets (DOM, SAX, XPP, etc.)

– Pipelining for faster, layered work on data– Schematrons for semantic validation– Schema-based editors like InfoPath, Oxygen,

XMLSpy, Arbortext, XMetaL and Xopus

Page 21: Agile xml

Leverage Can Tip the Scale

• Complex, modular data• Complex things being done with data• Integration in an XML-rich space

– Healthcare, for example

When is it enough?…to offset the additional complexity versus JSON/JavaScript

Page 22: Agile xml

Sweet Spot for Complex Data?

Flexibility (lack of resistance to change)

Levera

ge

(pro

vid

ed

by e

cosy

stem

)

SQL

JSON NoSQL

Enterprise

XML

AgileXML

Page 23: Agile xml

The Methodology…

Page 24: Agile xml

Agile XML: the Methodology

Dynamic

•No knee-jerk object/data mapping

•Limit friction (schemata, namespaces)

Modular

•Resource orientation

•Layered, re-usable transformations

Testable

•Assert data features as well as behavior

•Unit-test XQuery, XSLT

Page 25: Agile xml

Agile XML: the Methodology Treat data dynamically

› Don't reflexively serialize to/from objects› Transform and aggregate resources as

needed for specific use cases (wrap data around problems)

› Make a schema only when you really need one

› Avoid or circumscribe high-friction features like namespaces and attributes

› Don't generate code based on XML data structures (e.g., JAXB)

Page 26: Agile xml

Agile XML: the Methodology

Be resource-oriented› Implement use cases by transforming and

assembling resources› Write transforms and other resource

processing code wherever it's most maintainable

› Favor functional approaches, where applicable, over imperative code and state

› Seek re-use, granularity and clarity in your resource transformations as you would seek them in object-oriented abstractions

Page 27: Agile xml

Agile XML: the Methodology

Ensure integrity through tests› Test specific data features instead of broadly

schema-validating› Apply test-driven development practices to

resource transformation and aggregation code, including XSLT and XQuery

› Run comprehensive data tests for continuous integration and surveillance

Page 28: Agile xml

Test-driving XSLT

• My team attempted several approaches before we got this right

• Lots of testing frameworks, not much community or clear adoption trends

• Existing frameworks emphasize deep comparison of output with expected contents (XML deep equals)

• Most are based on XML-basedtesting DSLs

Page 29: Agile xml

Test-driving XSLT

Typical framework-based XSL unit test:

<test> <code>my-stylesheet.xsl</code> <input>input-doc.xml</input> <expect>output-doc.xml</expect></test>

<foo> <bar /> <baz /></foo>

input-doc.xml

<moe> <larry /> <curly /></moe>

output-doc.xml

<moe> <larry /> <shemp /></moe>

Page 30: Agile xml

Test-driving XSLT

• Problems with this approach:– Test input and output must be data-complete

• Verbose, laborious and brittle• Easy to lose track of what's important in individual tests• Tests force a modularity that mightn't otherwise make sense

in XSL code

– XML-based testing DSLs limit flexibility• Limited set of assertions• No general-purpose programming language

features available for writing test fixtures andother clever things

• Even if JUnit output is supported, can'tmake full use of JUnit features in IDE

Page 31: Agile xml

Test-driving XSLT

<output> <stuff/></output>

<output> <stuff/></output><output> <stuff/></output><output> <stuff/></output><output> <stuff/></output>

<input> <stuff/></input>

<input> <stuff/></input><input <stuff/></input><input <stuff/></input><input <stuff/></input>

TEST

TEST

TEST

TEST

TEST

XSLT Stylesheet

input expectations

output behavior

new inputnew input

new input

new input

new input

new input

new output

new output

new output

new output

new output

new output • Burdensome• Discourages

test-writing

Test Suite

The Brittleness Problem

Page 32: Agile xml

Test-driving XSLT

The Forced Modularity Problem

Output-driven XSLT<html> <body> <xsl:for-each select="//item"> <div> <xsl:value-of select="text()"/> </div> </xsl:for-each> </body></html>

Input-driven XSLT<xsl:template match="/"> <html><body> <xsl:apply-templates select=".//item"/> </body></html></xsl:template>

<xsl:template match="item"> <div> <xsl:value-of select="text()"/> </div></xsl:template>

• Need to simplify expected test outputs may push you to the right

• Many problems are simpler to solve on the left

Page 33: Agile xml

Test-driving XSLT

• We use code to simplify inputs and narrow dependencies on outputs– Focus on features, not unnecessary detail

• We settled on straight JUnit for execution• Use test fixtures to manufacture

complex inputs• Use GPath, etc., to assert only

what we care about in output

Page 34: Agile xml

Test-driving XSLT

Example<movieList> <movie> <title>Citizen Kane</title> <genre>drama</genre> </movie> <movie> <title>Tommy Boy</title> <genre>comedy</genre> <rating>R</rating> </movie> <movie> <title>Annie Hall</title> <genre>Comedy</genre> </movie></movieList>

<catalog> <genre name="Comedy"> <film>Annie Hall</film> <film mpaa="R"> Tommy Boy </film> </genre> <genre name="Drama"> <film>Citizen Kane</film> </genre></catalog>

XSLT

Page 35: Agile xml

Test-driving XSLT

Test fixtures simplify complex inputs:

FragmentsString makeMovie(Map fields) { """<movie> <title>${ fields?.title ?: 'Fake Title' }</title> <genre>${ fields?.genre ?: 'fakumentary' }</genre> ${fields?.rating ? '<rating>' + fields.rating + '</rating>' : ''} </movie>"""}

DocumentsString makeMovieList(List movies) { """<movieList> ${movies.join('\n')} </movieList>"""}

Usage@Testvoid shouldTransformMovies() { String input = makeMovieList([ makeMovie(title: 'Primer') ])

// act, assert…}

Page 36: Agile xml

Test-driving XSLT

Tests analyze specific features:@Testvoid shouldAggregateIntoGenres() { List movies = (1..2).collect { makeMovie(genre: 'drama') } movies << makeMovie(genre: 'comedy')

def result = parseXml(transform(makeMovieList(movies)))

assert result.genre.size() == 2}

@Testvoid shouldCanonicalizeGenreNames() { String input = makeMovieList([ makeMovie(genre: 'science fiction'), makeMovie(genre: 'SCIENCE FICTION') ])

def result = parseXml(transform(input))

assert result.genre.size() == 1 assert result.genre.'@name' == 'Science Fiction'}

Page 37: Agile xml

Test-driving XSLT

Use of rich language features can help keep tests short and expressive:

@Testvoid shouldSortMovieTitles() { String input = makeMovieList([ makeMovie(title: 'Zorro', rating: '2'), makeMovie(title: 'Catching Fire', rating: '1'), makeMovie(title: 'case insensitive', rating: '0') ])

def result = parseXml(transform(input))

0..2.each { assertEquals( it.toString(), result.genre.film[it].'@mpaa' ) }}

Page 38: Agile xml

Test-driving XQuery

• xray, xquery-unit, XQUT and others for MarkLogic

• XQSuite for eXist-db• TDD works for developing XQuery code

– My team does it (though we could be more disciplined)

• Same tools can be used for integration tests– Check features of data stored in database– Re-use for DB integrity checks, monitoring

Page 39: Agile xml

Test-driving XQuery

• HOWEVER: Functional language makes test diagnosis more painful (no "print to console")– Alternatively, tests can produce diagnostic information

as query results on failure

• Some messiness required to test code that modifies the database in MarkLogic

Page 40: Agile xml

Test-driving XQuery

Query result contains diagnostic information to help understand test failures.

Tests a specific feature, not just "XML deep equals"

Page 41: Agile xml

Test-driving XQuery

Testing side-effects of code that writes to the database can be tricky. Here, we use MarkLogic's xdmp:eval function to launch transactions in sequence.

Page 42: Agile xml

Writing Maintainable XSLT and XQuery

• Test-driven development is critical– Well-written tests document the code they're

testing– Comprehensive tests document

comprehensively– Without an infrastructure that at least supports

test-driven development, comprehensive tests will never be written

Page 43: Agile xml

• Readable tests use names to tell a story– Test (function) names should follow some

Given-When-Then-like convention– Variable names should be thoughtfully chosen

with storytelling in mind– Add a variable, even if unneeded, just to give

a name to something if it needs explanation

Writing Maintainable XSLT and XQuery

Page 44: Agile xml

• XSLT is a flexible language, so employ patterns that work for your team

• Most people find output-driven stylesheets easier to read than input-driven ones– Easier to think in terms of end product– Named templates add more context– Imperative queries map more closely to imperative

programming experience– CAUTION: Performance cost can get significant

Writing Maintainable XSLT

Page 45: Agile xml

• Again, use variables to help tell stories even when they're unnecessary

• Use xsl:include for modularity and re-use• Use modes only when necessary; they are easy

to ignore and add to cognitive load

Writing Maintainable XSLT

Page 46: Agile xml

• Ummm… variables!– Lift deeply nested expressions out into "let" variables,

where possible

• Use function modules for modularity and re-use– Try to curate them as deliberately as you do other

kinds of source modules

• Prefer literal XML to element constructors when element names aren't dynamic

Writing Maintainable XQuery

Page 47: Agile xml

Database Migrations• Migrations framework took only a few days to

write and integrate into our CI pipeline– Includes easy data ingestion facility based on file

system

• Arbitrary XQuery scripts can make whatever changes they want

• Migrations run in split seconds– If data size (running time) becomes an issue, the

ecosystem offers us several approaches

Page 48: Agile xml

Database Migrations

One day, we decided to stop version-managing a category of documents. Here's what the migration looked like:

for $doc in cts:search( /citation, dls:documents-query() )return dls:document-unmanage( fn:base-uri($doc), fn:false(), fn:true() )

And here's a fix for some damaged data:

for $empty-desc in /somePath/description[ fn:string-length() = 0 ]return xdmp:node-delete( $empty-desc )

Page 49: Agile xml

Protecting Database Integritywith XML Schemata

When we decided that we had a real need to make ingestion ironclad for certain data, we started using schema validation.

XML Schema has rich features for validating both structure and values, though some find the semantics cumbersome (thus, the existence of a popular alternative, RELAX NG)

Interactive editors provide gracefully interchangeable text and diagrammatic views

Required namespaces, but we quarantined their use at the DB layer (showcased)

Page 50: Agile xml

Protecting Database Integritywith XML Schemata

Schema validation usually requires namespace usage. We wanted schema validation in our database layer, so we implemented namespaces in just the database layer and quarantined it there with simple transformations:

<doc xmlns="…"> <stuff></doc>

<doc> <stuff></doc>

Add namespace based on what's being written

Strip all namespaces onout-bound data

Easy for us, being XCC-based,

but alternatives exist.

Page 51: Agile xml

Treating Data Dynamically

You don't care about all the extra stuff on a jQuery event object, as

long as it's got what you need.

If jQuery adds stuff, it won't affect you.

If you owned this object and you changed or removed stuff, you'd

use tests to make sure the rest of your code still works.

Page 52: Agile xml

Treating Data Dynamically

• In an Agile XML application, your code is also loosely coupled to its resources

• No need to care about data noise or changes that don't affect you– Changes from/for other code– xml:base, xml:type and schema

location attributes from other systems

Page 53: Agile xml

Treating Data Dynamically

• Your code doesn't care– It's not mapping data to objects– It's not schema-validating data

• Your tests don't care– They're not using "XML deep equals"– They're modeling and examining

only what's important about the data• Dynamic data is changeable data!

Page 54: Agile xml

Treating Data Dynamically

• Manage change through tests– Unit tests where your changes originate

(and wherever else you remember)– Integration tests cover data that cross

boundaries (i.e., code you forgot)– Database-layer tests can cover persistent

data changes comprehensively• Continuous integration step• Integrity monitoring

– Functional tests cover changed data as they are manufactured, stored, retrieved, transformed and consumed

Page 55: Agile xml

Chaos?

Page 56: Agile xml

Resource Orientation = Chaos?

Modularizing through resources can scatter business logic.• Variety of solution technologies to handle variety

of problems• XQuery makes investment of logic at database

layer more attractive– Real (though bizarre) functional programming

language– Proximity to data (reduction of round trips)

Page 57: Agile xml

BREAKING DOWN "CHECKLIST RELEASE"Exclusive Feature Agile XML

VenueUser saving with "released" status means release, otherwise save.

Both UI and Web API controller

User is not allowed to release a new checklist. UI

Wrap multiple write queries into a transaction, rolled back on error.

Checklist service

Gather PubMed citation IDs from scoped intervention outcome measurements.

Checklist service + checklist DB library

Acquire citation contents from PubMed Web API. Checklist service + PubMed service

Transform (boil down) PubMed citations and store them in database.

PubMed service + PubMed DB library

Change status of referenced scoped interventions to "released" and save new versions of them.

Checklist + scoped intervention DB libraries

Add released scoped intervention version # to references in checklist.

Checklist DB library

Save new version of checklist. Checklist DB libraryShared Feature Agile XML Venue

Stored checklists contain distilled scoped intervention references.

Checklist DB library

Data structure details are needed for XML /JSON conversion.

Web API controller

Page 58: Agile xml

BREAKING DOWN "CHECKLIST RELEASE"Exclusive Feature Agile XML

VenueSQL Venue

User saving with "released" status means release, otherwise save.

Both UI and Web API controller

Both UI and controller or model

User is not allowed to release a new checklist. UI UI

Wrap multiple write queries into a transaction, rolled back on error.

Checklist service Service/model

Gather PubMed citation IDs from scoped intervention outcome measurements.

Checklist service + checklist DB library

Service/model

Acquire citation contents from PubMed Web API. Checklist service + PubMed service

Service

Transform (boil down) PubMed citations and store them in database.

PubMed service + PubMed DB library

Service and maybe DB (XML ingestion)

Change status of referenced scoped interventions to "released" and save new versions of them.

Checklist + scoped intervention DB libraries

Service/model and DB

Add released scoped intervention version # to references in checklist.

Checklist DB library Service/model

Save new version of checklist. Checklist DB library Service/model and DBShared Feature Agile XML Venue

SQL Venue

Stored checklists contain distilled scoped intervention references.

Checklist DB library Model

Data structure details are needed for XML /JSON conversion.

Web API controller Controller

Page 59: Agile xml

BREAKING DOWN "CHECKLIST RELEASE"Exclusive Feature Agile XML

VenueSQL Venue

User saving with "released" status means release, otherwise save.

Both UI and Web API controller

Both UI and controller or model

User is not allowed to release a new checklist. UI UI

Wrap multiple write queries into a transaction, rolled back on error.

Checklist service Service/model

Gather PubMed citation IDs from scoped intervention outcome measurements.

Checklist service + checklist DB library

Service/model

Acquire citation contents from PubMed Web API. Checklist service + PubMed service

Service

Transform (boil down) PubMed citations and store them in database.

PubMed service + PubMed DB library

Service and maybe DB (XML ingestion)

Change status of referenced scoped interventions to "released" and save new versions of them.

Checklist + scoped intervention DB libraries

Service/model and DB

Add released scoped intervention version # to references in checklist.

Checklist DB library Service/model

Save new version of checklist. Checklist DB library Service/model and DBShared Feature Agile XML Venue

SQL Venue

Stored checklists contain distilled scoped intervention references.

Checklist DB library Model

Data structure details are needed for XML /JSON conversion.

Web API controller Controller

UI

Page 60: Agile xml

BREAKING DOWN "CHECKLIST RELEASE"Exclusive Feature Agile XML

VenueSQL Venue

User saving with "released" status means release, otherwise save.

Both UI and Web API controller

Both UI and controller or model

User is not allowed to release a new checklist. UI UI

Wrap multiple write queries into a transaction, rolled back on error.

Checklist service Service/model

Gather PubMed citation IDs from scoped intervention outcome measurements.

Checklist service + checklist DB library

Service/model

Acquire citation contents from PubMed Web API. Checklist service + PubMed service

Service

Transform (boil down) PubMed citations and store them in database.

PubMed service + PubMed DB library

Service and maybe DB (XML ingestion)

Change status of referenced scoped interventions to "released" and save new versions of them.

Checklist + scoped intervention DB libraries

Service/model and DB

Add released scoped intervention version # to references in checklist.

Checklist DB library Service/model

Save new version of checklist. Checklist DB library Service/model and DBShared Feature Agile XML Venue

SQL Venue

Stored checklists contain distilled scoped intervention references.

Checklist DB library Model

Data structure details are needed for XML /JSON conversion.

Web API controller Controller

UI

APP

Page 61: Agile xml

BREAKING DOWN "CHECKLIST RELEASE"Exclusive Feature Agile XML

VenueSQL Venue

User saving with "released" status means release, otherwise save.

Both UI and Web API controller

Both UI and controller or model

User is not allowed to release a new checklist. UI UI

Wrap multiple write queries into a transaction, rolled back on error.

Checklist service Service/model

Gather PubMed citation IDs from scoped intervention outcome measurements.

Checklist service + checklist DB library

Service/model

Acquire citation contents from PubMed Web API. Checklist service + PubMed service

Service

Transform (boil down) PubMed citations and store them in database.

PubMed service + PubMed DB library

Service and maybe DB (XML ingestion)

Change status of referenced scoped interventions to "released" and save new versions of them.

Checklist + scoped intervention DB libraries

Service/model and DB

Add released scoped intervention version # to references in checklist.

Checklist DB library Service/model

Save new version of checklist. Checklist DB library Service/model and DBShared Feature Agile XML Venue

SQL Venue

Stored checklists contain distilled scoped intervention references.

Checklist DB library Model

Data structure details are needed for XML /JSON conversion.

Web API controller Controller

UI

APP

DB

Page 62: Agile xml

Case Study: Clinical Order View

Page 63: Agile xml

Case Study: Clinical Order View

checklistsscoped

interventions etc.

XQuery

XSLT

Client

Page 64: Agile xml

Case Study: Clinical Order View

checklistsscoped

interventions etc.

JavaScript

Client

What if?

Page 65: Agile xml

Fetching the Datadeclare private function enriched-performance-measure($perfMeasure as node()) { return element performanceMeasure { $perfMeasure/*, /performanceMeasure[fn:normalize-space(id) = fn:normalize-space($perfMeasure/*[fn:local-name() = 'id'])]/abbreviation }};

declare private function enriched-impact-threshold($impactThreshold as node()) { return element impactThreshold { $impactThreshold/*, element pubMedCitation { let $citation := /pubMedCitation[fn:normalize-space(id) = fn:normalize-space($impactThreshold/*[fn:local-name() = 'pubMedId']/text())] return ( element title {zpmc:get-article-title($citation)}, element journalInfo {zpmc:get-journal-info($citation)}, element authorList {zpmc:get-authors-list($citation)} ) } }};

declare function enrich-scoped-intervention($element as element()) as element() { return element { fn:node-name($element) } { $element/@*, for $n in $element/node() return typeswitch ($n) case element(si:performanceMeasure) return enriched-performance-measure($n) case element(si:impactThreshold) return enriched-impact-threshold($n) case element() return enrich-scoped-intervention($n) default return $n }};

declare private function produce-enriched-checklist($element as element()) as element() { element { fn:node-name($element) } { $element/@* , for $n in $element/node() return typeswitch ($n) case $siRef as element(scopedIntervention) return let $original := zsi:get-scoped-intervention-by-id($siRef/id, $siRef/version/version-id cast as xs:unsignedInt) return zsi:enrich-scoped-intervention($original)

case $e as element() return produce-enriched-checklist($e, $fields-to-include)

default return $n }};

declare function get-checklist($id as xs:string, $version as xs:unsignedInt) { let $uri := checklist-uri-from-id($id) let $doc := c:get-document-with-version-metadata-embedded($uri, $version) return produce-enriched-checklist($doc)};

55 Xquery lines

1 round trip

Page 66: Agile xml

Fetching the Datadeclare private function enriched-performance-measure($perfMeasure as node()) { return element performanceMeasure { $perfMeasure/*, /performanceMeasure[fn:normalize-space(id) = fn:normalize-space($perfMeasure/*[fn:local-name() = 'id'])]/abbreviation }};

declare private function enriched-impact-threshold($impactThreshold as node()) { return element impactThreshold { $impactThreshold/*, element pubMedCitation { let $citation := /pubMedCitation[fn:normalize-space(id) = fn:normalize-space($impactThreshold/*[fn:local-name() = 'pubMedId']/text())] return ( element title {zpmc:get-article-title($citation)}, element journalInfo {zpmc:get-journal-info($citation)}, element authorList {zpmc:get-authors-list($citation)} ) } }};

declare function enrich-scoped-intervention($element as element()) as element() { return element { fn:node-name($element) } { $element/@*, for $n in $element/node() return typeswitch ($n) case element(si:performanceMeasure) return enriched-performance-measure($n) case element(si:impactThreshold) return enriched-impact-threshold($n) case element() return enrich-scoped-intervention($n) default return $n }};

declare private function produce-enriched-checklist($element as element()) as element() { element { fn:node-name($element) } { $element/@* , for $n in $element/node() return typeswitch ($n) case $siRef as element(scopedIntervention) return let $original := zsi:get-scoped-intervention-by-id($siRef/id, $siRef/version/version-id cast as xs:unsignedInt) return zsi:enrich-scoped-intervention($original)

case $e as element() return produce-enriched-checklist($e, $fields-to-include)

default return $n }};

declare function get-checklist($id as xs:string, $version as xs:unsignedInt) { let $uri := checklist-uri-from-id($id) let $doc := c:get-document-with-version-metadata-embedded($uri, $version) return produce-enriched-checklist($doc)};

function enrichedScopedIntervention(id) { var si = db.scopedInterventions.findOne({'id': id}); si.performanceMeasures.forEach(function (pm) { pm.abbreviation = db.performanceMeasures.findOne({'id': pm.id}).abbreviation; }); si.impactThresholds.forEach(function (th) { var citation = db.pubMedCitations.findOne({'id': th.pubMedId}) th.pubMedCitation = { title: getArticleTitle(citation), journalInfo: getJournalInfo(citation), authorList: getAuthorList(citation) }; });}

function getChecklist(id, version) { var checklist = db.checklists.findOne({'id': id + '_' + version}); checklist.groups.forEach(function (group) { for (i = 0; i < group.scopedInterventions.length; ++i) { group.scopedInterventions[i] = enrichedScopedIntervention(group.scopedInterventions[i].id); } }); return checklist;}

26 JavaScript lines

> 200round trips

Page 67: Agile xml

Generating the View<xsl:template name="intervention"> <xsl:param name="intervention-group-key"/> <xsl:param name="intervention-name" /> <intervention> <id> <xsl:copy-of select="normalize-space($intervention-group-key)"/> </id> <displayName><xsl:value-of select="$intervention-name" /></displayName> <xsl:copy-of select="current-group()[1]/shouldAvoid"/>

<hasOutcomes> <xsl:value-of select="exists(current-group()/outcomes/outcomeContainer/outcome)" /> </hasOutcomes> <hasGuidelines> <xsl:value-of select="exists(current-group()/guidelines/guideline)" /> </hasGuidelines>

<scopes> <xsl:for-each-group select="current-group()" group-by="concat(local:canonicalize-field-value-ids(., 'careSetting'), '_', local:canonicalize-field-value-ids(., 'ageGroup'))"> <xsl:sort> <xsl:variable name="care-setting-names" select="local:canonicalize-field-values-for-sorting(., 'careSetting')" /> <xsl:variable name="age-group-names" select="local:canonicalize-field-values-for-sorting(., 'ageGroup')" /> <xsl:value-of select="concat($care-setting-names, '__', $age-group-names)" /> </xsl:sort>

<xsl:variable name="sub-group-key" select="current-grouping-key()"/> <scope> <xsl:variable name="first-si" select="current-group()[1]"/> <ageGroupName> <xsl:value-of select="local:format-field-values-for-display($first-si, 'ageGroup')"/> </ageGroupName> <careSettingName> <xsl:value-of select="local:format-field-values-for-display($first-si, 'careSetting')"/> </careSettingName> <id> <xsl:value-of select="$intervention-group-key"/> <xsl:text>__</xsl:text> <xsl:value-of select="$sub-group-key"/> </id> <scopedInterventions> <xsl:for-each select="current-group()"> <xsl:copy-of select="." /> </xsl:for-each> </scopedInterventions> </scope> </xsl:for-each-group> </scopes> </intervention></xsl:template>

84 linesof XSLT

Page 68: Agile xml

<xsl:template name="intervention"> <xsl:param name="intervention-group-key"/> <xsl:param name="intervention-name" /> <intervention> <id> <xsl:copy-of select="normalize-space($intervention-group-key)"/> </id> <displayName><xsl:value-of select="$intervention-name" /></displayName> <xsl:copy-of select="current-group()[1]/shouldAvoid"/>

<hasOutcomes> <xsl:value-of select="exists(current-group()/outcomes/outcomeContainer/outcome)" /> </hasOutcomes> <hasGuidelines> <xsl:value-of select="exists(current-group()/guidelines/guideline)" /> </hasGuidelines>

<scopes> <xsl:for-each-group select="current-group()" group-by="concat(local:canonicalize-field-value-ids(., 'careSetting'), '_', local:canonicalize-field-value-ids(., 'ageGroup'))"> <xsl:sort> <xsl:variable name="care-setting-names" select="local:canonicalize-field-values-for-sorting(., 'careSetting')" /> <xsl:variable name="age-group-names" select="local:canonicalize-field-values-for-sorting(., 'ageGroup')" /> <xsl:value-of select="concat($care-setting-names, '__', $age-group-names)" /> </xsl:sort>

<xsl:variable name="sub-group-key" select="current-grouping-key()"/> <scope> <xsl:variable name="first-si" select="current-group()[1]"/> <ageGroupName> <xsl:value-of select="local:format-field-values-for-display($first-si, 'ageGroup')"/> </ageGroupName> <careSettingName> <xsl:value-of select="local:format-field-values-for-display($first-si, 'careSetting')"/> </careSettingName> <id> <xsl:value-of select="$intervention-group-key"/> <xsl:text>__</xsl:text> <xsl:value-of select="$sub-group-key"/> </id> <scopedInterventions> <xsl:for-each select="current-group()"> <xsl:copy-of select="." /> </xsl:for-each> </scopedInterventions> </scope> </xsl:for-each-group> </scopes> </intervention></xsl:template>

Generating the View

function makeScopeSubGroups(group, interventionGroupKey) { var result = []; var scopeSubGroups = []; var makePredicate = function (scopedIntervention) { return function (scopeSubGroup) { if (scopeSubGroup.id === makeScopeSubGroupKey(scopedIntervention)) { scopeSubGroup.members.push(scopedIntervention); return true; } else { return false; } } }; section.scopedInterventions.forEach(function (si) { if (! scopeSubGroups.some(makePredicate(si))) { scopeSubGroups.push({ id: makeScopeSubGroupKey(si), members: [si] }); } }); scopeSubGroups.sort(function (first, second) { var firstCanonical = canonicalizeFieldValuesForSorting(first.careSettings) + '__' + canonicalizeFieldValuesForSorting(first.ageGroups); var secondCanonical = canonicalizeFieldValuesForSorting(second.careSettings) + '__' + canonicalizeFieldValuesForSorting(second.ageGroups); return first.localeCompare(second); }); scopeSubGroups.forEach(function (subGroup) { var firstSi = subGroup.members[0]; var realSubGroup = { ageGroupName: formatFieldValuesForDisplay(firstSi.ageGroups), careSettingName: formatFieldValuesForDisplay(firstSi.careSettings), id: interventionGroupKey + '__' + subGroup.id, scopedInterventions: [] }; subGroup.members.forEach(function (si) { realSubGroup.scopedInterventions.push(si); }); result.push(realSubGroup); }); return result;}

118 lines ofJavaScript

Page 69: Agile xml

<xsl:with-param name="intervention-group-key"> <xsl:text>section-</xsl:text> <xsl:value-of select="normalize-space(current-group()[1]/sections/section[1]/id)"/> <xsl:text>-intervention-</xsl:text> <xsl:value-of select="normalize-space(current-group()[1]/intervention/id)"/></xsl:with-param>

var key = "section-" + interventionGroup.members[0].sections[0].id + "-intervention-" + interventionGroup.id;

In XSLT, concatenating values happens to be wordy:

VS.

Page 70: Agile xml

<xsl:for-each-group select="scopedInterventions/scopedIntervention" group-by="normalize-space(intervention/id)">

var result = []; var interventionGroups = []; var makePredicate = function (scopedIntervention) { return function (interventionGroup) { if (interventionGroup.id === scopedIntervention.intervention.id) { interventionGroup.members.push(scopedIntervention); return true; } else { return false; } } }; section.scopedInterventions.forEach(function (si) { if (! interventionGroups.some(makePredicate(si))) { interventionGroups.push({ id: si.intervention.id, members: [si] }); } }); interventionGroups.forEach(function (interventionGroup) { var key = "section-" + interventionGroup.members[0].sections[0].id + "-intervention-" + interventionGroup.id; result.push( makeInterventionGroup(interventionGroup.members, key, interventionGroup.members[0].scopedInterventionName) ); });

But JavaScript lacks transformation features like"for-each-group" that reduce real complexity:

VS.

Page 71: Agile xml

So, There are Trade-offs

• Any XML-based architecture presents a minimum level of friction versus other document NoSQL stacks

• The more complex your application's use cases become, the stronger the argument for agile XML

• Integration with external XML data and/or services (e.g., HIE) tips the scale

Page 72: Agile xml

72 © 2011  Zynx Health Incorporated  |  The information contained herein is confidential and proprietary to Zynx Health Incorporated and is intended for its authorized recipient. Unauthorized review, use, disclosure or distribution is strictly prohibited. All rights reserved.

Thank You