Service-Oriented Architecture for automatic markup of documents

Service-Oriented Architecture for automatic markup of documents.

An use case for legal documents.

Francisco Adolfo Cifuentes-Silva

Library of Congress of Chile - BCN

2014-08-19

“Digital law libraries at the crossroads: Innovative solutions to complex challenges.”

Project contextStrategic decisions - SOA - Linked Open Data - Akoma-NtosoAutomatic markup - Named Entity Recognizer - URI assignment - Structural Markup - Akoma-Ntoso translatorResults and discussionConclussionsFuture workAcknowledgements

Project context

It borns in response to two (2) problems:

To be able for to obtain all the parliamentary interventions, within the legislative process (Congress sessions and related documents)

To know the evolution and the discussion around a law, since that this is defined as a bill until it is published as law

Francisco Adolfo Cifuentes-Silva - Library of Congress of Chile 2

11

22


Project context

It borns in response to two (2) problems:




And in an automated way!

And in an automated way!

11

22


Project context

How to: Two (2) sibling projects:

Parliamentary Labor project (PL):


History of the Law project (HL):



11

22


Project context

“Sibling projects” because both are possible processing the same documents:

• Session dailies• Debate reports• Reports• Amendments• Bills• etc.




Project context



Congress and legal resources

Project context



Chilean Congress

- Senate- Chamber of Deputies

Project context



Legal resources production

- Session dailies- Debate reports- Bills, etc

Project context




Workflow

Project context



Business Processes

- Each type of document has an own process flow

- BCN implements a Workflow Management System for PL & HL

Project context




Tools

Project context

Workflow



Support tools

- Automatic XML Marker - Web XML Editor- XSD in the base of support tools

Project context




Tools

XMLStorage

Project context

Workflow



XML Storage

- SVN server for XML documents- Allow us manage all XML versions - REST access: HTTP GET, PUT

Project context



Tools

XMLStorage

Information extraction

Linked Open Data


Project context

Workflow



Information Extraction

New information is extracted from enriched XML in two formats:

- Linked Open Data- Relational data (facts table)

Project context



Tools

XMLStorage


Linked Open Data


Project context

Workflow



Tools

XMLStorage


Linked Open Data


New data is used for a new process

Project context

Workflow


Strategic decisions

Service Oriented ArchitectureOur focus:- HTTP is the base- REST Web Services- W3C Web Standards



Strategic decisions

Service Oriented Architecture


Workflow Management SystemWorkflow Management System

Automatic MarkupAutomatic Markup XML EditorXML Editor RDF TriplestoreRDF Triplestore

SVN XMLSVN XML MediatorMediator Web ServicesWeb Services


Strategic decisions

Linked Open Data - LODSince 2011 BCN publishes LOD:

Dataset of legal norms Dataset of legislative documents Datasets and ontologies about:

People Geographic places Organizations Others like roles, bills, congress structure, etc.

Please visit http://datos.bcn.cl !!


http://datos.bcn.cl/


Strategic decisions

Linked Open Data For automatic markup we are using:• URIs for legal documents• URIs for metadata• URIs for named entities:

– URIs for people– URIs for organizations– URIs for roles– URIs for events– URIs for locations– …. URIs for all



Strategic decisions

The definition of a XML Schema

We need a XML Schema for markup of documents, and eventually interchange the documents, so we have two big choices:

• Own XML Schema = low interoperability, reusability and high cost

• Standard XML Schema = high interoperability, reusability and low cost



Strategic decisions



Strategic decisions

The definition of a XML Schema

Standard XML Schema = high interoperability, reusability and low cost

Ok but, why Akoma-Ntoso?



Strategic decisions

Akoma-Ntoso

- XML Schema for legal documents designed and supported by “great minds” in OASIS Group

- Support to many types of documents:(session daily, bills, debate reports, amendments, among others)



Strategic decisions

Akoma-Ntoso

- There is a growing set of tools for working with him, such as Web XML editors or office editor tools, example:– LegisProWeb – Bungeni– Lime Editor


Project contextStrategic decisions - SOA - Linked Open Data - Akoma-NtosoAutomatic markup in XML - Named Entity Recognizer - URI assignment - Structural Markup - Akoma-Ntoso translatorResults and discussionConclussionsFuture workAcknowledgements

Automatic markup in XML


PlainText

Named Entities recognitionNamed Entities recognition

URI assignmentURI assignment

Structural MarkupStructural Markup

Akoma-Ntoso translationAkoma-Ntoso translationXMLAKN

Automatic XMLMarker



PlainText





Automatic XMLMarker




Named Entity Recognizer (NER)

- We need to identify entities in the text- We are using a spanish adapted version

of Stanford NER which uses a CRF classifier.

- The classifier was trained with large documents achieving results over 80% of effectivity in entity recognition




Named Entity Recognizer (NER)

Web service, written in Java and based in the Stanford NER





PlainText





Automatic XMLMarker



URI assignment

- Once the NER find all entities, we need to assign its URI

- This tool is called “The Mediator” and it has been developed in collaboration with the Weso Research Group of the University of Oviedo.

Francisco Adolfo Cifuentes-Silva - Francisco Adolfo Cifuentes-Silva - Library of Congress of Chile 34



Mediator output in XML

Web service, written in Java and based in Apache Lucene




Mediator features

- Connected to SPARQL Endpoint- It allows to set context information for each

work session (ex: date, chamber, type of doc. in markup)

- Using the context information, it applies a set of heuristics for each entity type, identifying correctly the URI for each one





PlainText





Automatic XMLMarker



Structural markup

- The problem is to detect structural sections

- Combination of methods:- Regular expressions- Algorithms for detecting sequences- Rules and algorithms




Structural markup

- The combination of methods depends on each document type

- Finally, the object representation of document (simmilar to DOM) is converted to ad-hoc XML




Structural markup

Web service and written in Java





PlainText





Automatic XMLMarker



Akoma-Ntoso translator

- We need AKN documents for edition, enrichment and extraction

- AKN is a complex schema- The best solution was to build a web

service for convert ad-hoc XML to AKN




Akoma-Ntoso translator

Web service and written in Java



Results and discussion

Positive impact in the work, reducing dramatically time of XML markup compared to manual labeling of documents

reducing time and cost of product generation



Results and discussion


Time for completing a History of the Law in distinct scenarios


Conclusions

SOA has provided to improve each component separately impacting positively the final result (ex. Datasets, NER training, heuristics)

It is possible to integrate aditional XML Schemas to output



Conclusions

The automatic markup of XML documents, and subsequent manual enrichment of metadata provides an excelent source for data extraction

Our solution based on SOA allow us an easy integration of exceptions and new cases in the markup



Future Work

Alfonso Pérez, Director of the BCN, has installed the concept of “Semantic Library” like one of the main objectives of the BCN in the institutional strategic plan.

This new concept implies to apply the automatic markup schema to all BCN areas, developing new markup schemas and possible new challenges in terms of identify document sections and semantic content.



Acknowledgements

• Library of Congress of Chile Team • Developers team

– Ricardo Muñoz– Claudio Devia– Eridan Otto– David Vilches– Me


Thanks for your attention!

fcifuentes <at> bcn <dot> cl

twitter.com/fcifuentes

www.slideshare.net/francisco.cifuentes

www.linkedin.com/in/fcifuentes


Me

If you need more details, you can contact me:

http://www.slideshare.net/francisco.cifuentes

http://www.linkedin.com/in/fcifuentes

Service-Oriented Architecture for automatic markup of documents

Software

Transcript of Service-Oriented Architecture for automatic markup of documents