Morgan Stanley-Prototype an XBRL Taxonomy Object … Stanley-Prototype an... · Validation of...
Transcript of Morgan Stanley-Prototype an XBRL Taxonomy Object … Stanley-Prototype an... · Validation of...
Final Report
Open Source XBRL API
And Client-Server Application
NYU Student Team Yao-Tsung (Dave ) Tsai Tarun Brij Bh Aggarwal Steven D. Ching Brian K. Ales Chisato Enomoto Professor Arthur Goldberg Morgan Stanley’s Manager Mark Schnitzer, Senior manager, Executive Director, Equity Research
May 12th, 2003
Page 2 of 16
Table of Contents
1. User Guide ................................................................................................................................ 3
2. System Configuration.............................................................................................................. 6
3. Detail List of Code ................................................................................................................... 9
4. Deployment Process .............................................................................................................12
5. Future Development Plan.....................................................................................................16
Page 3 of 16
1. User Guide
1.1 Summary of functionalities
Initial release has following functionalities. 1.1.a – Repository with version control
Users can upload a local client file to the database.
i. Taxonomies – composed of six component files in one zip file.
1. Check name uniqueness for
a. Target namespaces in schema files – no two taxonomy schema
files can define the same namespace.
b. Taxonomy names – taken from the filename by default or
supplied by the user.
2. Users can perform Validation on upload before writing to database (see
validation requirement below) if they choose to do so.
ii. Instance documents – single XML-compliant files
Check name uniqueness for an Instance document file names – taken from the filename by
default or supplied by the user.
Users can download Taxonomies and Instance documents
1.1.b - Validation function
Validate Taxonomy Document – validation against XBRL and W3C specifications.
Schema – validate against W3C Schema for Schemas (S4S).
Linkbases – validate against XBRL-Linkbase Schema.
1.1.c - View and Edit ( partial ) function
View Taxnomoy Document – display document contents
Schema and Linkbases
Page 4 of 16
1.2 Process Flow Diagram
User reaches HomeSelects a) Open b) Upload c) Download d) Exit
List boxes enumeratingexisting Taxonomies andInstance Documents in
database.User selects one
Upload HomeUser selects:
I) a) Published Taxonomy b) Unpublished Taxonomy
c) Instance DocumentII) Zip file to upload
III) Database file name
Close browser window
List box enumerating existingversions of selected taxonomy.
User selectsa) Version
Open/Download Upload
UnpublishedTaxonomy
List box enumerating existingversions of selected taxonomy.
User selectsa) Version
b) Type of document
PublishedTaxonomy/Instance
Document
Download
Open
UnpublishedTaxonomy
List box enumerating types ofdocuments of selected taxonomy.
User selectsa) Type of document
PublishedTaxonomy
Exit
Confirmationof successful upload
Confirmationof successful download
Browser View
View
InstanceDocument
Upload
Page 5 of 16
1.3 Initial Release Limitation
Current system handles individual document operations and validation. It does not handle the
interrelations between the main taxonomy schema document and the related Linkbases.
• The document versioning system we will implement through having multiple saved documents
(representing a change history) will not allow branching. This means that documents older than
either the most recently saved one or the current working copy can only be viewed, not edited.
• Administration of user accounts and user authorizations is deemed to be out of scope for this
project. Our team therefore assumes that an application user has administrative privileges and
can set whether a taxonomy is published or not during upload. In addition, we will assume that
these uploaded taxonomies are initially valid.
• Although the ability of a user to upload an invalid document would be useful in situations where
the user wanted to use our application to find and fix the problem, for simplicity and project scope
purposes, we have constrained the system to store only valid documents. The exception would
be the active copy in the database that reflects both valid and invalid document changes, and
hence may be invalid until validation is performed.
• Concurrency issues for online applications are also out-of-scope for our project. Therefore, there
will only be one document instance under modification at any given time.
Page 6 of 16
2. System Configuration
2.1 Overall Architecture
The flowing is a high-level diagram operational diagram of WEBXBRL. Although HTTP Request/Response parameters are used for passing files between client and server, our system uses HTTP Session variables to:
• Store the name of the next operation • Pass and return data between servlets • Store error messages
The central servlet in our system is UINavigator. A user sends HTML forms to this servlet, which then calls the appropriate ‘Service Servlet’:
• Archive • DB • Download (DownloadTaxonomy, DownloadInstance) • EditTaxonomy (used for View in version 1.0) • ParseValidate • UpLoad (UploadTaxonomy, UpnloadInstance)
The service servlet returns XML ByteStreams to the UINavigator, which then calls Xalan to process the XML into HTML with the appropriate XSL template.
Page 7 of 16
2.2 System Requirements
Frameworks:
Xalan 2.5.D1
Xerces 2.4.0
JAXP 1.2
XUpdate specification
`
Servlet Engine: Apache Tomcat 4.1.24
Database Server: Xindice 1.1b
Page 8 of 16
2.3 Database Directory Design
Following diagram shows how directories are structured in Xindice.
Page 9 of 16
3. Detail List of Code
3.1 All code is saved and documented in the CD-R in following format.
Page 10 of 16
3.2 List of files
1. Base package : com.msdw.xbrl 2. archive Archive.java 3. common Common.java 4. db.connect DBAccess.java DBAccessImpl.java DBServlet.java 5. db.update EditTaxonomy.java EditTaxonomyImpl.java 6. exception DBException.java UnzippingException.java UpdateException.java ValidationException.java ViewException.java WebXBRLException.java ZippingException.java 7. io Download.java DownloadInstance.java DownloadTaxonomy.java Upload.java UploadInstance.java UploadTaxonomy.java 8. ui UIFormatter.java UINavigator.java 9. utility Attribute.java DocumentPage.java DOWNLOAD_UNPUBLISHED_TAXONOMYPage.java DOWNLOADPage.java Element.java HOMEPage.java OPENPage.java Page.java ParseValidate.java StringFormatter.java TaxLinkbaseErrorHandler.java TaxSchemaErrorHandler.java UPLOAD_HOMEPage.java UPLOADPage.java
Page 11 of 16
VALIDATEPage.java VIEWPage.java
3.3 XSLT Templates
Template Names Function Generates following HTML pages. OpenUnpublishedTaxonomy_HTML.xslt OpenUnpublishedTaxonomy SaveInstanceDocumentError_HTML.xslt SaveInstanceDocumentError UploadHome_HTML.xslt UploadHome DownloadUnpublishedTaxonomy_HTML.xslt DownloadUnpublishedTaxonomy OpenDownload_HTML.xslt OpenDownload OpenPublishedTaxonomy_HTML.xslt OpenPublishedTaxonomy View_HTML.xslt View Templates for Errors DefaultError_HTML.xslt DefaultError SavePublishedTaxonomyError_HTML.xslt SavePublishedTaxonomyError SaveUnpublishedTaxonomyError_HTML.xslt SaveUnpublishedTaxonomyError UploadInstanceDocumentError_HTML.xslt UploadInstanceDocumentError UploadPublishedTaxonomyError_HTML.xslt UploadPublishedTaxonomyError UploadUnpublishedTaxonomyError_HTML.xslt UploadUnpublishedTaxonomyError ValidateInstanceDocumentError_HTML.xslt ValidateInstanceDocumentError ValidatePublishedTaxonomyError_HTML.xslt ValidatePublishedTaxonomyError ValidateUnpublishedTaxonomyError_HTML.xslt ValidateUnpublishedTaxonomyError ValidationError_HTML.xslt ValidationError
Page 12 of 16
4. Deployment Process
4.1 Software requirements:
1) Java2 SDK 1.4.1_02 • Available from http://java.sun.com/j2se/1.4.1/download.html
2) Apache Tomcat 4.1.24 • Available from
http://jakarta.apache.org/builds/jakarta-tomcat-4.0/release/v4.1.24/
3) Xindice 1.1b • Required files included in the CD, under the xindice directory
4) Xalan 2.4.0 • Available from http://xml.apache.org/xalan-j/downloads.html • Required files also included in the CD, under the libraries directory
5) Xerces 2.4.0
• Available from http://xml.apache.org/dist/xerces-j • Required files also available in the CD, under the libraries directory
4.2 Installation instructions:
1) Install the Java2 SDK. The installation directory will be referred to as
<JAVA_HOME> for the rest of this documentation
2) Install Apache Tomcat. The installation directory will be referred to as <CATALINA_HOME> for the rest of this documentation
3) From the resources directory in the CD, unzip the xslt.zip file into a new directory.
This directory will be referred to as <XSLT_DIR> for the rest of this documentation
Page 13 of 16
4) Create a new temporary directory with read and write permissions, or an existing temporary directory with these permissions can also be used. This directory will be referred to as <TEMP_DIR> for the rest of this documentation
5) From the libraries directory in the CD, copy the following files into
<JAVA_HOME>\jre\lib\endorsed
a. dom.jar b. jaxp-api.jar c. sax.jar d. xalan-2.4.0.jar e. xercesImpl.jar f. xml-apis.jar g. xmlParserAPIs.jar h. xsltc.jar
6) From the libraries directory in the CD, copy the following files into <CATALINA_HOME>\common\endorsed
a. xalan-2.4.0.jar b. xercesImpl.jar c. xmlParserAPIs.jar
7) From the xindice directory in the CD, copy the Xindice.war file into
<CATALINA_HOME>\webapps
8) Copy the webxbrl directory from the CD into <CATALINA_HOME>\webapps 9) Under the <CATALINA_HOME>\webapps\webxbrl\WEB-INF directory, the
following modifications need to be made to the web.xml file, using any standard text editor:
a. Under the <servlet> tag, with the <servlet-name> as Database, the
<param-value> of the <param-name> as DBServer, has to be changed to reflect the server and port of the Xindice database server. By default, Xindice is assumed to be running on the same server as the WebXBRL app, and so the default value for this parameter is localhost:8080. Below is the portion of the web.xml file with its default value. <servlet> <servlet-name>Database</servlet-name>
Page 14 of 16
<servlet-class>com.msdw.xbrl.db.connect.DBServlet</servlet-class> <init-param> <param-name>DBServer</param-name> <param-value>localhost:8080</param-value>
<description>Server and port where Xindice is running</description>
</init-param> </servlet>
b. Under the <servlet> tag, with the <servlet-name> as UIFormatter, the <param-value> of the <param-name> as xsltDirectory, has to be changed to the <XSLT_DIR>. By default, this value is c:/xslt/. Please ensure the trailing / is also entered as part of the directory name. Below is the portion of the web.xml file with its default value. <servlet> <servlet-name>UIFormatter</servlet-name> <servlet-class>com.msdw.xbrl.ui.UIFormatter</servlet-class> <init-param> <param-name>xsltDirectory</param-name> <param-value>c:/xslt/</param-value> <description>Absolute path to directory containing XSL
Templates</description> </init-param>
c. Under the <servlet> tag, with the <servlet-name> as UIFormatter, the
<param-value> of the <param-name> as tempDirectory, has to be changed to the <TEMP_DIR>. By default, this value is c:/temp/. Please ensure the trailing / is also entered as part of the directory name. Below is the portion of the web.xml file with its default value. <servlet> <servlet-name>UIFormatter</servlet-name> <servlet-class>com.msdw.xbrl.ui.UIFormatter</servlet-class> <init-param> <param-name>tempDirectory</param-name> <param-value>c:/temp/</param-value> <description>Absolute path to temp directory</description> </init-param> </servlet>
Page 15 of 16
4.3 Running the WebXBRL application 1) Start the Apache Tomcat servlet engine. By default it is configured to run on port
8080
2) It will load up the two new web applications which have been placed in its webapps directory
3) Open a browser window, and on typing the following URL, the WebXBRL
application should show:
http://localhost:8080/webxbrl/index.html
Page 16 of 16
5. Future Development Plan
Proposed Future Enhancements The second phase of WEBXML would involve the following: Validation of Instance documents While it would be relatively trivial to create new ParseValidation methods to accommodate Instance Documents, the logic and User Interface work necessary to search for the appropriate Taxonomy Schema within our database and alert the user if it was not found prevented us form completing this stage in the time allowed. Document Editing There is code in the EditTaxonomy class that implements basic editing functions such as edit operations (via XUpdate) to specific attribute name-value pairs in both taxonomy (.xsd) and linkbase (.xml) documents. These edit operations consist of updates of specific attribute name-value pairs and deletion of tags via a key attribute. This is a possible starting point for a future front end to begin harnessing XUpdate query operations. Composite Schema/Linkbase Objects While it would be complex, XSLT templates could be written to create a ‘composite document’ representing the graph relationship between Taxonomy Schema elements as defined in the 5 LinkBase documents. This was considered briefly but decided against both for the complexity involved as well because of the possibility the XBRL consortium moves away from the XLink implementation in the near future. When this question is settled, this would be a valuable (if complicated) enhancement. Alternate Client Types We chose to use XSLT – but Excel etc. are also possible.