EIC and OAGi Addressing the Challenges of Cross- Industry Interoperability.
National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou...
Transcript of National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou...
![Page 1: National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou Sidi Presented to OAGi April 30 th, 2008.](https://reader037.fdocuments.in/reader037/viewer/2022110402/56649e4f5503460f94b4753b/html5/thumbnails/1.jpg)
National Institute of Standards and
Technology
1
Testing and Validating OAGi NDRs
Puja Goyal
Salifou Sidi
Presented to OAGi
April 30th, 2008
![Page 2: National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou Sidi Presented to OAGi April 30 th, 2008.](https://reader037.fdocuments.in/reader037/viewer/2022110402/56649e4f5503460f94b4753b/html5/thumbnails/2.jpg)
National Institute of Standards and
Technology
2
Outline
• Naming and Design Rules• Quality of Design Tool (QOD)• OAGi NDR v9.0 Encoding
– Testability
– Observations
– Demo
• NDR Authoring/Sharing• What’s Next
![Page 3: National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou Sidi Presented to OAGi April 30 th, 2008.](https://reader037.fdocuments.in/reader037/viewer/2022110402/56649e4f5503460f94b4753b/html5/thumbnails/3.jpg)
National Institute of Standards and
Technology
3
NDR: Naming and Design Rules
• Rules applied when creating standards for the exchange of information– Any of a series of rules documents since 2003
that restrict XML Schema flexibility– Indicate features which should, may, may not,
should not, must, or must not be used.
![Page 4: National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou Sidi Presented to OAGi April 30 th, 2008.](https://reader037.fdocuments.in/reader037/viewer/2022110402/56649e4f5503460f94b4753b/html5/thumbnails/4.jpg)
National Institute of Standards and
Technology
4
What’s in an NDR?
• Naming Rules: – Names (or tags) represent information to be
exchanged– Subject names to consistent naming practices
for clarity and consistency in an XML schema.
• Design Rules– Address how information is structured
• Superstructures: namespaces
• Microstructures: global vs. local elements
![Page 5: National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou Sidi Presented to OAGi April 30 th, 2008.](https://reader037.fdocuments.in/reader037/viewer/2022110402/56649e4f5503460f94b4753b/html5/thumbnails/5.jpg)
National Institute of Standards and
Technology
5
Benefits of Encoding Rules
• NDRs written in English– Easy to misinterpret
– Schema testing and validation not automated
• Encoded NDRs– Result in better rules
• Verifies rule is clear
• Validates rule is being used as intended
– Can be computationally enforced
– Cycle time from requirements for a schema to production XML is reduced
![Page 6: National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou Sidi Presented to OAGi April 30 th, 2008.](https://reader037.fdocuments.in/reader037/viewer/2022110402/56649e4f5503460f94b4753b/html5/thumbnails/6.jpg)
National Institute of Standards and
Technology
6
QOD – Apply rules directly
• Interface to write Test Requirements (rules) and associate with Test Cases (encoded rules)– Schematron– Java-Expert System Shell (JESS)
• Automate testing and validation against schema(s)– Displays and stores results from execution against
Schematron/JESS
• Repository to store rules– Easy to reuse– Group rules to create Test Profile
![Page 7: National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou Sidi Presented to OAGi April 30 th, 2008.](https://reader037.fdocuments.in/reader037/viewer/2022110402/56649e4f5503460f94b4753b/html5/thumbnails/7.jpg)
National Institute of Standards and
Technology
9
OAGi NDR v9.0 Encoding
• Rule Testability
• Additional observations
• Demo
![Page 8: National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou Sidi Presented to OAGi April 30 th, 2008.](https://reader037.fdocuments.in/reader037/viewer/2022110402/56649e4f5503460f94b4753b/html5/thumbnails/8.jpg)
National Institute of Standards and
Technology
10
Rule Testability
• Fully Testable – Rule can be encoded without any assumptions
• Partially Testable– Some assumptions are needed
• Not Testable– unable to encode the rule
![Page 9: National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou Sidi Presented to OAGi April 30 th, 2008.](https://reader037.fdocuments.in/reader037/viewer/2022110402/56649e4f5503460f94b4753b/html5/thumbnails/9.jpg)
National Institute of Standards and
Technology
11
Testability of OAGi Rules
OAGi Rules Testability
53%
13%
34%
Fully Testable
Partially Testable
Not Testable
![Page 10: National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou Sidi Presented to OAGi April 30 th, 2008.](https://reader037.fdocuments.in/reader037/viewer/2022110402/56649e4f5503460f94b4753b/html5/thumbnails/10.jpg)
National Institute of Standards and
Technology
12
Not Testable Rules – From Our Perspective
• QOD Limitations– Related to XML instance
• Lack of clarity
• Nondeterministic– Lack of information in the OAGi schemas
• Documentation or Guidance– Supplementary information
![Page 11: National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou Sidi Presented to OAGi April 30 th, 2008.](https://reader037.fdocuments.in/reader037/viewer/2022110402/56649e4f5503460f94b4753b/html5/thumbnails/11.jpg)
National Institute of Standards and
Technology
13
Examples: OAGi-76, OAGi-70
- Ambiguous: OAGi-76 : “xsd:notation MUST NOT be used.”
– Which notation must not be used? xsd:notation element or xsd:NOTATION type ?
- Nondeterministic: OAGi-70
“Minor version schema MUST incorporate all XML constructs from the immediately preceding major or minor version schema” - No minor version information in OAGi schemas
![Page 12: National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou Sidi Presented to OAGi April 30 th, 2008.](https://reader037.fdocuments.in/reader037/viewer/2022110402/56649e4f5503460f94b4753b/html5/thumbnails/12.jpg)
National Institute of Standards and
Technology
14
Additional Observations
• Overlaps– “Fully Overlapped”
• Generalization vs. Restriction
• Generic rule may be removed for same results
– “Partially Overlapped”• Occasionally can combine rules if same type of schema
• Links (one rule refers to another)– For testing purposes, all referenced rules have to be
included
AB
Rule ARule B
![Page 13: National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou Sidi Presented to OAGi April 30 th, 2008.](https://reader037.fdocuments.in/reader037/viewer/2022110402/56649e4f5503460f94b4753b/html5/thumbnails/13.jpg)
National Institute of Standards and
Technology
15
Overlaps between rules
• Example :
OAGi-11 “XML element, attribute and type names constructed from dictionary entry names MUST NOT include periods, spaces, or other separators; or characters not allowed by W3C XML 1.0 for XML names.”
OAGi-10 “Element, attribute and type names MUST be drawn from the following set: a – z and A – Z.”
• OAGi-10 is restriction of OAGi-11
• OAGi-11 can be deleted from the NDR without affecting the original intention
![Page 14: National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou Sidi Presented to OAGi April 30 th, 2008.](https://reader037.fdocuments.in/reader037/viewer/2022110402/56649e4f5503460f94b4753b/html5/thumbnails/14.jpg)
National Institute of Standards and
Technology
17
Demo
• Overview of the QOD main functionalities (Public vs Private)
• Demo Test Profile :
- Rules related to the BOD
- Rules related to the usage of annotation and documentation
elements
• Schema to be tested:
- OAGi Developer BOD version 9.0
• Full Test Results of the execution of OAGi Developer BOD schemas version 9.0, 9.1 and 9.2 against the rules that we have encoded.
![Page 15: National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou Sidi Presented to OAGi April 30 th, 2008.](https://reader037.fdocuments.in/reader037/viewer/2022110402/56649e4f5503460f94b4753b/html5/thumbnails/15.jpg)
National Institute of Standards and
Technology
18
OAGi Developer BOD Testing :
Materials
• Schemas tested- Developer BOD schemas version 9.0 : 434 files
- Developer BOD schemas version 9.1 : 434 files
- Developer BOD schemas version 9.2 : 494 files
• Profiles tested- General Profile : 25 rules
- BOD Profile : 5 rules
![Page 16: National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou Sidi Presented to OAGi April 30 th, 2008.](https://reader037.fdocuments.in/reader037/viewer/2022110402/56649e4f5503460f94b4753b/html5/thumbnails/16.jpg)
National Institute of Standards and
Technology
19
General Profile BOD Profile TotalDeveloper BOD v9.0 errors
Developer BOD v9.1 errors
Developer BOD v9.2 errors
Rule OAGi 90 : 1 error found
- GetReceiveItem.xsd
Rule OAGi 98 : 6 errors found
- ShowCustomerPartyMaster.xsd
- ShowPartyMaster.xsd
- ShowPayable.xsd
- ShowSupplierPartyMaster.xsd
- SyncCatalog.xsd
- SyncSupplierPartyMaster.xsd
Rule OAGi 100 : 6 errors found
- GetLocationService.xsd
- ShowLocationService.xsd
Error details
Rule OAGi 98 : 6 errors found
- ShowCustomerPartyMaster.xsd
- ShowPartyMaster.xsd
- ShowPayable.xsd
- ShowSupplierPartyMaster.xsd
- SyncCatalog.xsd
- SyncSupplierPartyMaster.xsd
Error details
13
6
0
2
1
0
Rule OAGi 31 : 2 errors found
- GetReceiveItem.xsd
- GetRouting.xsd
Rule OAGi 31 : 1 error found
- GetRouting.xsd
15
7
0
OAGi Developer BOD Testing : Results
![Page 17: National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou Sidi Presented to OAGi April 30 th, 2008.](https://reader037.fdocuments.in/reader037/viewer/2022110402/56649e4f5503460f94b4753b/html5/thumbnails/17.jpg)
National Institute of Standards and
Technology
20
NDR Sharing Environment
• Facilitate reuse of rules– Different groups have different requirements
but some overlap– Group relevant tests to create customized test
profile– Upload new tests to share with others
![Page 18: National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou Sidi Presented to OAGi April 30 th, 2008.](https://reader037.fdocuments.in/reader037/viewer/2022110402/56649e4f5503460f94b4753b/html5/thumbnails/18.jpg)
National Institute of Standards and
Technology
21
NDR Authoring Environment
• Capability to export/import rules– Edit rules offline – Display in multiple ways
• HTML, XML, PDF, Spreadsheet, DocBook, etc
– Customize NDR to fit your needs– Based on NDR Profile Schema
• Standard structured vocabulary for describing NDR rules
• Available: http://ecc02.eccnet.com/ndr/
![Page 19: National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou Sidi Presented to OAGi April 30 th, 2008.](https://reader037.fdocuments.in/reader037/viewer/2022110402/56649e4f5503460f94b4753b/html5/thumbnails/19.jpg)
National Institute of Standards and
Technology
26
What’s next
• Create stand alone system
• Enhance support for rule sharing by including rule traceability
• Pursue similar support for developing tests for constraints on instance data
• Incorporate feedback into our tools!
![Page 20: National Institute of Standards and Technology 1 Testing and Validating OAGi NDRs Puja Goyal Salifou Sidi Presented to OAGi April 30 th, 2008.](https://reader037.fdocuments.in/reader037/viewer/2022110402/56649e4f5503460f94b4753b/html5/thumbnails/20.jpg)
National Institute of Standards and
Technology
28
Contacts• KC Morris , NIST – [email protected] • Simon Frechette, NIST – [email protected]• Josh Lubell, NIST – [email protected] • Puja Goyal, NIST –[email protected]• Salifou Sidi, NIST – [email protected]• Betty Harvey, ECC, Inc. -- [email protected]
Websites
• QOD onlinehttp://syseng.nist.gov/b2bTestbed/projects/QOD/
• Developer’s site for QOD tool and executable rules: http://qod.sourceforge.net/
• XML Tools available from MSID: http://www.mel.nist.gov/msid/xml_related.htm
• DAS XSI Working Group: https://www.collab.core.gov/CommunityBrowser.aspx?id=2234