REGNET Gloria Lau, Kincho Law, Gio Wiederhold June 8th, 2004 Legal Information Retrieval and...

REGNETREGNET

Gloria Lau, Kincho Law, Gio WiederholdJune 8th, 2004

Legal Information Retrieval Legal Information Retrieval and Application to E-Rulemakingand Application to E-Rulemaking

http://www.stanford.edu/

http://soe.stanford.edu/

22

MotivationMotivation

Multiple sources of regulations Multiple jurisdictions: federal, state, local, etc. Different formats, terminologies, contexts

UK DDA in HTMLADAAG in HTML

Amending rules, conflicting ideas

IBC in PDF

33

MotivationMotivation

Multiple sources of regulations Multiple jurisdictions: federal, state, local, etc. Different formats, terminologies, contexts Amending rules, conflicting ideas

Need for a repository Locate relevant information E.g., small business: penalty fees for violations

Need for analysis tool Complexity of regulations

Multiple jurisdictions Understanding of regulations & their relationships

44

Example 1: Related ProvisionsExample 1: Related Provisions

ADAAG Appendix 4.6.3

… Such a curb ramp opening must be located within the access aisle boundaries, not within the parking space boundaries.

CBC 1129B.4.3

… Ramps shall not encroach into any parking space.

Exception: 1. Ramps located at the front of accessible parking spaces may encroach into the length of such spaces …

CBC allows curb ramps encroaching into accessible parking stall access aisles, while ADA disallows encroachment into any portion of the stall.

55

Example 2: Related but Conflicting Example 2: Related but Conflicting ProvisionsProvisions

ADAAG 4.7.2Slope. …Transitions from ramps to walks, gutters, or streets shall be flush and free of abrupt changes…

CBC 1127B.5.5Beveled lip. The lower end of each curb ramp shall have a ½ inch (13mm) lip beveled at 45 degrees as a detectable way-finding edge for persons with visual impairments.

ADAAG focuses on wheelchair traversal; CBC focuses on the visually impaired when using a cane.

66

Relatedness analysis

Repository development

ScopeScope

generic features

domain-specific features

shallow parser

regulations in HTML, PDF,plain text, etc

feature extractor

Ontology

XML regulations

measurements exceptions definitions

Semio

concepts

author-prescribed

indicesglossary termsrefined XML regulations

DomainExpert

chemicals

effective dates

Similarity Analysis Core

domain knowledge

score refinements

feature matching

measurements

concepts

effective dates

drinking watercontaminants

base score

neighbor inclusion

reference distribution

refined score

discard belowthreshold pairs

related pairs

author-prescribed

indices

ontology (synonymicinformation) . . .

refined XMLregulations

. . .

domain-specificscoring algorithm

+

Repository development Relatedness analysis Performance evaluation , results and

applications

77

Repository developmentRepository development

generic features


shallow parser


feature extractor

Ontology

XML regulations


Semio

concepts

author-prescribed


DomainExpert

chemicals

effective dates

88

Sources of dataSources of data

Accessibility standards Americans with Disabilities Act Accessibility Guide

(ADAAG) Drafted chapter for rights-of-way access Associated public comments

Uniform Federal Accessibility Standards (UFAS) British Standard BS 8300 Scottish Technical Standards, Part S International Building Code (IBC), Chapter 11

Drinking water standards Code of Federal Regulations, Title 40 (40 CFR) California Code of Regulations, Title 22 (22 CCR)

Fire code International Building Code (IBC), Chapter 9

99

Computational properties of regulationsComputational properties of regulations

4.7

4

4.1 4.5 Ground andFloor Surfaces.

4.9

4.7.4 Surface.Slopes of curb rampsshall comply with 4.5.

ADAAG

unbounded number of descendents

unboundedtree depth

child nodereference node

Hierarchical tree structure Referential structure Discipline-centered, e.g., ADAAG for accessibility Shallow parser to capture computational properties

<regulation id="ibc" name="international building code" type="private">

<regElement id="ibc.1107" name="special occupancies"> …

<regElement id="ibc.1107.2" name=“assembly area seating">

<reference id="ibc.1107.2.4.1" times="1" />

<concept name="assembl area" times="1" /> …

<regText>Assembly areas with fixed seating shall comply … </regText>

<regElement id="ibc.1107.2.1" name="services">...</regElement>

<regElement id="ibc.1107.2.2" name=“wheelchair …">...</regElement>

</regElement>

</regElement>

</regulation>

1010



ScopeScope

generic features


shallow parser


feature extractor

Ontology

XML regulations


Semio

concepts

author-prescribed


DomainExpert

chemicals

effective dates


domain knowledge

score refinements

feature matching

measurements

concepts

effective dates


base score

neighbor inclusion


refined score


related pairs

author-prescribed

indices



. . .


+

Repository development Relatedness analysis Performance evaluation , results and

applications

1111

Related elements: door and entrance

Relatedness analysisRelatedness analysis


domain knowledge

score refinements

feature matching

measurements

concepts

effective dates


base score

neighbor inclusion


refined score


related pairs

author-prescribed

indices



. . .


+

ADAAG4.1.6(3)(d) Doors(i) Where it is technically infeasible to comply with clear opening width requirements of 4.13.5, a projection ...

UFAS4.14.1 Minimum NumberEntrances required to be accessible by 4.1 shall be part of an accessible route and shall comply with ...

1212

Relatedness analysisRelatedness analysis

To utilize the computational properties of regulations for a complete comparison

Measure Degree of relatedness: similarity score f(A, U) (0,

1) Nodes A and U are provisions from two different

regulation trees

f (0, 1)A U

ADAAG UFAS

parent

sibling

child

psc(A) psc(U) ref(U)

child node

reference node

nodes in comparison

1313

Base score Base score ff00 computation computation

Linear combination of feature matching

F(A,U,i) = similarity score between Sections (A,U) based on feature i

N = total number of features = weighting coefficient


domain knowledge

score refinements

feature matching

measurements

concepts

effective dates


base score

neighbor inclusion


refined score


related pairs

author-prescribed

indices



. . .


+

Feature matching Based on the Vector model using cosine similarity as the

distance between feature vectors Non-Boolean features

A measurement of “2 inches max” can be a 70% match to “2 inches”

Synonyms exist, e.g., ontology defined for chemicals Perform vector-space transformation prior to cosine

computation

N

i i iUAFUAf10 ),,(),(

11

N

i i

1414

Score refinements based on regulation Score refinements based on regulation structurestructure Neighbor inclusion

Diffusion of similarity between clusters of nodes in the tree Similarity Analysis Core

domain knowledge

score refinements

feature matching

measurements

concepts

effective dates


base score

neighbor inclusion


refined score


related pairs

author-prescribed

indices



. . .


+

A1 U1

ADAAG UFAS

psc(A1) psc(U1)

A2

U2psc(A2)

psc(U2)

child node

nodes in comparison

spread of similarity

red: similar nodes

blue: dissimilar nodes

1515

Score refinements based on regulation Score refinements based on regulation structurestructure

Reference distribution Diffusion of similarity between referencing nodes and

referenced nodes in the tree E.g., f(A5.3, U6.4(a)) updates f(A2.1, U3.3)

ADAAG--------------------------

Section 2.1-----------------------------------------------------------------

Section 5.3--------------------------

UFAS---------------------------------------

Section 3.3-----------------------------------------------------------------

Section 6.4(a)-------------

no crossreference

similarsections: fo != 0

reference

1616



ScopeScope

generic features


shallow parser


feature extractor

Ontology

XML regulations


Semio

concepts

author-prescribed


DomainExpert

chemicals

effective dates


domain knowledge

score refinements

feature matching

measurements

concepts

effective dates


base score

neighbor inclusion


refined score


related pairs

author-prescribed

indices



. . .


+

Repository development Relatedness analysis Performance evaluation, results and

applications

1717

Performance evaluationPerformance evaluation

Conduct a user survey of rankings of similarity 10 randomly chosen sections from the ADAAG and

UFAS Ranks 1 to 100 in the order of relevance

Root mean square error (RMSE)

= user-generated ranking vector = machine-predicted ranking vector

hr

mr

RMSEN

rrrr

r

rr mNhNmh

h

mh22

112 )()(

oflength

||

1818

Survey results - Tabulated RMSE’sSurvey results - Tabulated RMSE’s Compared our analysis to Latent Semantic Indexing (LSI)

= structural weighting coefficient = feature weighting coefficient Average RMSE smaller than LSI Measurement feature performs best No improvement in result observed for structural comparison

1919

Results of comparisons: ADAAG vs. UFASResults of comparisons: ADAAG vs. UFAS Related accessible elements:

door and entrance Neighbor inclusion reveals

higher similarity

ADA Accessibility Guidelines 4.1.6(3)(d) Doors (i) Where it is technically infeasible to comply with clear opening width requirements of 4.13.5, a projection of 5/8 in maximum will be permitted for the latch side stop. (ii) If existing thresholds are 3/4 in high or less, and have (or are modified to have) a beveled edge on each side, they may remain.

Uniform Federal Accessibility Standards 4.14.1 Minimum Number 4.14 Entrances 4.14.1 Minimum Number Entrances required to be accessible by 4.1 shall be part of an accessible route and shall comply with 4.3. Such entrances shall be connected by an accessible route to public transportation stops, to accessible parking and passenger loading zones, and to public streets or sidewalks if available (see 4.3.2(1)). They shall also be connected by an accessible route to all accessible spaces or elements within the building or facility.

4.1.6(3) 4.14

4.1.6(3)(d)Doors

4.14.1Entrances:

Minimum Number

4.14.2

4.1.6(3)(a)

4.1.6(3)(c)

4.1.6(3)(h)

ADAAG UFAS

parent

sibling

2020

Results of comparisons : UFAS vs. BS8300Results of comparisons : UFAS vs. BS8300

4.13 Doors 12.5.4 Doors

4.13.9Door Hardware

12.5.4.2Door Furniture

12.5.4.14.13.1

4.13.3

4.13.2

4.13.12

UFAS BS8300

parent

sibling

Terminological differences - revealed through neighbor inclusion

Uniform Federal Accessibility Standards 4.13.9 Door Hardware 4.13 Doors 4.13.1 General ... 4.13.9 Door Hardware Handles, pulls, latches, locks, and other operating devices on accessible doors shall have a shape that is easy to grasp with one hand and does not require tight grasping ...

... 4.13.12 Door Opening Force

British Standard 8300 12.5.4.2 Door Furniture 12.5.4 Doors 12.5.4.1 Clear Widths of Door Openings 12.5.4.2 Door Furniture Door handles on hinged and sliding doors in accessible bedrooms should be easy to grip and operate by a wheelchair user or ambulant disabled person ...

2121

Results of comparisons : UFAS vs. Scottish Results of comparisons : UFAS vs. Scottish Technical StandardsTechnical Standards Terminological differences - revealed through reference

distribution Stairs and ramps

UFAS 4.1.2 Accessible Buildings: New Construction (4) Stairs connecting levels that are not connected by an elevator shall comply with 4.9. Scottish Technical Standards 3.17 Pedestrian Ramps A ramp must have (a) a width at least the minimum required for the equivalent type of stair in S3.4; and (b) a raised kerb at least 100mm high on any exposed side of a flight or landing, except – a ramp serving a single dwelling.

2222

Application domain: e-rulemaking Comparison between draft of rules and the associated

public comments ADAAG Chapter 11, rights-of-way draft

Less than 15 pages Over 1400 public comments received within 4 months Comments ~ 10MB in size; most are several pages long New regulation draft can easily generate a huge amount of

data that needs to be reviewed and analyzed Parsing of the draft and comments

From HTML to XML Recreate structure of the draft using our shallow parser Extract features from the draft and comments Treat individual comments as provisions

Application: e-rulemakingApplication: e-rulemaking

2323

E-rulemakingE-rulemaking

Drafted regulations compared with public comments

Content ofSection 1105.4

6 Related Public Comments

1105.4 [6]

2424

Related section in draft and public comment

Results from e-rulemaking applicationResults from e-rulemaking application

ADAAG Chapter 11 Rights-of-way Draft 1105.4.1 Length Where signal timing is inadequate for full crossing of all traffic lanes or where the crossing is not signalized, cut-through medians and pedestrian refuge islands shall be 72 inches (1830 mm) minimum in length in the direction of pedestrian travel.

Public Comment Deborah Wood, October 29, 2002 I am a member of The American Council of the Blind. I am writing to express my desire for the use of audible pedestrian traffic signals to become common practice. Traffic is becoming more and more complex, and many traffic signals are set up for the benefit of drivers rather than of pedestrians. This often means walk lights that are so short in duration that by the time a person who is blind realizes they have the light, the light has changed or is about to change, and they must wait for the next walk light. ...

2525

Results from e-rulemaking applicationResults from e-rulemaking application

No related provisions identified Concern not addressed in the draft

ADAAG Chapter 11 Rights-of-way Draft [None Retrieved] No relevant provision identified

Public Comment Donna Ring, September 6, 2002 If you become blind, no amount of electronics on your body or in the environment will make you safe and give back to you your freedom of movement. You have to learn modern blindness skills from a good teacher. You have to practice your new skills. Poor teaching cannot be solved by adding beeping lights to every big Street corner! I am blind myself. I travel to work in downtown Baltimore and back home every workday by myself. I go to meetings and musical events around town. I use the city bus and I walk, sometimes I take a cab or a friend drives me. Some of the blind people who work where I do are so poor at travel they can only use that lousy “mobility service” or pay a cab. Noisy street corners won’t help them ...

2626

ConclusionsConclusions

A framework for regulatory repository Structure of regulations recreated in XML Feature extractions

Prototype for similarity comparisons Contextual comparisons Domain knowledge Structural comparisons

Performance Evaluation, Results and Applications User survey and comparisons with LSI Observations of comparisons between Federal, State, non-

profit organization mandated codes and European standards

Application on e-rulemaking

2727

Future research directionsFuture research directions In the legal domain

Regulatory competition Cross border data transfer laws Especially in the polyglot countries in EU

Regulatory updates Track changes in updates Track cross references between regulations

Extension of application to other domains of semi-structured documents Software specifications User manuals

Similarity/relatedness is settled - how about differences and conflicts? A lot of almost identical provisions

2828

Thank You!Thank You!

REGNET Gloria Lau, Kincho Law, Gio Wiederhold June 8th, 2004 Legal Information Retrieval and...

Documents

Transcript of REGNET Gloria Lau, Kincho Law, Gio Wiederhold June 8th, 2004 Legal Information Retrieval and...