.lusoftware verification & validationVVS

Artificial Intelligence in Law:!Trends and Experience

Mehrdad (Mike) Sabetzadeh University of Luxembourg

Third Information Security Education Day (ISED)!April 28, 2017

About the Speaker • Background: Software Requirements, Regulatory Compliance

• Senior Scientist, SnT / UL (2012 - present)

• Scientist,!Simula Research Laboratory, Norway (2009 - 2012)

• Visiting Researcher, !University College London, UK (2009)

• PhD in Computer Science,!University of Toronto, Canada (2008)

2

About the Talk

3

•  Focus will on technical aspects

•  Ethical implications not covered

•  AI limitations will be briefly mentioned

Agenda for this presentation:

•  Overview of AI in Law

•  Emerging trends in Legal Compliance Tech

•  Practical experience from projects

AI in Law: A Whirlwind Tour

4

Machine Learning

Natural Language Processing

Rule-basedReasoning

Knowledge Extraction &

Discovery

Content Classification

Question AnsweringAnalyticsPrediction

AI in Law

AI T

echn

ique

s

Task

s

5

Big data,!Predictive analytics,!

Deep learning, …

The Tides have Turned

6

Geoffrey Hinton accepting the 2016 IEEE/RSE James Clerk Maxwell Medal for pioneering and sustained contributions to machine learning

“The crazy approach is winning” BUT

7

•  Machine Learning (ML) relies critically on training data

•  Biased/incomplete training à Biased/incomplete results

•  Results from many ML algorithms are opaque

•  Outcomes are not readily justifiable (going against due diligence)

•  Training not a replacement for rules and norms!

•  Do you want your car to learn the !traffic rules on the road?

Rule-based AI here to stay

8

•  Rule-based AI no longer the “only kid on the block” but will thrive and continue to play an important role

•  Business process automation critically depends on rules

•  For example, eGovernment IT applications

•  ML and NLP can facilitate the extraction of rules

•  The way rules are represented nevertheless needs to improve

•  … to cover the knowledge gap between legal and IT experts

Hype but not only …

9

•  Lot of hype about ML and NLP, but not all of it is hype

•  Seeing through the hype can be difficult

•  IT professionals can help, BUT ONLY IF they understand the business context and needs of the legal profession

•  Fostering interaction between legal and IT experts is key

•  Disruptive technologies are emerging, like it or not

•  Not joining the band wagon will almost certainly mean falling behind

What is on the horizon?

10

Legal Tech Value Chain

11

Legal ComplianceCompanies

Knowledge Management and

Analytics Companies

Legal Publishers

Governments

Legal Business Entities + Companies Subject to

Legal Compliance

Citizens

provide services to

provide services to

Services

Govt. Publishing Legal Comp. 12 KMA

•  Navigable legal portals •  Semantic metadata for legal texts •  Rule-based laws (eventually …)

•  Content management (document storage, !classification, search)

•  Content analytics (e.g., case analytics) •  Predictive assistance (e.g., contract analysis) •  Simulation support (e.g., for the insurance domain)

•  Electronic and printed legal material •  Legal interpretation and insights (e.g., based on case law) •  Monitoring and dissemination of changes in the law

•  Audits and legal advisory •  Identifying artifacts that need to be reviewed for !

compliance after a change in laws and standards •  Automated generation of legally-compliant documents !

(compliance by construction) •  Auto-completion of legal forms

Emerging Applications (1/3)

13

•  Intelligent search (for information relevant to compliance)

The Secret Rules of Modern Living: Algorithms, BBC (2015)

Emerging Applications (2/3)

14

•  Legal metadata extraction

•  Identification of the structure and semantics of legal provisions

•  Continuous compliance monitoring

•  Identification of drifts and deviations from normal practice

•  Legal prediction

•  Litigation risk prediction

•  Court outcome prediction

Emerging Applications (3/3)

15

•  Transparency and due diligence

•  Electronic capture of legal !interpretations and compliance!evidence

•  Automated support for change impact analysis

•  The way changes affect other legal documents

•  The way changes affect the satisfaction of business goals and policies

Size

Establishment

erasePersonalData()

Controller Processor

Subject

Data Protection Officer (DPO)

designates ▶︎

From the Trenches:!Project Experience

16

Context

17

• Collaborative research projects with:

• SCL: Central Legislation Department

• ACD: Tax Authority

• CTIE: Government’s IT Centre

Acknowledgements

18

SnT

•  Lionel Briand

•  Nicolas Sannier

•  Ghanem Soltana

•  Morayo Adedjouma

Govt. of Luxembourg

•  John Dann

•  Thierry Prommenschenkel

•  Ludwig Balmer

•  Marc Blau

•  Anne-Catherine Ries

•  Jean-Paul Zens

Project 1!Metadata Extraction (SCL)

• Lack of human resources for annotating the plethora of existing legal texts with metadata

• Metadata is important prerequisite for machine analyzability

• Problem applies to virtually any government and jurisdiction

• Existing solutions are almost entirely manual

• … and further, the results are imprecise

19

Comparison

Cross reference not !handled systematically

Shallow metadata

Extremely tedious Requires expensive resources

Without AI With AI

Detailed, accurate metadata

Reduced manual effort Cost-effective use of expertise

Adherence to standardized !metadata schemas

20

Our Solution: ARMLET • Toolbox for legal metadata extraction

21

CustomisationInput

NLPEngine

TemplateNLP Scripts

Legal Corpus(non-markup)

MDEEngine

Corpus-tailored NLP Scripts(automatically-generated)

Legal Corpus Enhanced with Metadata and Represented

in Markup Format

XMLXML

XMLXML

Application to Luxembourg Codes

22

High-leveldivisions

Art.

Sub-ar6cledivisions

CRs

Total

Part Book Title Chap. Sec. Subsec. Par. Alinea L/N/D

CivilCode 3 3 36 131 154 43 2316 33 3474 361 997 7551

CommercialCode 0 4 21 14 12 0 261 14 489 85 232 1132

PenalCode 0 2 11 86 46 0 671 64 1094 471 912 3357

CodeofCriminalProcedure 0 3 15 33 36 7 529 542 1315 325 1065 3870

NewCodeforCivilProcedure 2 11 90 19 42 30 1322 206 2316 342 821 5201

Total 5 23 173 283 290 80 5099 859 8688 1584 4027 21111

Results for Structural Metadata

• >20,000 structural markup elements

•  Including >5000 articles and >4000 cross references

• ~91% of generated markup is fully correct

• ~8% of markup needs tweaks (often minor)

• ~1% of markup needs to be manually inserted

23

Next Steps

• Ongoing research with SCL on semantic metadata

• Conditions, consequences, modalities (rights, permissions, obligations), interpretation, case law

• Turning ARMLET into a mature product

24

Project 2!Policy Simulation (ACD, CTIE)

Simulation data /Test cases

Software system

Specification of legal policies Generate

(SyntheticLuxembourg)

1. Legal compliance

2. Change impact Simulate

0%

2%

4%

6%

8%

10%

12%

0%

5%

10%

15%

20%

25%

0-10

.000

10.0

00-2

0.00

0

20.0

00-3

0.00

0

30.0

00-4

0.00

0

40.0

00-5

0.00

0

50.0

00-6

0.00

0

60.0

00-7

0.00

0

70.0

00-8

0.00

0

80.0

00-9

0.00

0

90.0

00-1

00.0

00

100.

000-

110.

000

110.

000-

120.

000

120.

000-

130.

000

130.

000-

140.

000

140.

000-

150.

000

150.

000-

160.

000

160.

000-

170.

000

170.

000-

180.

000

180.

000-

190.

000

190.

000-

200.

000

200.

000-

250.

000

250.

000-

350.

000

350.

000-

500.

000

500.

000-

700.

000

700.

000-

1.00

0.00

0

>1.0

00.0

00

Gross annual income (in Euros)

Con

trib

utio

n to

rev

enue

Hou

seho

lds

Percentage of households

Percentage of contribution before change

Percentage of contribution after change

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

0

1-3.

000

3.00

1-6.

000

6.00

1-9.

000

9.00

1-12

.000

12.0

01-1

5.00

0

15.0

01-1

8.00

0

18.0

01-2

1.00

0

21.0

01-2

4.00

0

24.0

01-2

7.00

0

27.0

01-3

0.00

0

>30.

000

Annual income taxes due (in Euros)

Hou

seho

lds

Before change

After change

Policy Models

ExecutableSimulator

Simulator Code Generator

Historical Data

Statistical Guidance about the Real Simulation Population

Census / Survey Data

ExpertEstimates

Data Generator

SyntheticPopulation

(if a

vaila

ble)

SimulationResults

Domain Model

Approach

26

Example Policy Model

27

Domain Model for Personal Income Taxes (Withholding)

28

Dependent Type «enumeration»

annual_AEPDeduction

amount start_year end_year number_of_months

Allowance

relation_to_taxpayerDependent

income_type amount start_year end_year is_contributing_to_CNS is_contributing_to_Pension is_assisted_by_spouse

Income

union_status start_year end_year is_separated separation_cause is_living_apart

Household

is_widower widower_since residence_status

Taxpayer

Income Type - EMPLOYMENT- PENSION- AGRICULTURE_AND_FORESTRY- TRADE_AND_BUSINESS- CAPITAL_AND_INVESTMENTS

«enumeration»

- CHILD- RELATIVE- SPOUSE- CHILD_OUTSIDE_HOUSEHOLD- PARENT- OTHER

Legal Union Type- MARRIAGE- PARTNERSHIP

«enumeration»

Residence Type- RESIDENT- NON_RESIDENT

«enumeration»

Separation Cause- MUTUAL_AGREEMENT- BY_COURT- DE_FACTO (DE_FAIT)- UNSPECIFIED

«enumeration»

birth_year country_of_residence

Physical Person

Modeling Population Characteristics

29

- «from histogram» birthYear: Integer [1]

TaxPayer

Source: STATEC, Luxembourg

Fidelity of Policy Models and Synthetic Population

30

•  Ran the simulator over 10 synthetic populations

0%

2%

4%

6%

8%

10%

12%

0-10

K10

K-2

0K20

K-3

0K30

K-4

0K40

K-5

0K50

K-6

0K60

K-7

0K70

K-8

0K80

K-9

0K90

K-1

00K

100K

-110

K11

0K-1

20K

120K

-130

K13

0K-1

40K

140K

-150

K15

0K-1

60K

160K

-170

K17

0K-1

80K

180K

-190

K19

0K-2

00K

200K

-250

K25

0K-3

50K

350K

-500

K50

0K-7

00K

700K

-1M

> 1M

Brackets of gross annual income (in euro)

Con

trib

utio

n to

rev

enue Real %

Results of S1Results of S2Results of S3Results of S4Results of S5Results of S6Results of S7Results of S8Results of S9Results of S10

•  Results are closely in line with available tax statistics

Change Impact Analysis

31

• Used the simulator to predict the impact of a proposed policy change: abolishment of joint taxation

•  Impact on distribution of tax classes

•  Impact on household taxes

•  Impact on Government revenue

• Detailed report (targeted at legal !professionals) available

Next Steps

• Applying the simulation approach in other domains

• Social security, healthcare, corporate taxes

• Developing more advanced AI functions

• Example: Prediction of undesirable situations such as loop holes and violation of operating principles

32

Concluding Remarks

33

Outlook

34

•  Role of AI in law is to provide analytical assistance and cut mundane work in regulatory compliance activities

•  Reduction of effort will allow IT and legal professionals to concentrate their scarce time on what matters the most

•  As far as regulatory compliance goes, the function of IT and legal professionals is unlikely to be disruptively changed anytime soon, BUT

•  The way these professionals access and use legal knowledge is already undergoing disruptive change

Keeping Up to Date

35

Examples:

•  Thomson Reuters Digests

•  Stanford’s CODEX!(http://codex.stanford.edu)

•  Computational Legal Studies Blog!(https://computationallegalstudies.com)

Thank You!

36

Questions?