WebDam 1

provenance analysis of algorithms

10/1/13

V. Tannen University of Pennsylvania

someTowards ?

WebDam 210/1/13

Wish to extend data provenance analysis to (as) general (as possible) information processing.

Much such analysis was (and still is) done in the style of “activity logs”Many people are happy with this style of provenance as long as what is

recorded in each log entry is rich enough information

A new perspective arose in databases with the work of Buneman, Khanna, and Tan. This and subsequent developments allow us to be more ambitious about applications of provenance analysis.

Data provenance analysis

WebDam 3

Provenance

10/1/13

Input(structured) collection of items Output

Query

How does each output item depend on the input items?

Too vague! Look at applications to clarify.

Processing

WebDam 4

Provenance applications: some causal analysis

10/1/13

Consider an output item o .• Deletion propagation: If some input items disappear does o disappear

from the output?

• Binary trust: Is o obtained only from trusted input items?

• Access control: Associate with each input item a clearance procedure.

Would a given clearance procedure succeed for o ?

• Probabilistic analysis: How does the event “o is in the output” depend

on all the events “item blah is in the input”

WebDam 5

Provenance applications: some quantitative analysis

10/1/13

Consider an output item o .

• Cost: Assign a cost value to each input item. Compute the corresponding

cost of o .

• Degrees of trust: Assign a degree of trust to each input item. Compute

the degree of trust that corresponds to o .

WebDam 610/1/13

Provenance analysis

Provenance: an abstract way of describing the relationship between output items and input items.

Critical test: it can be specialized to solve causal or quantitative applications like those we saw.

In all those applications we must keep track of all the alternative ways that an output items is computed, and for each of these, of the joint way in which input items are used.

WebDam 710/1/13

For queries. (But see also longtime work in PL on security based on information flow.)

Why-provenance of Buneman, Khanna, and Tan offered a perspective radically different than that of activity logs: different ways (different witnesses) of obtaining the same output item should be recorded separately.

For causal analysis (deletion propagation, binary trust, access control, probabilistic

analysis) minimal witnesses suffice. This is the same as recording provenance with positive boolean expressions.

For quantitative analysis (cost, degrees of trust) witness (why-) provenance does not suffice (because it does not count how many times an input item is used in a witness). But we have Sorp.

A commutative semiring is absorptive if a+ab=a for all a,b.Sorp(X) is the free absorptive semiring generated by X. It specializes correctly to cost

and degrees of trust. Was designed in [DMST 2013, submitted].

Some things we have learned, so far

WebDam 810/1/13

We understand one particular case: datalog

Would like more, at least while positive queries for instance. But eventually graph algorithms such as Dijkstra, Prim, Kruskal and more?

Problem: provenance can be infinite if we count all alternatives.

Finite version: PosBool(X) for finite X. Good for causal analysis but not for quantitative analysis.

xy ∨ yz ∨ xzu

Sorp(X) for finite X is another finite version that is also good for quantitative analysis.

x2y + xy3 + xyu

Loops and fixpoints

WebDam 910/1/13

The shortest path from A to F is A-C-E-F and has length 60.

What if CE is lost? Then it’s A-B-D-F or A-C-D-F, both length 70.

If CE, BD, and DF are lost then it’s A-B-D-E-F, length 80.

Parts of input lost? Sounds like provenance analysis!

A D

B

E

F

C

20 20

10 30 30

40

30

10

1010/1/13 WebDam

Label edges with provenance tokens x, y, z, u, v, w, s, t.

Positive boolean provenance suffices for this application.

Similar to provenance for aggregation [ADT PODS11] computean expression for the length of the shortest path in involving numbers annotated with provenance.

E.g., (xy∨yu) 50

A D

B

E

F

C

x

y

z

u v

w

s

t

20 20

10 30 30

40

30

10

WebDam 1110/1/13

Y

X2

X1

x2

x1

a2

a1

EP(Y) = (EP(X1) ∧ x1) ∨ (EP(X2) ∧ x2)

LSP(Y) = [(x1 LSP(X1)) + (EP(X1) ∧ x1) a1] min [(x2 LSP(X2)) + (EP(X2) ∧ x2) a2]

EP(X): provenance of“there exists a path from A to X”

LSP(X): length of shortest path from A to X provenance-annotation expression

WebDam 1210/1/13

EP(D) = xz ∨yu

LSP(D) = = [u(y 10) + yu 30] min [z(x 20) + xz 20] =

= yu 40 min xz 40 =

= xz yu ∨ 40

A D

B

E

F

C

x

y

z

u v

w

s

t

20 20

10 30 30

40

30

10

WebDam 1310/1/13

EP(F) = xzw yuw yst xzvt yuvt ∨ ∨ ∨ ∨

LSP(F) = = [yuw xzw∨ 70] min [ [yst 50 min (yuvt xzvt)∨ 70] + yst yuvt ∨ 10 ]

A D

B

E

F

C

x

y

z

u v

w

s

t

20 20

10 30 30

40

30

10

WebDam 1410/1/13

Handling large aggregations: algebraic laws help in shrinking size.

Abstraction: group them by categories, track only the categories

Provisioning [DIMT CIDR13]. Seemingly causal analysis only. Scenarios correspond to input items!

Too many input items to track!

WebDam 1510/1/13

y = u = v = t = true

LSP(F) = = w 70 min [ [ s 50 min true 70 ] + true 10 ]

A D

B

E

F

C

x

y

z

u v

w

s

t

20 20

10 30 30

40

30

10

WebDam 16

Thank you!

10/1/13