Privacy and Contextual Integrity: Framework and Applications
description
Transcript of Privacy and Contextual Integrity: Framework and Applications
![Page 1: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/1.jpg)
Anupam DattaAnupam DattaCMUCMU
Joint work with Adam Barth, John Mitchell Joint work with Adam Barth, John Mitchell (Stanford), Helen Nissenbaum (NYU) and Sharada (Stanford), Helen Nissenbaum (NYU) and Sharada Sundaram (TCS)Sundaram (TCS)
Privacy and Contextual Integrity:Framework and Applications
![Page 2: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/2.jpg)
2
Problem Statement
Is an organization’s business process compliant with privacy regulations and internal policies?
Examples of organizations– Hospitals, financial institutions, other enterprises handling sensitive
information Examples of privacy regulations
– HIPAA, GLBA, COPPA, SB1386
Goal: Develop methods and tools to answer this question
![Page 3: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/3.jpg)
3
Privacy Project Space
What is Privacy?[Philosophy, Law, Public Policy]
Formal Model, Policy Language,Compliance-check Algorithms
[Programming Languages, Logic]
Implementation-level Compliance[Software Engg, Formal Methods]
Data Privacy[Databases, Cryptography]
![Page 4: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/4.jpg)
4
Project Overview
What is privacy?– Conceptual framework
Policy language – Privacy laws including HIPAA, COPPA, GLBA expressible
Compliance-check algorithms– Does system satisfy privacy and utility goals?
Case studies– Patient portal deployed at Vanderbilt Hospital– UPMC (ongoing discussions)– TCS
![Page 5: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/5.jpg)
5
Contextual Integrity [N2004]
Philosophical framework for privacy Central concept: Context
– Examples: Healthcare, banking, education What is a context?
– Set of interacting agents in roles Roles in healthcare: doctor, patient, …
– Norms of transmission Doctors should share patient health information as per the HIPAA rules
– Purpose Improve health
![Page 6: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/6.jpg)
6
Nurse
Secretary
MyHealth@Vanderbilt Workflow
Patient
Doctor
Health Answer
Health AnswerHealth Question
Appointment Request
Heal
th Q
uest
ion
Health Question
Now that I have cancer,Should I eat more vegetables?
Yes! except broccoli
Privacy: HIPAA compliance+
Humans + Electronic system
Utility: Schedule appointments, obtain health answers
![Page 7: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/7.jpg)
7
Nurse
Secretary
MyHealth@Vanderbilt Improved
Patient
Doctor
Health Answer
Health Answer
Health Question
Appointment Request
Health Question
Now that I have cancer,Should I eat more vegetables?
HealthQuestion
Yes! except broccoli
HealthAnswer
•Message tags used for policy enforcement•Minimal disclosure
Responsibility: Doctor should answer health questions
![Page 8: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/8.jpg)
8
Privacy vs. Utility
Privacy– Certain information should not
be communicated Utility
– Certain information should be communicated
Tension between privacy and utility
Permissiveness
WorkflowsViolatePrivacy
ViolateUtility
FeasibleWorkflows“Minimum
necessary”
![Page 9: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/9.jpg)
9
Design-time Analysis: Big Picture
ContextualIntegrity
Business Objectives Privacy Policy
Business ProcessDesign
PrivacyChecker
(LTL)
UtilityChecker(ATL*)
UtilityEvaluation
PrivacyEvaluation
NormsPurpose
Assuming agents responsible
![Page 10: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/10.jpg)
10
Auditing: Big Picture
Business ProcessExecution
AuditLogs
Run-time Monitor
Privacy PoliciesUtility Goals
AuditAlgos
Policy Violation+
Accountable Agent
Agents may not be responsible
![Page 11: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/11.jpg)
11
In more detail…
Model and logicPrivacy policy examples
– GLBA – financial institutions– MyHealth portal
Compliance checking– Design time analysis (fully automated)– Auditing (using oracle)
Language can express HIPAA, GLBA, COPPA [BDMN2006]
![Page 12: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/12.jpg)
12
Model
Alice
– Communication via send actions: Sender: Bob in role Patient Recipient: Alice in role Nurse Subject of message: Bob Tag: Health Question Message: Now that ….
– Data model & knowledge evolution: Agents acquire knowledge by:
– receiving messages – deriving additional attributes based on data model
Health Question Protected Health Information
BobNow that I have cancer,
Should I eat more vegetables?
HealthQuestion
contents(msg) vs. tags (msg)
Inspired by Contextual Integrity
![Page 13: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/13.jpg)
13
Model
State determined by knowledge of each agent
Transitions change state– Set of concurrent send actions– Send(p,q,m) possible only if
agent p knows m
K0
K13K11
......
K12
A11
A12
A13
Concurrent Game Structure
G = <k, Q, , , d, >
[BDMN06, BDMS07]
![Page 14: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/14.jpg)
14
Logic of Privacy and Utility
Syntax ::= send(p1,p2,m) p1 sends p2 message m
| contains(m, q, t) m contains attrib t about q | tagged(m, q, t) m tagged attrib t about q | inrole(p, r) p is active in role r | t t’ Attrib t is part of attrib t’ | | | x. Classical operators | U | S | O Temporal operators
| <<p>> Strategy quantifier Semantics
Formulas interpreted over concurrent game structure
![Page 15: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/15.jpg)
15
Specifying Privacy
MyHealth@Vanderbilt
In all states, only nurses and doctors receive health questions
G p1, p2, q, m send(p1, p2, m) contains(m, q, health-question)
inrole(p2, nurse) inrole(p2, doctor)
![Page 16: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/16.jpg)
16
Specifying Utility
MyHealth@Vanderbilt
Patients have a strategy to get their health questions answered
p inrole(p, patient) <<p>> F q, m. send(q, p, m) contains(m, p, health-answer)
![Page 17: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/17.jpg)
17
MyHealth Responsibilities
Tagging Nurses should tag health questions
G p, q, s, m. inrole(p, nurse) send(p, q, m) contains(m, s, health-question)
tagged(m, s, health-question) Progress
– Doctors should answer health questionsG p, q, s, m. inrole(p, doctor) send(q, p, m)
contains(m, s, health-question) F m’. send(p, s, m’) contains(m’, s, health-answer)
![Page 18: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/18.jpg)
18
Sender role Subject roleAttribute
Temporal condition
Gramm-Leach-Bliley Example
Recipient role
Financial institutions must notify consumers if they share their non-public personal information with non-affiliated companies, but the notification may occur either before or after the information sharing occurs
![Page 19: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/19.jpg)
19
Workflow Design Results
Theorems:Assuming all agents act responsibly, checking whether workflow
achieves – Privacy is in PSPACE (in the size of the formula describing the
workflow) Use LTL model-checking algorithm
– Utility is decidable for a restricted class of formulas ATL* model-checking is undecidable for concurrent game structures with
imperfect information, but decidable with perfect information
Idea: – Check that all executions satisfy privacy and utility properties
Definition and construction of minimal disclosure workflow
Algorithms implemented in model-checkers, e.g. SPIN, MOCHA
![Page 20: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/20.jpg)
20
Auditing Results
Who to blame? Accountability– Irresponsibility + causality
Design of audit log– Use Lamport causality structure, standard concept from distributed
computing Algorithms
– Finding agents accountable for policy violation in graph-based workflows using audit log
– Finding agents who act irresponsibly using audit log Algorithms use oracle:
– O(msg) = contents(msg)– Minimize number of oracle calls
![Page 21: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/21.jpg)
21
Conclusions
Framework inspired by contextual integrity Business Process as Workflow
Role-based responsibility for human and mechanical agents Compliance checking
Workflow design assuming agents responsible Privacy, utility decidable (model-checking) Minimal disclosure workflow constructible
Auditing logs when agents irresponsible From policy violation to accountable agents Finding irresponsible agents
Case studies– MyHealth patient portal deployed at Vanderbilt University hospital– Ongoing interactions with UPMC
Using oracle
Automated
![Page 22: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/22.jpg)
22
Future Work
Framework– Do we have the right concepts? – Adding time , finer-grained data model– Priorities of rules, inconsistency, paraconsistency
Compliance vs. risk management Privacy principles
– Minimum necessary one example; what else? Improve algorithmic results
– Utility decidability; small model theorem– Auditing algorithms
Privacy analysis of code– Current results apply to system specification
More case studies– Focusing on healthcare
Detailed specification of privacy laws– Immediate focus on HIPAA, GLBA, COPPA
Legal, economic incentives for responsible behavior
![Page 23: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/23.jpg)
23
Publications/Credits
– A. Barth, A. Datta, J. C. Mitchell, S. Sundaram Privacy and Utility in Business Processes, to appear in Proceedings of
20th IEEE Computer Security Foundations Symposium, July 2007.
– A. Barth, A. Datta, J. C. Mitchell, H. Nissenbaum Privacy and Contextual Integrity: Framework and Applications, in
Proceedings of 27th IEEE Symposium on Security and Privacy , pp. 184-198, May 2006.
Work covered in The Economist, IEEE Security &
Privacy editorial
![Page 24: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/24.jpg)
24
Thanks
Questions?
![Page 25: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/25.jpg)
25
Additional Technical Slides
![Page 26: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/26.jpg)
26
Related Languages
Model Sender Recipient Subject Attributes Past Future Combination
RBAC Role Identity
XACML Flexible Flexible Flexible o o
EPAL Fixed Role Fixed o
P3P Fixed Role Fixed o o
LPU Role Role Role
Legend: unsupportedo partially supported fully supported
LPU fully supports attributes, combination, temporal conditions
Utility not considered
![Page 27: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/27.jpg)
27
Deciding Utility
ATL* model-checking of concurrent game structures is – Decidable with perfect information– Undecidable with imperfect information
Theorem:There is a sound decision procedure for deciding whether workflow
achieves utility Intuition:
– Translate imperfect information into perfect information by considering all possible actions from one player’s point of view
![Page 28: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/28.jpg)
28
Local communication game
Quotient structure under invisible actions, Gp– States:
Smallest equivalence relation K1 ~p K2 if K1 K2 and a is invisible to p
– Actions:[K] [K’] if there exists K1 in [K] and K2 in [K’] s.t. K1 K2
Lemma: For all LTL formulas visible to p, Gp |= <<p>> implies G |= <<p>>
![Page 29: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/29.jpg)
29
Auditing Results
Definitions– Policy compliance, locally compliant– Causality, accountability
Design of audit log Algorithms
– Finding agents accountable for locally-compliant policy violation in graph-based workflows using audit log
– Finding agents who act irresponsibly using audit log Algorithms use oracle:
– O(msg) = contents(msg)– Minimize number of oracle calls
![Page 30: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/30.jpg)
30
Policy compliance/violation
Strong compliance [BDMN2006]– Action does not violate current policy requirements– Future policy requirements after action can be met
Locally compliant policy– Agents can determine strong compliance based on their local view of history
Policy
History
Contemplated ActionJudgment
Future Reqs
![Page 31: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/31.jpg)
31
Causality
Lamport Causality [1978]
“happened-before”
![Page 32: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/32.jpg)
32
Accountability & Audit Log
Accountability– Causality + Irresponsibility
Audit log design– Records all Send(p,q,m) and Receive(p,q,m) events executed– Maintains causality structure
O(1) operation per event logged
![Page 33: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/33.jpg)
33
Auditing Algorithm
GoalFind agents accountable for a policy violation
Algorithm(Audit log A, Violation v)1. Construct G, the causality graph for v in A2. Run BFS on G.
At each Send(p, q, m) node, check if tags(m) = O(m). If not, and p missed a tag, output p as accountable
Theorem: – The algorithm outputs at least one accountable agent for every
violation of a locally compliant policy in an audit log of a graph-based workflow that achieves the policy in the responsible
model
![Page 34: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/34.jpg)
34
Proof Idea
Causality graph G includes all accountable agents– Accountability = Causality + Irresponsibility
There is at least one irresponsible agent in G– Policy is satisfied if all agents responsible – Policy is locally compliant
In graph-based workflows, safety responsibilities violated only by mistagging
– O(msg) = tags(msg) check identifies all irresponsible actions
![Page 35: Privacy and Contextual Integrity: Framework and Applications](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816096550346895dcfbde5/html5/thumbnails/35.jpg)
35
MyHealth Example
1. Policy violation: Secretary Candy receives health-question mistagged as
appointment-request2. Construct causality graph G and search backwards using BFS
Candy received message m from Patient Jorge. O(m) = health-question, but tags(m) = appointment-request. Patient responsible for health-question tag. Jorge identified as accountable