Using Safety-Critical Concepts in Privacy Engineering
-
Upload
ian-oliver -
Category
Travel
-
view
311 -
download
0
Transcript of Using Safety-Critical Concepts in Privacy Engineering
1 © Nokia 2016
Using Safety-Critical Concepts in Privacy Engineering
Public
Dr. Ian Oliver
Bell Labs, Finland
2 November 2016
A Lecture Given to CRiM’16, Oulu, Finland
2 © Nokia 2016
Public
Auditing mobile services and associated infrastructure from an engineering perspective...
3 © Nokia 2016
Public
Auditing mobile services and associated infrastructure from an engineering perspective...
PrivacyOfficers & Lawyers
Privacy Engineers
4 © Nokia 2016
Public
The problem was (and still is, and will be) the lack of good (any?) techniques for reasoning about privacy in an engineering capacity
5 © Nokia 2016
So what is this privacy thing anyway?
Public
• vs Security...
• [[privacy]]
• Information wants to be free!• Freedom to do/be...?• Freedom from...?• Price, value
•Ownership
• You are the product
• ”Anti-privacy”
• Advertising, surveillance, hacking, oversharing
• Personal responsibility vs Technilogical complexity
• PII, personal data, pseudo-anonymous, anonymisation
•variability, entropy, Navier-Stokes
• f(p1...pn) -> R
•The Privacy Singularity•Unification of disciplines•Mathematical Foundations of Privacy•The Fundamental Theorem of Privacy
6 © Nokia 2016
Privacy as...
Public
• A legal construct •“The Right to Privacy” (Warren and Brandeis, 1890)• EU Data Protection Laws• Human Rights•...
7 © Nokia 2016
Privacy as...
Public
• A legal construct• A philisophical construct
• morals, ethics etc• political science? Kant etc.
8 © Nokia 2016
Privacy as...
Public
• A legal construct• A philisophical construct• An economic construct
• brand/shareholder value• customer relatonships• business• innovation
9 © Nokia 2016
Privacy as...
Public
• A legal construct• A philisophical construct• An economic construct• A guiding principle
10 © Nokia 2016
Privacy as...
Public
• A legal construct• A philisophical construct• An economic construct• A guiding principle• A sociological construct
11 © Nokia 2016
Privacy as...
Public
• A legal construct• A philisophical construct• An economic construct• A guiding principle• A sociological construct• A game theoretic construct
12 © Nokia 2016
Privacy as...
Public
• A legal construct• A philisophical construct• An economic construct• A guiding principle• A sociological construct• A game theoretic construct• A systems engineering construct
char collectDataFlag = 'Y'; // Future proofed boolean// Y for yes, N for no
void collectDataFunction(){//collect IMEI, IMSI, MSISDN, TimeStamp and location//and send to the hardcoded IP address...
}
void checkDataCollection(){switch(collectDataFlag){
case 'N' :// don't do anything
case 'Y' :// ok to collect everythingcollectDataFunction();
}}
compliance
13 © Nokia 2016
Privacy as...
Public
• A legal construct• A philisophical construct• An economic construct• A guiding principle• A sociological construct• A game theoretic construct• A systems engineering construct (pt.2)
• Ontological structures• Metrics / Risk Analysis• Modelling• Privacy Engineering• Compliance• Culture & Safety Critical
Systems• (Aviation, Medicine)
14 © Nokia 2016
Privacy as...
Public
• A legal construct• A philisophical construct• An economic construct• A guiding principle• A sociological construct• A game theoretic construct• A systems engineering construct• An optimisation construct
15 © Nokia 2016
Privacy as...
Public
• A legal construct• A philisophical construct• An economic construct• A guiding principle• A sociological construct• A game theoretic construct• A systems engineering construct• An optimisation construct• A mathematical construct
𝑡0
𝑡1
𝐷1 ×⋯ × 𝐷𝑛 < 𝜀𝑈
• metrics• topology• ontology• anonymisation & variability• turbulence, chaos theory• link back to economics & game theory• deanonymisation• information entropy
16 © Nokia 2016
Public
The problem was (and still is, and will be) the lack of good (any?) techniques for reasoning about privacy in an engineering capacity...
...so what did we do?
17 © Nokia 2016
We developed:
• Epics and Use cases for Privacy
• Checklists
• Software Development Process Integration
• Audit Procedures (non-functional aspects)
- privacy
- secuity
- performance
- continuity (resiliance)
and the result was...
18 © Nokia 2016
Failure
19 © Nokia 2016
Why didn’t it work?
• Despite highly trained personel
- Cessna Single Engine Failure
• FLY THE AIRCRAFT
- Air France AF447
• To much adherence to process
- Processes tell everyone the order of what to do
- Difficulty in handling exceptions and experts
- Aviation Checklists are status checks used to assist in due dilligence in preparation for the next and future phases of flight.
- Engineers aren’t stupid
• Checklist replaced responsibility and expertise
- For both the auditor and develoment teams
• Tick-box oriented
- Ask questions, Accept answers, TICK!
- Limited understanding and context of naswers
• Limited time-scale
- One-off review
20 © Nokia 2016
We developed:
• Simpler ”Checklists”
• Training Courses
• Realised that no-one understood each other
• Tried to ban (unsuccessfully) the term ”PII”
• Tried to formulate requirements
• Introduced more risk management ideas, eg: RCA, FMEA
and the result was...
21 © Nokia 2016
Failure2
22 © Nokia 2016
What’s the problem now?
• Communication
• Emphasis on process over method
• Lack of understanding of role
• Lack of legal and engineering techniques
• Lack of integration of legal and engineering
• The privacy organisation itself
• Humans
23 © Nokia 2016
What’s the problem now?
Actually it was much worse:
24 © Nokia 2016
What’s the problem now?
Actually it was much worse:
Total emphasis on ”compliance”
25 © Nokia 2016
What’s the problem now?
Actually it was much worse:
Total emphasis on ”compliance”
Whatever ”compliance” meant...
26 © Nokia 2016
Compliance
is fragile
Public
char collectDataFlag = 'Y'; // Future proofed boolean// Y for yes, N for no
void collectDataFunction(){//collect IMEI, IMSI, MSISDN, TimeStamp and location//and send to the hardcoded IP address...
}
void checkDataCollection(){switch(collectDataFlag){
case 'N' :// don't do anything
case 'Y' :// ok to collect everythingcollectDataFunction();
}}
27 © Nokia 2016
????!!!
28 © Nokia 2016
Help!
Not invented here!
Were there any industries or disciplines from which we could learn?
Or are software engineering and legal ’special’?
29 © Nokia 2016
29
Serendipity
© 2013 HERE | Title | Author | Company confidential
30 © Nokia 2016
Help!
Consider information to be a dangerous item
This has various meanings in aviation, medicine, civil engineering etc.
31 © Nokia 2016
31
A quick introduction to surgical infection control
© 2013 HERE | Title | Author | Company confidential
32 © Nokia 2016
32
A quick introduction to surgical infection control
© 2013 HERE | Title | Author | Company confidential
seriously!
33 © Nokia 2016
33
The Sterile Field
© 2013 HERE | Title | Author | Company confidential
Key:
• Sterile
• Non-sterile
34 © Nokia 2016
34
The Sterile Field
© 2013 HERE | Title | Author | Company confidential
Key:
• Sterile
• Non-sterile
Movement of materials from one area to the other must be controlled to prevent contamination of the sterile field with non-sterile items
Strict protocols prevent contamination
35 © Nokia 2016
Some things...
• Communication
• Culture
36 © Nokia 2016
Some things...
• Communication, Structure and Semantics
• Culture
Already solved...in other fields
37 © Nokia 2016
Standardised Communication
Public
38 © Nokia 2016
Standardised Communication
Probably not personal data/ Probably personal data
39 © Nokia 2016
Standardised Communication
Forget process, just get the information about what’s going on...
40 © Nokia 2016
Nokia Internal
41 © Nokia 2016
Nokia Internal
42 © Nokia 2016
Nokia Internal
43 © Nokia 2016
Public
Checklists
44 © Nokia 2016
Public
Checklists
45 © Nokia 2016
Public
Morbidity and MortalityAccident InvestigationReporting
46 © Nokia 2016
Public
Roles and Role Integration
R&D Team Checklist
(before review)
R&D Team Checklist
(post-review)
Audit Team Checklist(sign-in)
Audit Team Checklist
(time-out)
Audit Team Checklist(sign-out)
Project development & processes (time)
System
under
auditPrivacy
Officer
Legal
Security
Architects
47 © Nokia 2016
Public
Experience
DataCollection
CellID->Location
DataStorage
OperatorPrivacy
Preprocessing
Extraction Hashing
FileStorage
RawData
Processing &Enrichment
External Data
ExternalCross-
referencing
Atomic Data
Aggregation/Report
Generation
CustomerReception
ReportStorage
<<data subject>>Customer
48 © Nokia 2016
Public
Conclusions...
49 © Nokia 2016
Public
No heroes
50 © Nokia 2016
Public
Treat privacy as a safety-critical aspect