Boolean Searchable Symmetric Encryption with …...•from Matryoshka filters (new Bloom filter data...
Transcript of Boolean Searchable Symmetric Encryption with …...•from Matryoshka filters (new Bloom filter data...
Boolean Searchable Symmetric Encryption with Worst-Case Sub-Linear Complexity
Seny Kamara Tarik Moataz
2
Bob
2
Bob
2
Bob
I can’t search!
Many Approaches
• Stream ciphers [SWP00]
• Bucketing [HILM02]
• Structured and searchable encryption (STE/SSE) [CGKO06,CK10]
• Oblivious RAM (ORAM) [GO96]
• Functional encryption (e.g., PEKS) [BCOP06]
• Multi-party computation (MPC)
• Property-preserving encryption (PPE) [AKSX04,BBO06,BCLO09]
• Fully-homomorphic encryption [G09]
3
Efficiency
Security Expressiveness
4
Expressiveness
Efficiency
OXT
Blind Seer BOXT
Searchable Symmetric Encryption
5
RR Naïve RH Naive
Boolean
SNF
Expressiveness
Efficiency
OXT
Blind Seer BOXT
Searchable Symmetric Encryption
5
RR Naïve RH Naive This Work
Boolean
SNF
Related Work
• OXT [CJJKRS’13]
• Sub-linear for conjunctive queries
• Linear for disjunctive
• Linear for (arbitrary) Boolean queries
• Non-interactive
• Blind Seer [PKVKMCGKB’14]
• Sub-linear for arbitrary Boolean queries
• Interactive
• Logarithmic multiplicative overhead over the result set
6
Related Work
• OXT [CJJKRS’13]
• Sub-linear for conjunctive queries
• Linear for disjunctive
• Linear for (arbitrary) Boolean queries
• Non-interactive
• Blind Seer [PKVKMCGKB’14]
• Sub-linear for arbitrary Boolean queries
• Interactive
• Logarithmic multiplicative overhead over the result set
6
Related Work
• OXT [CJJKRS’13]
• Sub-linear for conjunctive queries
• Linear for disjunctive
• Linear for (arbitrary) Boolean queries
• Non-interactive
• Blind Seer [PKVKMCGKB’14]
• Sub-linear for arbitrary Boolean queries
• Interactive
• Logarithmic multiplicative overhead over the result set
6
Black-Box Constructions
• IEX: “purely” disjunctive SSE • from any single-keyword SSE
7
Black-Box Constructions
• IEX: “purely” disjunctive SSE • from any single-keyword SSE
• BIEX: Boolean SSE • from IEX
7
Black-Box Constructions
• IEX: “purely” disjunctive SSE • from any single-keyword SSE
• BIEX: Boolean SSE • from IEX
• DIEX: dynamic disjunctive SSE • from any dynamic single-keyword SSE
• Forward Secure
7
Concrete Constructions
• IEX-2Lev • from 2Lev [CJJJKRS14]
8
Concrete Constructions
• IEX-2Lev • from 2Lev [CJJJKRS14]
• BIEX-2Lev • from IEX-2Lev
8
Concrete Constructions
• IEX-2Lev • from 2Lev [CJJJKRS14]
• BIEX-2Lev • from IEX-2Lev
• ZMF: new single-keyword SSE • from Matryoshka filters (new Bloom filter data structure)
• Linear search complexity but very compact
8
Concrete Constructions
• IEX-2Lev • from 2Lev [CJJJKRS14]
• BIEX-2Lev • from IEX-2Lev
• ZMF: new single-keyword SSE • from Matryoshka filters (new Bloom filter data structure)
• Linear search complexity but very compact
• IEX-ZMF • from ZMF
8
Background: Data Structures
9
Background: Data Structures
• Dictionaries map labels to values
• Get: DX[w3] returns id2
9
w1
w2
w3
id1
id3
id2
Dictionary DX
Background: Data Structures
• Dictionaries map labels to values
• Get: DX[w3] returns id2
• Multi-maps map labels to tuples
• Get: MM[w3] returns (id2 , id4)
9
w1
w2
w3
id1
id3
id2
Dictionary DX
w1
w2
w3
id1 id3 id4
id3
id2 id4
Multi-map MM
Background: Encrypted Data Structures [CK’10]
10
w1
l2
w3
id1 id3 id4
id3
id2 id4
Multi-map MM
Setup 1k, , w2
Background: Encrypted Data Structures [CK’10]
10
w1
l2
w3
id1 id3 id4
id3
id2 id4
Multi-map MM w2
w1
id3
id3
Encrypted Multi-map EMM
w3
w1
id2
id4
w3 id4
w1 id1
Setup 1k, , w2
Background: Encrypted Data Structures [CK’10]
11
Token , w1
Background: Encrypted Data Structures [CK’10]
11
Token , w1 w1
Background: Encrypted Data Structures [CK’10]
12
Get , w1
w2
w1
id3
id3
Encrypted Multi-map EMM
w3
w1
id2
id4
w3 id4
w1 id1
Background: Encrypted Data Structures [CK’10]
12
Get , w1 id3 id4 id1
Response-hiding
w2
w1
id3
id3
Encrypted Multi-map EMM
w3
w1
id2
id4
w3 id4
w1 id1
Background: Encrypted Data Structures [CK’10]
13
Encrypted Multi-Map
Background: Encrypted Data Structures [CK’10]
13
Encrypted Multi-Map
Encrypted Inverted
Index
Background: Encrypted Data Structures [CK’10]
13
Single Keyword SSE
[SWP’00], [Goh’03], [CGKO’06], [CK10], [KPR’12], [KP’13], [CJJKRS’13], [CJJJKRS’14],
[Bost’16] …
Encrypted Multi-Map
Encrypted Inverted
Index
Adaptive Security
14
Adaptive Security
14
Real
Multi-map MM
Adaptive Security
14
Real
Multi-map MM
Encrypted Multi-map EMM
Adaptive Security
14
Real
Multi-map MM
Encrypted Multi-map EMM
wi
wi
Adaptive Security
14
Real
Multiple Time
Multi-map MM
Encrypted Multi-map EMM
wi
wi
Multi-map MM
Adaptive Security
14
Real Ideal
Multiple Time
Setup Leakage ℒ𝑆
Multi-map MM
Encrypted Multi-map EMM
wi
wi
Multi-map MM
Adaptive Security
14
Real Ideal
Multiple Time
Setup Leakage ℒ𝑆
Multi-map MM
Encrypted Multi-map EMM
wi
Encrypted Multi-map EMM Encrypted Multi-map EMM
wi
Multi-map MM
Adaptive Security
14
Real Ideal
Multiple Time
Setup Leakage ℒ𝑆
Query Leakage ℒ𝑄
Multi-map MM
Encrypted Multi-map EMM
wi wi
Encrypted Multi-map EMM Encrypted Multi-map EMM
wi
Multi-map MM
Adaptive Security
14
Real Ideal
Multiple Time
Setup Leakage ℒ𝑆
Query Leakage ℒ𝑄
Multi-map MM
Encrypted Multi-map EMM
wi wi
Encrypted Multi-map EMM Encrypted Multi-map EMM
wi wi wi
Multi-map MM
Adaptive Security
14
Real Ideal
Multiple Time
Setup Leakage ℒ𝑆
Query Leakage ℒ𝑄
Real ≈ Ideal
Multi-map MM
Encrypted Multi-map EMM
wi wi
Encrypted Multi-map EMM Encrypted Multi-map EMM
wi wi wi
Overview
• Multi-maps (indexes) can be viewed as collection of sets
15
Overview
• Multi-maps (indexes) can be viewed as collection of sets
• Disjunctive keyword queries can be viewed as set unions on those sets
15
Overview
• Multi-maps (indexes) can be viewed as collection of sets
• Disjunctive keyword queries can be viewed as set unions on those sets
• Naïve set union includes items with multiplicity (redundancy) • Implies sub-optimal communication complexity or heavy leakage
15
Overview
• Multi-maps (indexes) can be viewed as collection of sets
• Disjunctive keyword queries can be viewed as set unions on those sets
• Naïve set union includes items with multiplicity (redundancy) • Implies sub-optimal communication complexity or heavy leakage
• Inclusion/exclusion-based unions remove redundancy • Implies optimal communication complexity and less leakage
15
Overview
• Multi-maps (indexes) can be viewed as collection of sets
• Disjunctive keyword queries can be viewed as set unions on those sets
• Naïve set union includes items with multiplicity (redundancy) • Implies sub-optimal communication complexity or heavy leakage
• Inclusion/exclusion-based unions remove redundancy • Implies optimal communication complexity and less leakage
• New (plaintext) set structure with I/E-based union operations
15
Overview
• Multi-maps (indexes) can be viewed as collection of sets
• Disjunctive keyword queries can be viewed as set unions on those sets
• Naïve set union includes items with multiplicity (redundancy) • Implies sub-optimal communication complexity or heavy leakage
• Inclusion/exclusion-based unions remove redundancy • Implies optimal communication complexity and less leakage
• New (plaintext) set structure with I/E-based union operations
• Encrypted structure that supports I/E-based unions
15
Overview: Multi-Maps as Sets
16
w1
w2
w3
id1 id3 id4
id3
id2 id4
Multi-map MM
Overview: Multi-Maps as Sets
16
w1
w2
w3
id1 id3 id4
id3
id2 id4
Multi-map MM
id1
id3
Id4
Overview: Multi-Maps as Sets
16
w1
w2
w3
id1 id3 id4
id3
id2 id4
Multi-map MM
id1
id3
Id4
id3
Overview: Multi-Maps as Sets
16
w1
w2
w3
id1 id3 id4
id3
id2 id4
Multi-map MM
id1
id3
Id2
Id4
id3
Id4
Overview: Disjunctive Search as Set Union
17
Q = w1 w2 ∨ w3 ∨
Overview: Disjunctive Search as Set Union
17
id1 id3
Id2
Id4
Q = w1 w2 ∨ w3 ∨
Overview: Inclusion/Exclusion-based Union
18
id1 id3
Id2
Id4
Overview: Inclusion/Exclusion-based Union
18
id1 id3
Id2
Id4
Id2
Id4
id1 id3
Id4
id3
Overview: Inclusion/Exclusion-based Union
18
id1 id3
Id2
Id4
Id2
Id4
id3
Overview: Inclusion/Exclusion-based Union
18
id1 id3
Id2
Id4
Id2
Id4
Overview: Inclusion/Exclusion-based Union
18
id1 id3
Id2
Id4
Id2
Id4
𝑤𝑖
𝑛
𝑖=1
= (−1)𝑖+1 # 𝑀𝑀 𝑤𝑗1 ∩⋯∩𝑀𝑀 𝑤𝑗𝑖1≤𝑗1<⋯<𝑗𝑖≤𝑛
𝑛
𝑖=1
#Lookup
Overview: Set Structure with I/E-based Unions
19
id1 id3
Id2
Id4
Overview: Set Structure with I/E-based Unions
19
id1 id3
Id2
Id4
id1 id3
id4
id3
Id2
Id4
Pre-processing
Overview: Set Structure with I/E-based Unions
20
id1 id3
id4
id3
Id2
Id4
Overview: Set Structure with I/E-based Unions
20
id1 id3
id4
id3
Id2
Id4
w1
w2
w3
id1 id3 id4
id3
id2 id4
Global Multi-map MM
Overview: Set Structure with I/E-based Unions
20
id1 id3
id4
id3
Id2
Id4
w1
w2
w3
id1 id3 id4
id3
id2 id4
Global Multi-map MM
Overview: Set Structure with I/E-based Unions
20
id1 id3
id4
id3
Id2
Id4
w1
w2
w3
id1 id3 id4
id3
id2 id4
Global Multi-map MM
w1 ⋀ w2
w1 ⋀ w3
id3
id4
Local Multi-map MM1
w2 ⋀ w1 id3
Local Multi-map MM2
w3 ⋀ w1 id4
Local Multi-map MM3
IEX: Setup
21
w1
w2
w3
id1 id3 id4
id3
id2 id4
Multi-map MM
SetupIEX 1k,
IEX: Setup
21
w1
w2
w3
id1 id3 id4
id3
id2 id4
Multi-map MM
SetupIEX 1k,
w2 E(id3; w1)
Encrypted Global Multi-map EMM
w1 E(id3; l2)
w3 E(id2; w3)
w1 E(id4; w1)
w3 E(id4; w3)
w1 E(id1; w1)
w1 ⋀ w2 E(id3; w1)
Encrypted local Multi-map EMM1
E(id4; w1) w1 ⋀ w3
w2 ⋀ w1 E(id3; w2)
Encrypted local Multi-map EMM1
w3 ⋀ w1 E(id3; w3)
Encrypted local Multi-map EMM2
,
IEX: Setup
22
w1
w2
w3
id1 id3 id4
id3
id2 id4
Multi-map MM
SetupIEX 1k, ,
IEX: Setup
22
w1
w2
w3
id1 id3 id4
id3
id2 id4
Multi-map MM
SetupIEX 1k, ,
w2 E(id3; w1)
Encrypted Global Multi-map EMM
w1 E(id3; l2)
w3 E(id2; w3)
w1 E(id4; w1)
w3 E(id4; w3)
w1 E(id1; w1)
1
2
3
Encrypted Dictionary EDX
w1 ⋀ w2 E(id3; w1)
Encrypted local Multi-map EMM1
E(id4; w1) w1 ⋀ w3
w2 ⋀ w1 E(id3; w2)
Encrypted local Multi-map EMM1
w3 ⋀ w1 E(id3; w3)
Encrypted local Multi-map EMM2
IEX: Token
23
TokenIEX , w1 w3 ∨
IEX: Token
23
TokenIEX , w1 w3 ∨ w1
Global sub-token
IEX: Token
23
TokenIEX , w1 w3 ∨ w1 w3
Global sub-token
Global sub-token
IEX: Token
23
TokenIEX , w1 w3 ∨ w1 w3 1
Global sub-token
Global sub-token
dictionary sub-token
IEX: Token
23
TokenIEX , w1 w3 ∨ w1 w3 1 w1 ⋀ w3
Global sub-token
Global sub-token
dictionary sub-token
Local sub-token
IEX: Get
24
GetIEX , w1 w3 1 w1 ⋀ w3
w2 E(id3; w1)
Encrypted Global Multi-map EMM
w1 E(id3; l2)
w3 E(id2; w3)
w1 E(id4; w1)
w3 E(id4; w3)
w1 E(id1; w1)
1
2
3
Encrypted Dictionary EDX
w1 ⋀ w2 E(id3; w1)
Encrypted local Multi-map EMM1
E(id4; w1) w1 ⋀ w3
w2 ⋀ w1 E(id3; w2)
Encrypted local Multi-map EMM1
w3 ⋀ w1 E(id3; w3)
Encrypted local Multi-map EMM2
IEX: Get
25
Get , w1
w2 E(id3; w1)
Encrypted Global Multi-map EMM
w1 E(id3; l2)
w3 E(id2; w3)
w1 E(id4; w1)
w3 E(id4; w3)
w1 E(id1; w1)
IEX: Get
25
Get , E(id3; w1) E(id4; w1) E(id3; w1) w1
w2 E(id3; w1)
Encrypted Global Multi-map EMM
w1 E(id3; l2)
w3 E(id2; w3)
w1 E(id4; w1)
w3 E(id4; w3)
w1 E(id1; w1)
IEX: Get
25
Get , E(id3; w1) E(id4; w1) E(id3; w1)
Get ,
w1
w2 E(id3; w1)
Encrypted Global Multi-map EMM
w1 E(id3; l2)
w3 E(id2; w3)
w1 E(id4; w1)
w3 E(id4; w3)
w1 E(id1; w1)
w2 E(id3; w1)
Encrypted Global Multi-map EMM
w1 E(id3; l2)
w3 E(id2; w3)
w1 E(id4; w1)
w3 E(id4; w3)
w1 E(id1; w1)
w3
IEX: Get
25
Get , E(id3; w1) E(id4; w1) E(id3; w1)
Get , E(id2; w3) E(id4; w3)
w1
w2 E(id3; w1)
Encrypted Global Multi-map EMM
w1 E(id3; l2)
w3 E(id2; w3)
w1 E(id4; w1)
w3 E(id4; w3)
w1 E(id1; w1)
w2 E(id3; w1)
Encrypted Global Multi-map EMM
w1 E(id3; l2)
w3 E(id2; w3)
w1 E(id4; w1)
w3 E(id4; w3)
w1 E(id1; w1)
w3
IEX: Lookup
26
Get , 1
1
2
3
Encrypted Dictionary EDX
w1 ⋀ w2 E(id3; w1)
Encrypted local Multi-map EMM1
E(id4; w1) w1 ⋀ w3
w2 ⋀ w1 E(id3; w2)
Encrypted local Multi-map EMM1
w3 ⋀ w1 E(id3; w3)
Encrypted local Multi-map EMM2
IEX: Lookup
26
Get , 1
1
2
3
Encrypted Dictionary EDX
w1 ⋀ w2 E(id3; w1)
Encrypted local Multi-map EMM1
E(id4; w1) w1 ⋀ w3
w2 ⋀ w1 E(id3; w2)
Encrypted local Multi-map EMM1
w3 ⋀ w1 E(id3; w3)
Encrypted local Multi-map EMM2
w1 ⋀ w2 E(id3; w1)
Encrypted local Multi-map EMM1
E(id4; w1) w1 ⋀ w3
IEX: Lookup
27
Get , w1 ⋀ w2 E(id3; w1)
Encrypted local Multi-map EMM1
E(id4; w1) w1 ⋀ w3 w1 ⋀ w3
IEX: Lookup
27
Get , E(id4; w1) w1 ⋀ w2 E(id3; w1)
Encrypted local Multi-map EMM1
E(id4; w1) w1 ⋀ w3 w1 ⋀ w3
IEX: Lookup
27
Get , E(id4; w1) w1 ⋀ w2 E(id3; w1)
Encrypted local Multi-map EMM1
E(id4; w1) w1 ⋀ w3 w1 ⋀ w3
E(id3; w1) E(id4; w1) E(id3; w1)
E(id2; w3) E(id4; w3)
IEX: Lookup
27
Get , E(id4; w1) w1 ⋀ w2 E(id3; w1)
Encrypted local Multi-map EMM1
E(id4; w1) w1 ⋀ w3 w1 ⋀ w3
E(id3; w1) E(id4; w1) E(id3; w1)
E(id2; w3) E(id4; w3)
IEX: Lookup
27
Get , E(id4; w1)
Result sent to the client
w1 ⋀ w2 E(id3; w1)
Encrypted local Multi-map EMM1
E(id4; w1) w1 ⋀ w3 w1 ⋀ w3
E(id3; w1) E(id4; w1) E(id3; w1)
E(id2; w3) E(id4; w3)
E(id3; w1) E(id3; w1)
E(id2; w3) E(id4; w3)
IEX: Leakage
• Black-box setup leakage • Setup leakage of global EMM
• Setup leakage of EDX
• Black-box query leakage for disjunction • Query leakage of global EMM
• Query leakage of EDX
28
IEX: Leakage
• Black-box setup leakage • Setup leakage of global EMM
• Setup leakage of EDX
• Black-box query leakage for disjunction • Query leakage of global EMM
• Query leakage of EDX
• Concrete setup leakage • Size of global MM
• Total size of local MM
• Concrete query leakage • Search and access pattern of global MM
• Search pattern of accessed local MMs
• Access pattern of accessed local MMs
• Tags of accessed local MMs
• Setup leakage of local MMs
• Search and access pattern of DX
28
IEX: Leakage
• Black-box setup leakage • Setup leakage of global EMM
• Setup leakage of EDX
• Black-box query leakage for disjunction • Query leakage of global EMM
• Query leakage of EDX
• Concrete setup leakage • Size of global MM
• Total size of local MM
• Concrete query leakage • Search and access pattern of global MM
• Search pattern of accessed local MMs
• Access pattern of accessed local MMs
• Tags of accessed local MMs
• Setup leakage of local MMs
• Search and access pattern of DX
28
Less leakage than OXT
IEX: Asymptotics
• Communication complexity is optimal
29
IEX: Asymptotics
• Communication complexity is optimal
• Worst-case search complexity (q keywords) • Sub-linear in where
29
IEX: Asymptotics
• Communication complexity is optimal
• Worst-case search complexity (q keywords) • Sub-linear in where
• Storage
29
Improving IEX Storage Overhead
• Can we make IEX more compact? • Problem is local EMMs are too large
30
Improving IEX Storage Overhead
• Can we make IEX more compact? • Problem is local EMMs are too large
• Use Z-IDX [Goh03] as local EMM? • Linear search complexity is OK
• Very compact (based on Bloom filters)
• Not adaptively-secure!
30
Improving IEX Storage Overhead
• Can we make IEX more compact? • Problem is local EMMs are too large
• Use Z-IDX [Goh03] as local EMM? • Linear search complexity is OK
• Very compact (based on Bloom filters)
• Not adaptively-secure!
• Z-IDX can be made adaptively-secure • But token size too large (far from optimal)
30
Improving IEX Storage Overhead
• Matryoshka filters • New nested Bloom filters with variable size and fixed hash functions
31
Improving IEX Storage Overhead
• Matryoshka filters • New nested Bloom filters with variable size and fixed hash functions
• Encrypted Matryoshka filters • Based on online ciphers
• Adaptively-secure
• Compact structure
• Optimal token size
• Linear search complexity
31
Improving IEX Storage Overhead
• Matryoshka filters • New nested Bloom filters with variable size and fixed hash functions
• Encrypted Matryoshka filters • Based on online ciphers
• Adaptively-secure
• Compact structure
• Optimal token size
• Linear search complexity
31
Improving IEX Storage Overhead
• Matryoshka filters • New nested Bloom filters with variable size and fixed hash functions
• Encrypted Matryoshka filters • Based on online ciphers
• Adaptively-secure
• Compact structure
• Optimal token size
• Linear search complexity
31
Evaluation (up to 61M keyword/id pairs)
32
Evaluation (up to 61M keyword/id pairs)
32 OXT 200 ms
Evaluation (up to 61M keyword/id pairs)
32 OXT 200 ms
10×
Clusion
• Encrypted search library • Open source under GPLv3 • Java
33
Clusion
• Encrypted search library • Open source under GPLv3 • Java
• Currently implements • SSE: 2Lev & ZMF • Dynamic SSE: forward-secure 2Lev (new) • Disjuntive SSE: IEX-2Lev & IEX-ZMF • Boolean SSE: BIEX-2Lev & BIEX-ZMF
33
Clusion
• Encrypted search library • Open source under GPLv3 • Java
• Currently implements • SSE: 2Lev & ZMF • Dynamic SSE: forward-secure 2Lev (new) • Disjuntive SSE: IEX-2Lev & IEX-ZMF • Boolean SSE: BIEX-2Lev & BIEX-ZMF
• In progress • Dynamic SSE: forse-1, forse-2 • Graph encryption: LGX
33
Thank you!
34
https://github.com/encryptedsystems/Clusion