Towards Searchable and Verifiable Blockchain · BasicSolution...

Towards Searchable and Verifiable Blockchain

Cheng Xu Ce ZhangApril 8, 2019

Department of Computer ScienceHong Kong Baptist University

Background

• Blockchain 6= Bitcoin

• Blockchain is a distributed ledger maintainedby a community of (untrusted) users

• Decentralization• Consensus

• Immutability• Provenance

• A wide range of applications

• Record Keeping• Smart Contracts• · · ·

1/17

Background

• Blockchain 6= Bitcoin• Blockchain is a distributed ledger maintainedby a community of (untrusted) users



• A wide range of applications

• Record Keeping• Smart Contracts• · · ·

Fig. 1: Blockchain Structure [Credit: Wikipedia]

1/17

https://commons.wikimedia.org/wiki/File:Blockchain_workflow.png

Background

• Blockchain 6= Bitcoin• Blockchain is a distributed ledger maintainedby a community of (untrusted) users



• A wide range of applications• Record Keeping• Smart Contracts• · · · Fig. 2: Blockchain Applications [Credit: FAHM Technology Partners]

1/17

http://www.fahmpartners.com/solutions/blockchain/

Blockchain Database Solutions

• Increasing demand to search the data stored in blockchains• Blockchain database solutions to support SQL-like queries

Blockchain Database Solutions

3/23/19 5

SwarmDB

• Increasing demand to search the data stored in blockchains

• Blockchain database solutions to support SQL-like queries

Issue: relying on a trusted party who can faithfully execute user queriesFig. 3: Blockchain Database Solutions

• Issue: relying on a trusted party who can faithfully execute user queries

2/17

Blockchain Search Problem

• Integrity assurance: query results retrieved from the blockchain should be publiclyverifiable

• Becoming full node• High cost

• Storage: to store a complete replicate (240 GB for Bitcoin as of Mar 2019)• Computation: to verify the consensus proofs• Network: to synchronize with the network

• Alternative approach: becoming light node and outsource computation

• Low cost: maintaining block headers only (< 50 MB for Bitcoin)

Miner Full Node

Light Node

Full Node Miner

· · ·

Q

R

Q

R

• Question: how to ensure integrity?

3/17





• Alternative approach: becoming light node and outsource computation• Low cost: maintaining block headers only (< 50 MB for Bitcoin)

Miner Full Node

Light Node

Full Node Miner

· · ·

Q

R

Q

R

• Question: how to ensure integrity?

3/17





• Alternative approach: becoming light node and outsource computation• Low cost: maintaining block headers only (< 50 MB for Bitcoin)

Miner Full Node

Light Node

Full Node Miner

· · ·

Q

R

Q

R

• Question: how to ensure integrity?3/17

Solution #1: Smart Contract

• A trusted program to execute user-defined computationupon the blockchain

• Smart Contract reads and writes blockchain data• Execution integrity is ensured by the consensus protocol

• Offer trusted storage and computation capabilities• Function as a trusted virtual machine

TraditionalComputer

BlockchainVM

Storage RAM BlockchainComputation CPU Smart Contract

4/17


• Leverage Smart Contract for trusted computation• Users submit query parameters to blockchain• Miners execute computation and write results into blockchain• Users read results from blockchain

[Credit: Oscar W]

• Drawbacks• Long latency: long time for consensus protocol to confirm a block• Poor scalability: transaction rate of the blockchain is limited• Privacy concern: query history is permanently and publicly stored in blockchain• High cost: executing smart contract in ETH requires paying gas to miners

(INFOCOM 2018 requires 4 201 232 gas = 0.18 Ether = 24 USD per query)

S. Hu, C. Cai, Q. Wang, C. Wang, X. Luo, and K. Ren, “Searching an encrypted cloud meets blockchain: Adecentralized, reliable and fair realization,” in IEEE INFOCOM, Honolulu, HI, USA, 2018, pp. 792–800.

5/17

https://hackernoon.com/ai-smart-contracts-the-past-present-and-future-625d3416807b


• Leverage Smart Contract for trusted computation• Users submit query parameters to blockchain• Miners execute computation and write results into blockchain• Users read results from blockchain

[Credit: Oscar W]• Drawbacks• Long latency: long time for consensus protocol to confirm a block• Poor scalability: transaction rate of the blockchain is limited• Privacy concern: query history is permanently and publicly stored in blockchain• High cost: executing smart contract in ETH requires paying gas to miners

(INFOCOM 2018 requires 4 201 232 gas = 0.18 Ether = 24 USD per query)

S. Hu, C. Cai, Q. Wang, C. Wang, X. Luo, and K. Ren, “Searching an encrypted cloud meets blockchain: Adecentralized, reliable and fair realization,” in IEEE INFOCOM, Honolulu, HI, USA, 2018, pp. 792–800.

5/17

https://hackernoon.com/ai-smart-contracts-the-past-present-and-future-625d3416807b

Solution #2: Verifiable Computation

• Verification Computation (VC)• Computation is outsourced to untrusted service provider• Service provider returns results with cryptographic proof• Users verify integrity of results using the proof

• Outsource queries to full node and verify the results using VC• General VC: Expressive but high overhead• Authenticated Data Structure (ADS)-based VC: Efficient but requiring customized designs

6/17

Our Solutions

• vChain: Enabling Verifiable Boolean Range Queries over BlockchainDatabases (SIGMOD 2019)

• GEM2-Tree: Enabling Gas-Efficient Authenticated Range Queries for Hy-brid Storage in Blockchain (ICDE 2019)

7/17

vChain: Enabling Verifiable Boolean RangeQueries over Blockchain Databases

Cheng Xu, Ce Zhang, and Jianliang Xu

ACM SIGMOD 2019

vChain — Problem Definition

• Problem: integrity-assured search over blockchain data

• System Model:

• Users become light nodes• Queries are outsourced to full nodes

• Full node not trusted

• Program glitches• Security vulnerabilities• Commercial interest• · · ·

Miner Full Node

Light Node

Full Node Miner

· · ·Q

〈R, VO〉

Q〈R,VO〉

• Security requirements• Soundness: none of the objects returned as results have been tampered with and all ofthem satisfy the query conditions

• Completeness: no valid result is missing regarding the query window or subscription period

8/17


• Problem: integrity-assured search over blockchain data• System Model:


• Full node not trusted

• Program glitches• Security vulnerabilities• Commercial interest• · · ·

Miner Full Node

Light Node

Full Node Miner

· · ·Q

R

Q

R



8/17




• Full node not trusted• Program glitches• Security vulnerabilities• Commercial interest• · · ·

Miner Full Node

Light Node

Full Node Miner

· · ·Q

R

Q

R



8/17




• Full node not trusted• Program glitches• Security vulnerabilities• Commercial interest• · · ·

Miner Full Node

Light Node

Full Node Miner

· · ·Q

〈R, VO〉

Q〈R,VO〉



8/17

vChain — System Overview

• Miner: constructs each block with additional ADSto achieve VC scheme

• Service Provider: is a full node and computes theresults with the verification object (VO)

• Query User: is a light node; uses the VO and blockheader to verify the results

(Full Node). . . . . .

PreBkHash . . .

oi ti Vi Wi

o1 t1 1 {a,b}o2 t2 2 {a}o3 t3 3 {c}o4 t4 5 {a,b}o5 t5 1 {c,d}

blocki

(Light Node)

. . . . . .

q1 : 〈[ti, ti], [1, 2],b〉

q2 : 〈−, [1, 2], c ∨ d〉

q3 : 〈−, [1, 2],a ∧ d〉

(Miner) Block HeadersBloc

k Heade

rs &Data

(incl. AD

S)

Q

Service Provider (SP) Query User

〈{o1}, VO〉

〈{o5}, VO〉

〈∅, VO〉

Block Header & DataBlock Header

Fig. 4: System Model of vChain

9/17

vChain — Data Model & Queries

• Data Model• Each block contains several temporal objects {o1, o2, . . . , on}• oi is represented by 〈ti, Vi,Wi〉(timestamp, multi-dimensional vector, set valued attribute)

• Boolean Range Queries• Time-window queries:q = 〈[2018-05, 2018-06], [10,+∞], “send:1FFYc” ∧ “receive:2DAAf”〉

• Subscription queries:q = 〈−, [200, 250], “Sedan” ∧ (“Benz” ∨ “BMW”)〉

10/17

Cryptographic Building Block

• Merkle Hash Tree [Mer89]• Support efficient membership/range queries• Limitations

• An MHT supports only the query keys on which the Merkle tree is built• MHTs do not work with set-valued attributes• MHTs of different blocks cannot be aggregated

N0 = H(N1|N2)

N1 = H(N3|N4) N2 = H(N5|N6)N2 = H(N5|N6)

N3 = H(o1)N3 = H(o1) N4 = H(o2)

o1 o2

N5 = H(o3) N6 = H(o4)

o3 o4

sig(N0)

Q

Fig. 5: Merkle Hash Tree

• Cryptographic Multiset Accumulator [PTT11]• Map a multiset to an element in cyclic multiplicative group in a collision resistant fashion• Utility: prove set disjoint• Protcols:

• KeyGen(1λ) → (sk, pk): generate keys• Setup(X, pk) → acc(X): return the accumulative value w.r.t. X• ProveDisjoint(X1, X2, pk) → π:on input two multisets X1 and X2 , where X1 ∩ X2 = ∅, output a proof π

• VerifyDisjoint(acc(X1), acc(X2), π, pk) → {0, 1}:on input the accumulative values acc(X1), acc(X2), and a proof π, output 1 iff X1 ∩ X2 = ∅

11/17

Basic Solution

• Consider a single object and boolean time-window query• Each block stores a single object oi = 〈ti,Wi〉

• ADS generation (Miner)• Extend the block header with AttDigest• AttDigest = acc(Wi) = Setup(Wi,pk)

• Constant size regardless of number of elements in Wi• Support ProveDisjoint(·) & VerifyDisjoint(·)

. . . PreBkHash TS ConsProof ObjectHash AttDigest

blocki

oi

. . .

Fig. 6: Extended Block Structure

• Verifiable Query

• Match:

return oi as a result; integrity is ensured by the ObjectHash in the block header

• Mismatch:

use AttDigest to prove the mismatch of oi

Example of Mismatch• Transform query condition to a list of sets: q = “Sedan” ∧ (“Benz” ∨ “BMW”) → {“Sedan”}, {“Benz”, “BMW”}

• Consider oi : {“Van”, “Benz”}, we have {“Sedan”} ∩ {“Van”, “Benz”} = ∅

• Apply ProveDisjoint({“Van”, “Benz”}, {“Sedan”}, pk) to compute proof π

• User retrieves AttDigest = acc({“Van”, “Benz”}) from the block header and usesVerifyDisjoint(AttDigest, acc({“Sedan”}), π, pk) to verify the mismatch

12/17

Basic Solution

• Consider a single object and boolean time-window query• Each block stores a single object oi = 〈ti,Wi〉• ADS generation (Miner)

• Extend the block header with AttDigest• AttDigest = acc(Wi) = Setup(Wi,pk)



blocki

oi

. . .


• Verifiable Query

• Match:


• Mismatch:






12/17

Basic Solution





blocki

oi

. . .


• Verifiable Query• Match:


• Mismatch:






12/17

Basic Solution





blocki

oi

. . .


• Verifiable Query• Match: return oi as a result; integrity is ensured by the ObjectHash in the block header• Mismatch:






12/17

Basic Solution





blocki

oi

. . .


• Verifiable Query• Match: return oi as a result; integrity is ensured by the ObjectHash in the block header• Mismatch: use AttDigest to prove the mismatch of oi





12/17

Basic Solution





blocki

oi

. . .


• Verifiable Query• Match: return oi as a result; integrity is ensured by the ObjectHash in the block header• Mismatch: use AttDigest to prove the mismatch of oi




• User retrieves AttDigest = acc({“Van”, “Benz”}) from the block header and usesVerifyDisjoint(AttDigest, acc({“Sedan”}), π, pk) to verify the mismatch 12/17

Basic Solution

• Support time-window queries• Find the blocks whose timestamp is within the query window• Invoke previous algorithm for each object in theses blocks

Example

• q = “Sedan” ∧ (“Benz” ∨ “BMW”)

• Objects within the time window o1 : {“Van”, “Benz”}, o2 : {“Sedan”, “Audi”}, o3 : {“Van”, “Benz”}• Query processing

• o1 is returned as a result• ProveDisjoint(·) is applied for o2 and o3

• Mismatch condition “Benz” ∨ “BMW” for o2• Mismatch condition “Sedan” for o3

13/17

Extension to Range Queries

• Idea: transform numerical attributes into set-valued attributes

• Numerical value can be transformed into a set ofbinary prefix elements

• Example: trans(4) = {1∗, 10∗, 100}* denotes wildcard matching operator

• Range can be transformed into a equivalentboolean expression using a binary tree

• Example: [0, 6] → 0∗ ∨ 10∗ ∨ 110Equivalence set: {0∗, 10∗, 110}

∗

0∗

00∗

000 001

01∗

010 011

1∗

10∗

100 101

11∗

110 111

q = [0, 6]

Fig. 7: Example of Transformation

• Range queries can be processed in a similar manner as Boolean queries• Transform vi ∈ [α, β] → trans(vi) ∩ EquiSet([α, β]) 6= ∅• Example:

• 4 ∈ [0, 6] → {1∗, 10∗, 100} ∩ {0∗, 10∗, 110} = {10∗} 6= ∅• 7 /∈ [0, 6] → {1∗, 11∗, 111} ∩ {0∗, 10∗, 110} = ∅

14/17

Batch Verification & Subscription Queries

• Observation: objects may share common attributes that mismatch query condition• Idea: we can aggregate them to speed up query processing

• Intra-Block Index: aggregate objects inside same block using MHT• Inter-Block Index: aggregate objects across blocks using skip list• Inverted Prefix Tree: aggregate similar subscription queries from users

PreBkHash TS ConsProof MerkleRoot

NrN5

N1 N2N6

N3 N4

hashr Wr AttDigestr

hash1 W1 AttDigest1

o1

blocki. . . . . .

Node Object Set AttributesN1 o1 W1 = {“Sedan”, “Benz”}N2 o2 W2 = {“Sedan”, “Audi”}N3 o3 W3 = {“Van”, “Benz”}N4 o4 W4 = {“Van”, “BMW”}

Fig. 8: Intra-Block Index

PreBkHash MerkleRoot SkipListRoot

L2L4. . .

PreSkippedHashL2 WL2 AttDigestL2PreSkippedHashL4 WL4 AttDigestL4. . .

blocki. . .PreBkHash . . . . . .

. . .

blocki−2. . .PreBkHash . . . . . .

. . .

blocki−4. . . . . .

Fig. 9: Inter-Block Index

q1

q2

q3q4

N1 N2

N3 N4x

y

0 1 2 3

0

1

2

3 N0

N1

N5 N6 N7 N8

N2 N3 N4

oi = 〈ti, (0, 2), {“Van”, “Benz”}〉= 〈ti, {001, 102, “Van”, “Benz”}〉

Query Range Boolean Conditionq1 [(0, 2), (1, 3)] {“Van” ∧ “Benz”}q2 [(0, 0), (1, 3)] {“Van” ∧ “BMW”}q3 [(0, 2), (0, 2)] {“Sedan” ∧ “Audi”}q4 [(2, 0), (3, 3)] {“Sedan” ∧ “Benz”}

Grid Cell : [(0, 2), (1, 3)] → {0∗1 ∧ 1∗2}RCIF: Query Cover Type

q1 fullq2 fullq3 partial

Query Condition Set Υ Queries{“Van”} q1,q2{“Benz”} q1{“BMW”} q2

BCIF:

Fig. 10: Inverted Prefix Tree

15/17



• Intra-Block Index: aggregate objects inside same block using MHT

• Inter-Block Index: aggregate objects across blocks using skip list• Inverted Prefix Tree: aggregate similar subscription queries from users


NrN5

N1 N2N6

N3 N4

hashr Wr AttDigestr

hash1 W1 AttDigest1

o1

blocki. . . . . .




L2L4. . .



. . .


. . .

blocki−4. . . . . .


q1

q2

q3q4

N1 N2

N3 N4x

y

0 1 2 3

0

1

2

3 N0

N1

N5 N6 N7 N8

N2 N3 N4






BCIF:


15/17



• Intra-Block Index: aggregate objects inside same block using MHT• Inter-Block Index: aggregate objects across blocks using skip list

• Inverted Prefix Tree: aggregate similar subscription queries from users


NrN5

N1 N2N6

N3 N4

hashr Wr AttDigestr

hash1 W1 AttDigest1

o1

blocki. . . . . .




L2L4. . .



. . .


. . .

blocki−4. . . . . .


q1

q2

q3q4

N1 N2

N3 N4x

y

0 1 2 3

0

1

2

3 N0

N1

N5 N6 N7 N8

N2 N3 N4






BCIF:


15/17



• Intra-Block Index: aggregate objects inside same block using MHT• Inter-Block Index: aggregate objects across blocks using skip list• Inverted Prefix Tree: aggregate similar subscription queries from users


NrN5

N1 N2N6

N3 N4

hashr Wr AttDigestr

hash1 W1 AttDigest1

o1

blocki. . . . . .




L2L4. . .



. . .


. . .

blocki−4. . . . . .


q1

q2

q3q4

N1 N2

N3 N4x

y

0 1 2 3

0

1

2

3 N0

N1

N5 N6 N7 N8

N2 N3 N4






BCIF:


15/17

Performance Evaluation

• Evaluation metrics• Query processing cost interms of SP CPU time

• Query verification cost interms of user CPU time

• Size of the VO transmittedfrom the SP to the user

• Numerical range selectivity• 10% for 4SQ & WX• 50% for ETH

• Disjunctive Boolean functionsize

• 3 for 4SQ & WX• 9 for ETH

4SQ0

100

200

300

400

2 (240)

4 (480)

6 (720)

8 (960)

10 (1200)

SP

CP

U T

ime

(s)

Time Window (Hour)/(Blocks)

nil-acc1nil-acc2intra-acc1

intra-acc2both-acc1both-acc2

0.01

0.1

1

10

100

2 (240)

4 (480)

6 (720)

8 (960)

10 (1200)

Use

r C

PU

Tim

e (

s)




1

10

102

103

104

2 (240)

4 (480)

6 (720)

8 (960)

10 (1200)

VO

Siz

e (

KB

)




WX0

5

10

15

20 (20)

40 (40)

60 (60)

80 (80)

100 (100)

SP

CP

U T

ime

(s)




0.01

0.1

1

10

20 (20)

40 (40)

60 (60)

80 (80)

100 (100)

Use

r C

PU

Tim

e (

s)




10

102

103

20 (20)

40 (40)

60 (60)

80 (80)

100 (100)

VO

Siz

e (

KB

)




ETH0

50

100

150

2 (480)

4 (960)

6 (1440)

8 (1920)

10 (2400)

SP

CP

U T

ime

(s)




0.01

0.1

1

10

100

2 (480)

4 (960)

6 (1440)

8 (1920)

10 (2400)

Use

r C

PU

Tim

e (

s)




1

10

102

103

104

2 (480)

4 (960)

6 (1440)

8 (1920)

10 (2400)

VO

Siz

e (

KB

)




Fig. 11: Time-Window Query Performance 16/17

GEM2-Tree: Enabling Gas-Efficient AuthenticatedRange Queries for Hybrid Storage in Blockchain

Ce Zhang, Cheng Xu, Jianliang Xu, Yuzhe Tang, and Byron Choi

IEEE ICDE 2019

GEM2-Tree

• Storing data on chain is not scalable• Hybrid storage:

• Raw data is stored off-chain• A hash of the data is keep on chain to ensure integrity• Smart contract maintains on-chain index to facilitateauthenticated query processing

• Question: How to reduce transaction fee a.k.a gas?

• More details• Section: Research (14) — Query Processing, Indexing andOptimization

• Time: 14:35–16:05, April 10, Wednesday• Location: 7004

... ...

HybridStorage

Cloud ServiceProvider

ClientDataOwners

R, VOsp

VOchain

Q

SmartContract

<key, value>

...

Smart Contract<key, h(value)>

Fig. 12: Authenticated Query Framework inHybrid-Storage Blockchain

17/17

http://conferences.cis.umac.mo/icde2019/?page_id=516#research14

GEM2-Tree

• Storing data on chain is not scalable

• Hybrid storage:• Raw data is stored off-chain• A hash of the data is keep on chain to ensure integrity• Smart contract maintains on-chain index to facilitateauthenticated query processing




... ...

HybridStorage


ClientDataOwners

R, VOsp

VOchain

Q

SmartContract

<key, value>

...



17/17


GEM2-Tree






... ...

HybridStorage


ClientDataOwners

R, VOsp

VOchain

Q

SmartContract

<key, value>

...



17/17


GEM2-Tree



• Question: How to reduce transaction fee a.k.a gas?• More details

• Section: Research (14) — Query Processing, Indexing andOptimization


... ...

HybridStorage


ClientDataOwners

R, VOsp

VOchain

Q

SmartContract

<key, value>

...



17/17


ThanksQuestions?

17/17

References

[HCW+18] S. Hu, C. Cai, Q. Wang, C. Wang, X. Luo, and K. Ren, “Searching an encrypted cloud meets blockchain: Adecentralized, reliable and fair realization,” in IEEE INFOCOM, Honolulu, HI, USA, 2018, pp. 792–800.

[Mer89] R. C. Merkle, “A certified digital signature,” in CRYPTO, 1989, pp. 218–238.

[PTT11] C. Papamanthou, R. Tamassia, and N. Triandopoulos, “Optimal verification of operations on dynamic sets,” inCRYPTO, Santa Barbara, CA, USA, 2011, pp. 91–110.

[XZX19] C. Xu, C. Zhang, and J. Xu, “vChain: Enabling verifiable boolean range queries over blockchain databases,” in ACMSIGMOD, Amsterdam, Netherlands, 2019.

[ZXX+19] C. Zhang, C. Xu, J. Xu, Y. Tang, and B. Choi, “GEM2-Tree: A gas-efficient structure for authenticated range queriesin blockchain,” in IEEE ICDE, Macau SAR, China, 2019.

Towards Searchable and Verifiable Blockchain · BasicSolution...

Documents

Transcript of Towards Searchable and Verifiable Blockchain · BasicSolution...