Extracting Randomness

David ZuckermanUniversity of Texas at Austin

Randomness extremely useful

• Algorithms– Approximation, optimization, factoring polys.

• Monte Carlo simulations• Cryptography• Distributed computing

– Consensus, Byzantine agreement, load balancing.

Randomness wonderful, but …

• Computers typically don’t have access to true randomness.

Is Randomness Necessary?

• Essential for distributed computing and cryptography:– Must choose secret key randomly.

• Unclear for algorithms.

Is Randomness Necessary?

• Major open question in field: does every efficient randomized algorithm have an efficient deterministic counterpart?– Does RP = P?

Is Randomness Necessary?

• Major open question in field: does every efficient randomized algorithm have an efficient deterministic counterpart?– Does RP = P? Appears very difficult.– Does RSPACE(S) = SPACE(S)? Difficult but

some hope.

What is minimal randomness requirement?

• Can we eliminate randomness completely?• If not:

– Can we minimize quantity of randomness?

– Can we minimize quality of randomness?• What does this mean?

What is minimal randomness requirement?

• Can we eliminate randomness completely?• If not:

– Can we minimize quantity of randomness?• Pseudorandom generator

– Can we minimize quality of randomness?• Extractor

Pseudorandom Generators

• Computers rely on pseudorandom generators:

PRG71294 141592653589793238

short random string

long “random-enough”string

Classical approach: ad hoc. Many failures.Modern approach: provably good PRGs.

Quality: von Neumann’s model

Extvery long

weakly randomlong

random

• Bits independent.• Each bit has same bias:

–Pr[Xi=1] = p, p unknown.

• Can’t use directly.Goal:

Quality: von Neumann’s model

• Extractor:– Group bits in pairs.– Pr[01]=Pr[10] = p(1-p).– Map 01 to 0, 10 to 1, ignore 00 and 11.

• Example: 01 01 11 10 11 01 00 maps to 0 0 1 0

Use in Practice

• Intel has random number generator (not PRG) which uses white noise.

• Temperature may influence bias.• Intel applies von Neumann’s extractor to

output.

General Weakly Random Sources

• What if bits are correlated?• Many models studied [Blum,

Santha-Vazirani, Chor-Goldreich].• Most general model - upper bound

probability of each string [Zuckerman].• Similar to lower bounding entropy.

General Weakly Random Sources

• Weakly random distribution on n bits: each string has probability ≤ 2-k.

• Example: weakly random integer in [1,1000].

• Distribution unknown.0

0.0010.0020.0030.0040.0050.0060.0070.0080.0090.01

1 2 3 4 5 6 7 8

weaklyrandomalmostrandomtrulyrandom

Goal

Extvery long

weakly randomlong

almost random

Should work for all (n,k) weakly random sources.

Goal

Extvery long

weakly randomlong

almost random

Should work for all (n,k) weakly random sources.

Problem: impossible.

Solution: Extractor[Nisan-Zuckerman]

Extvery long

weakly randomlong

almost random

short truly random

Extractor Parameters[NZ,…, Lu-Reingold-Vadhan-Wigderson]

Ext n bits

weakly randomPr[each string] ≤ 2-k

.99k bits

almost random

O(log n) truly random

Power of Extractors

• Sometimes can eliminate true randomness by cycling over all possibilities.

Power of Extractors

• Sometimes can eliminate true randomness by cycling over all possibilities.

• Useful even when no weakly random source apparently present.

Power of Extractors

• Sometimes can eliminate true randomness by cycling over all possibilities.

• Useful even when no weakly random source apparently present.

• Mathematical reason for power: extractor constructions beat “eigenvalue bound.”

Applications of Extractors• PRGs for Space-Bounded Computation [Nisan-Z]• PRGs for Random Sampling [Z]• Cryptography [Lu, Vadhan, Dodis-Smith]• Expander graphs and highly connected networks[Wigderson-Z]• Coding theory [Ta-Shma- Z]• Hardness of approximation [Z, Mossel-Umans]• Efficient deterministic sorting [Pippenger]• Time-space tradeoffs [Sipser]• Implicit data structures [Fiat-Naor, Z]

New Extractor and Application[Z]

• Extractor requires log n + O(1) random bits.• NP-complete to approximate MAX

CLIQUE and CHROMATIC NUMBER to within n1-, any >0.– Previously same inapproximability ratio

required NP ZPP [Hastad, Feige-Kilian].– We use new extractor to derandomize previous

reductions.

The Future for Extractors

• Current extractors near optimal.• Where to go from here?• Two interesting directions:

– Deterministic extractors for specialized sources.– Extractors for independent sources and a new

technique.

Bit-Fixing Sources

• Adversary fixes all but k of the n bits.• Remaining k bits chosen randomly.• Parity can extract 1 bit if k≥1.

Bit-Fixing Sources

• Adversary fixes all but k of the n bits.• Remaining k bits chosen randomly.• Parity can extract 1 bit if k≥1.• This model seems unrealistic:

– What good is it?

Bit-Fixing Sources

• Adversary fixes all but k of the n bits.• Remaining k bits chosen randomly.• Parity can extract 1 bit if k≥1.• This model seems unrealistic:

– What good is it?• Applications in cryptography and more

realistic models.

Bit-Fixing Sources

• Adversary fixes all but k of the n bits.• Remaining k bits chosen randomly.• Parity can extract 1 bit if k≥1.• To extract 2 truly random bits, need k>n/3.• Can extract k2/n almost-random bits

deterministically [Kamp-Zuckerman].• Improved to (1-o(1))k [Gabizon-

Raz-Shaltiel].

Exposure-Resilient Cryptography

• Standard cryptography: secret keys totally secret.• What if adversary learns some bits of secret key?• Deterministic extractors for bit-fixing sources can

help foil such adversaries [Dodis-Sahai-Smith]. Need exponentially small error.

• Kamp-Z extractor has small enough error to apply ([GRS] error too large).

More realistic sources: Generalizing von Neumann’s Model• Bits independent, allow different biases.• Deterministic extractors for bit-fixing

sources also work for these new sources [Kamp-Vadhan-Zuckerman].

• Goal: deterministic extractors for more general sources.

• Some preliminary results allowing correlations.

Technique:Additive Number Theory

• For set A, A+A={a1+a2: a1, a2 in A}• Thm: either |A+A|>|A|1.01 or |AA| > |A|1.01

[Bourgain-Katz-Tao, Konyagin].• Can extract from 3 independent sources

[Barak-Kindler-Shaltiel-Sudakov-Wigerson].• Promising technique -- other applications?• Anup Rao: improvements without additive

number theory.

Conclusions

• Extractors fundamental: diverse applications.• Future in extractors:

– Deterministic extractors– 2-source extractors– More applications– Practical variants

• Can we make progress towards RP=P or RSPACE(S) = SPACE(S)?

Students

• Jesse Kamp - extractors• Anindya Patthak - coding theory• Anup Rao - extractors

Extractors in Cryptography

• Alice and Bob know s = “secret” random 1000 bit string.• Eavesdropper Eve knows 600 bits of s.• Alice and Bob don’t know which 600 bits.• Eve can see all communication.

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

Extractors in Cryptography

• Alice and Bob compute a shared secret string of 300 bits, about which Eve has negligible information:

• To Eve, s appears like output of known bit-fixing source. So Ext(s) will appear almost random.

• Hence shared secret = Ext(s).