Fooling Gaussian PTFs via Local...
Transcript of Fooling Gaussian PTFs via Local...
Fooling Gaussian PTFs via Local Hyperconcentration
Li-Yang Tan (Stanford)
Joint work with Ryan O’Donnell (CMU) and Rocco Servedio (Columbia)
Polynomial threshold functions (PTFs) over Rn
§ High-dimensional (𝑛 → ∞) geometric objects
§ Intensively studied within TCS and beyond TCS
polynomial
F (x) = sign(p(x))
<latexit sha1_base64="NZz65HbCm88gkCxYoAVGwDRupZQ=">AAACA3icdZDLSgMxFIYzXmu9jbrTTbAI7abMSL10IRQFcVnBXqAdSiZN29AkMyQZsQwFN76KGxeKuPUl3Pk2ZtoRqugPgZ/vnEPO+f2QUaUd59Oam19YXFrOrGRX19Y3Nu2t7boKIolJDQcskE0fKcKoIDVNNSPNUBLEfUYa/vAiqTduiVQ0EDd6FBKPo76gPYqRNqhj717m7wrwDLY50gPJY0X7YpwPDSx07JxTdCaCM+bIccvHLnRTkgOpqh37o90NcMSJ0JghpVquE2ovRlJTzMg4244UCREeoj5pGSsQJ8qLJzeM4YEhXdgLpHlCwwmdnYgRV2rEfdOZrKp+1xL4V60V6d6pF1MRRpoIPP2oFzGoA5gEArtUEqzZyBiEJTW7QjxAEmFtYsuaEL4vhf+b+mHRLRXL16Vc5TyNIwP2wD7IAxecgAq4AlVQAxjcg0fwDF6sB+vJerXepq1zVjqzA37Iev8C6wSWdQ==</latexit>
𝐹 𝑥 = −1p : Rn ! R
<latexit sha1_base64="rBRCuGEtsCnGnInL4O4NDj4vENE=">AAACCHicdVDLSgMxFM3UV62vUZeKBovgqsyIr7oqunHZin1AZyyZNG1DM5khyQhl6LIbf8WNC8W69RPc+Q3+hJm2QhU9EDiccy+553gho1JZ1oeRmpmdm19IL2aWlldW18z1jYoMIoFJGQcsEDUPScIoJ2VFFSO1UBDke4xUve5l4lfviJA04DeqFxLXR21OWxQjpaWGuRvCc+j4SHU8L77u33LoqGBKaJhZK2eNAKfIsWXnT2xoT5RsYXtY+hzsDIsN891pBjjyCVeYISnrthUqN0ZCUcxIP+NEkoQId1Gb1DXlyCfSjUdB+nBfK03YCoR+XMGROr0RI1/Knu/pyeRC+dtLxL+8eqRaZ25MeRgpwvH4o1bEoM6atAKbVBCsWE8ThAXVt0LcQQJhpbvL6BK+k8L/SeUwZx/l8iXdxgUYIw22wB44ADY4BQVwBYqgDDAYgAfwBJ6Ne+PReDFex6MpY7KzCX7AePsC7EidBw==</latexit>
𝐹 𝑥 = 1
p(x) =x21
a2+
x22
b2+
x23
c2� 1
<latexit sha1_base64="MI3RTfSRvgsc5KrYIHkpwCD/WYw=">AAACLHicdZDLSgMxFIYzXmu9jbp0EyxCRSwztV66EIrduKxgL9DWkkkzbWjmQpKRlqEP5MZXEcSFRdz6HGamU6miBxJ+vv8ckvNbPqNCGsZEW1hcWl5ZTa2l1zc2t7b1nd2a8AKOSRV7zOMNCwnCqEuqkkpGGj4nyLEYqVuDcuTXHwgX1HPv5MgnbQf1XGpTjKRCHb3sZ4dH8Aq2bI5wOOyY9/lxiNQFj2coHyErRt9tpxHDETuBZkfPGDkjLjgnzgyzeG5CMyEZkFSlo7+0uh4OHOJKzJAQTdPwZTtEXFLMyDjdCgTxER6gHmkq6SKHiHYYLzuGh4p0oe1xdVwJYzo/ESJHiJFjqU4Hyb747UXwL68ZSPuyHVLXDyRx8fQhO2BQejBKDnYpJ1iykRIIc6r+CnEfqTykyjetQphtCv8XtXzOLOSKt4VM6TqJIwX2wQHIAhNcgBK4ARVQBRg8gmfwBibak/aqvWsf09YFLZnZAz9K+/wCQA+lzg==</latexit>
F : Rn ! {±1}
<latexit sha1_base64="/dqBoDak1SC4IC7Uo2sXjKe6H6Y=">AAACCHicdVDLSsNAFJ34rPUVdenCwSK4KonUR10VBXFZxT6giWUynbRDJ5MwMxFKyNKNv+LGhSJu/QR3/o2TNkIVPTBwOOde5p7jRYxKZVmfxszs3PzCYmGpuLyyurZubmw2ZRgLTBo4ZKFoe0gSRjlpKKoYaUeCoMBjpOUNzzO/dUeEpCG/UaOIuAHqc+pTjJSWuubOBTyFToDUwPOS6/SWQ0eF0EmgEwXQdtKuWbLK1hhwihxadvXIhnaulECOetf8cHohjgPCFWZIyo5tRcpNkFAUM5IWnViSCOEh6pOOphwFRLrJOEgK97TSg34o9OMKjtXpjQQFUo4CT09mJ8vfXib+5XVi5Z+4CeVRrAjHk4/8mEGdNWsF9qggWLGRJggLqm+FeIAEwkp3V9QlfCeF/5PmQdmulKtXlVLtLK+jALbBLtgHNjgGNXAJ6qABMLgHj+AZvBgPxpPxarxNRmeMfGcL/IDx/gWd25h8</latexit>
living in R3
<latexit sha1_base64="6ce1+w0ksslEYMsBEdXEqQFwCOg=">AAACCHicdVDLTgJBEJzFF+IL9ejBiWDiiewqPrgRvXhEI48EkMwOA0yYnd3M9BLJhqMXf8WLB43x6id482+cBUzQaCWdVKq6093lBoJrsO1PKzE3v7C4lFxOrayurW+kN7cq2g8VZWXqC1/VXKKZ4JKVgYNgtUAx4rmCVd3+RexXB0xp7ssbGAas6ZGu5B1OCRipld5tALuDSPABl13MJc42PAI9142uR7dH2VErnbFz9hh4hhzbTuHEwc5UyaApSq30R6Pt09BjEqggWtcdO4BmRBRwKtgo1Qg1Cwjtky6rGyqJx3QzGj8ywvtGaeOOr0xJwGN1diIintZDzzWd8ZX6txeLf3n1EDpnzYjLIAQm6WRRJxQYfByngttcMQpiaAihiptbMe0RRSiY7FImhO9P8f+kcphz8rnCVT5TPJ/GkUQ7aA8dIAedoiK6RCVURhTdo0f0jF6sB+vJerXeJq0JazqzjX7Aev8CgdmZqQ==</latexit>
Main complexity measure for this talk: degree of p
Think of d as growing with n = dimension of Rn
For concreteness, can think of d = log n
More complex
F (x) = sign(p(x)), deg(p) d
<latexit sha1_base64="XYDso2xbcdhQru7sbAH9FgJ/YfQ=">AAACHnicdZDLSgMxFIYzXmu9jbp0ExShRSkz4q0LoSiIywq2Cp1SMpnTNpjJjElGLEPfwp0bX8WNC0UEV/ZtTC+Cih4I/Hz/OeSc3485U9pxetbY+MTk1HRmJjs7N7+waC8tV1WUSAoVGvFIXvpEAWcCKpppDpexBBL6HC78q+O+f3EDUrFInOtODPWQtARrMkq0QQ179yR3m8eHXkh0W4apYi3RzcWG5be864QE+MsJoGWMPPY44KBhrzsFZ1D4m9h13OKei90RWS+teZt3vVKn3LDfvSCiSQhCU06UqrlOrOspkZpRDt2slyiICb0iLagZKUgIqp4OzuviDUMC3IykeULjAf0+kZJQqU7om87+suq314d/ebVENw/qKRNxokHQ4UfNhGMd4X5WOGASqOYdIwiVzOyKaZtIQrVJNGtC+LoU/y+q2wV3p1A8M2kcoWFl0CpaQznkon1UQqeojCqIonv0iJ7Ri/VgPVmv1tuwdcwazaygH2V9fAJNO6Rg</latexit>
d=1 d=2 d=5 d=6
. . . . . .
... then F accepts (Δ + 0.01) fraction of points
Discrepancy Sets for PTFs over Gaussian space
For all degree-d PTFs F,if F ’s Gaussian volume is Δ
Consider Rn endowed with N(0,1)n. Want small set of points such that:
Random set of points works great. Want explicit set.
–Rn
<latexit sha1_base64="Btyd+c2axux+W+a7+3zMoUA6dkY=">AAAB83icdVDLSsNAFL2pr1pfVZduBovgqiRSH90V3bisYh/QxDKZTtqhk0mYmQgl9DfcuFDErT/jzr9x0kaoogcGDufcyz1z/JgzpW370yosLa+srhXXSxubW9s75d29tooSSWiLRDySXR8rypmgLc00p91YUhz6nHb88VXmdx6oVCwSd3oSUy/EQ8ECRrA2kuuGWI98P72d3ot+uWJX7RnQAjm1nfqZg5xcqUCOZr/84Q4ikoRUaMKxUj3HjrWXYqkZ4XRachNFY0zGeEh7hgocUuWls8xTdGSUAQoiaZ7QaKYubqQ4VGoS+mYyy6h+e5n4l9dLdHDhpUzEiaaCzA8FCUc6QlkBaMAkJZpPDMFEMpMVkRGWmGhTU8mU8P1T9D9pn1SdWrV+U6s0LvM6inAAh3AMDpxDA66hCS0gEMMjPMOLlVhP1qv1Nh8tWPnOPvyA9f4FaA+R9A==</latexit>
Pseudorandom generators over Gaussian space
��� Ex⇠N(0,1)n
[F (x)]� Es⇠{0,1}r
[F (G(s))]��� "
<latexit sha1_base64="Ub9By025V1IJ0gQdY4VS2UE/IC0=">AAAClHichVHdShtBFJ5dbauxrdGCN70ZGgoJtGFXbK3Qi2jQ9koUGhUya5idnMTB2ZllZlYapvtEfQcfoheFvk1nkxSMWDwwnI/vnPPN+UlzwY2Noj9BuLT85OmzldXa2vMXL9frG5tnRhWaQY8pofRFSg0ILqFnuRVwkWugWSrgPL3uVvHzG9CGK/nNTnJIMjqWfMQZtZ4a1H+SAz7+gUlG7ZXK3dSnqTssy4EjqRJDM8m8c99LYniGj5vRu7h1Kcv+UXMx3Erwe1x7VMfMdIjzOqS81JXQl+ZiRquVzLsSgMkN1ZAbLqpuG1E7mhq+Az5E8d7HGMdzptHpNn7f3m6dngzqv8hQsSIDaZmgxvTjKLeJo9pyJqCskcJATtk1HUPfQ0kzMImbLrXEbz0zxCOl/ZMWT9m7FY5mpmrZZ1bDmvuxinwo1i/s6FPiuMwLC5LNPhoVAluFqwvhIdfArJh4QJnmvlfMrqimzPo71vwS/k2K/w/OttvxTnvv1G/jAM1sBb1Gb1ATxWgXddBXdIJ6iAUbwW7QCfbDrfBz2A0PZ6lhMK95hRYsPP4LxtzOkw==</latexit>
Definition: An ε-PRG for a class C is an explicit function G : {0,1}r Rn
such that: for every function F in C, !
C = { degree-d PTFs } This work: r = “seed length”
= log(size of discrepancy set)
Rn
<latexit sha1_base64="Btyd+c2axux+W+a7+3zMoUA6dkY=">AAAB83icdVDLSsNAFL2pr1pfVZduBovgqiRSH90V3bisYh/QxDKZTtqhk0mYmQgl9DfcuFDErT/jzr9x0kaoogcGDufcyz1z/JgzpW370yosLa+srhXXSxubW9s75d29tooSSWiLRDySXR8rypmgLc00p91YUhz6nHb88VXmdx6oVCwSd3oSUy/EQ8ECRrA2kuuGWI98P72d3ot+uWJX7RnQAjm1nfqZg5xcqUCOZr/84Q4ikoRUaMKxUj3HjrWXYqkZ4XRachNFY0zGeEh7hgocUuWls8xTdGSUAQoiaZ7QaKYubqQ4VGoS+mYyy6h+e5n4l9dLdHDhpUzEiaaCzA8FCUc6QlkBaMAkJZpPDMFEMpMVkRGWmGhTU8mU8P1T9D9pn1SdWrV+U6s0LvM6inAAh3AMDpxDA66hCS0gEMMjPMOLlVhP1qv1Nh8tWPnOPvyA9f4FaA+R9A==</latexit>
Our main result: PRG for PTFs over Gaussian space
§ Previous best seed lengths: 2!(#) dependence
§ Probabilistic method: 𝑂(𝑑 log 𝑛 + log(1/𝜀))
§ Gaussian space special case of Boolean space
§ Current best Boolean seed length: still 2!(#)
An ε-PRG for degree-d PTFs over R% with seed length:
𝑑𝜀
!(#$% &)⋅ log 𝑛
Comparison with prior results
The rest of this talk
§ Key technical ingredient: New structural lemma about polynomials
“An arbitrary polynomial, when hit with a mild random restriction, collapses to a simple polynomial
with many enjoyable properties”
§ Sketch of the proof of this lemma
With this lemma, still lots of technical work to get PRG!
Bulk of project and paper
Concentrated polynomials
Space of all degree-d polynomials
Concentrateddegree-d polynomials
Var[p] ⌧ E[p]2
<latexit sha1_base64="NMXuQeVJG5wN5DfuTILipX+nzzM=">AAACDXicdZDLSgMxFIYz9VbrrerSTbAKrobMtNV2VxTBZQXbCp2xZNJMG5q5kGSEMvQF3Pgqblwo4ta9O9/GTFtBRX8I/HznHHLO78WcSYXQh5FbWFxaXsmvFtbWNza3its7bRklgtAWiXgkrj0sKWchbSmmOL2OBcWBx2nHG51l9c4tFZJF4ZUax9QN8CBkPiNYadQrHjgBVkMRpG0sJt3YhdDhHE6h56XnGbqxe8USMlG1Xq1VIDLLlapdtrWxashGx9Ay0VQlMFezV3x3+hFJAhoqwrGUXQvFyk2xUIxwOik4iaQxJiM8oF1tQxxQ6abTaybwUJM+9COhX6jglH6fSHEg5TjwdGe2pvxdy+BftW6i/JqbsjBOFA3J7CM/4VBFMIsG9pmgRPGxNpgIpneFZIgFJkoHWNAhfF0K/zdt27QqZv2yUmqczuPIgz2wD46ABU5AA1yAJmgBAu7AA3gCz8a98Wi8GK+z1pwxn9kFP2S8fQKK5Jvj</latexit>
A polynomial p is concentrated if
§ “Morally constant”, i.e. “morally degree 0”
§ sign(p(x)) = sign(E[p]) for most x’s
Our new structural lemma for polynomials
An arbitrary degree-d polynomial, when hit with a mild random restriction,
becomes concentrated with high probability. Arbitrary degree-d
polynomial p
Concentrateddegree-d polynomials
Theorem:
In fact, we show that they become hyperconcentrated:
Hyperconcentrated polynomials
HyperVar[p] ⌧ E[p]2
<latexit sha1_base64="UuDFeWMr6IusmnwR3TroMaJOaAs=">AAACI3icdZBLS8NAEMc3vq2vqEcvi0XwVJLiq55EETxWsLXQxLLZbtqlmwe7E7GEfBcvfhUvHhTx4sHv4iaNoKIDy/75zQwz8/diwRVY1rsxNT0zOze/sFhZWl5ZXTPXN9oqSiRlLRqJSHY8opjgIWsBB8E6sWQk8AS79kZnef76lknFo/AKxjFzAzIIuc8pAY165rED7A48Py1+DumFLpJZ5gQEhjJI20Rm3djFjhC4YJ6Xnufkpt4zq1bNKgJ/E/uW3TiwsV2SKiqj2TNfnX5Ek4CFQAVRqmtbMbgpkcCpYFnFSRSLCR2RAetqGZKAKTctbszwjiZ97EdSvxBwQb93pCRQahx4ujJfU/3O5fCvXDcB/8hNeRgnwEI6GeQnAkOEc8Nwn0tGQYy1IFRyvSumQyIJBW1rRZvwdSn+X7TrNXuv1rjcq56clnYsoC20jXaRjQ7RCbpATdRCFN2jR/SMXowH48l4Nd4mpVNG2bOJfoTx8QkA6qZR</latexit>
Random restrictions over Gaussian space
Definition: Let p be a polynomial. A λ-random restriction of p is the polynomial
q(x) = p(p�x+
p1� �y), y ⇠ N(0, 1)n.
<latexit sha1_base64="9yXRU1YmTuVINAmrRZQtAP/b6sE=">AAACYnicdZFbS8MwFMfTepvzNvVRHw6KsKGOVrw+CENffBIFp8I6R5pmM5imXZLKSum38JP55pMvfhCzG0zRAyH//M4lJyd+zJnSjvNh2VPTM7NzhfniwuLS8kppde1eRYkktE4iHslHHyvKmaB1zTSnj7GkOPQ5ffBfLvv+h1cqFYvEnU5j2gxxR7A2I1gb1Cqlnh/xQKWh2bJuXu5V4Bzisqe6UmceN4UCnPdgF4bE3R+zybxJneZ5ZQ+8boID+MlNCRbCddnZcytPotoqbTtVZ2AwIY4c9+zYBXdEtmtb3u7bRy29aZXevSAiSUiFJhwr1XCdWDczLDUjnOZFL1E0xuQFd2jDSIFDqprZYEQ57BgSQDuSZgkNAzqZkeFQ9Ts1kSHWz+q3rw//8jUS3T5tZkzEiaaCDC9qJxx0BP15Q8AkJZqnRmAimekVyDOWmGjzK0UzhPFL4X9xf1B1D6tnt2YaF2hoBbSBtlAZuegE1dAVukF1RNCnNWMtWyvWl120V+31YahtjXJG57HZm9+bfLqk</latexit>
§ The smaller 𝜆 is, the “harsher” the random restriction
§ q = p’s behavior in a local neighborhood around y, where 𝜆 = radius of neighborhood
Our new structural lemma, restated
Arbitrary degree-d polynomial p
Hyperconcentrateddegree-d polynomials
“For most points y, the polynomial p behaves like a constant function in a fairly large neighborhood around y.”
𝜆 = 𝑑!"#$ %
𝜆 = 𝑑!"#$ %
Local Hyperconcentration Theorem
The proof of our Local Hyperconcentration Theorem
Degree-d polynomial Hyperconcentrated
(“morally degree 0”)
Thanks Avi!
“morally” degree !
"
“morally” degree !
#
Degree-d polynomial
. . .
§ Hermite analysis
§ Carbery-Wright anticoncentration inequality
Technical ingredients:
log 𝑑 iterations
Open Problems
Arbitrary degree-d polynomial p
Hyperconcentrateddegree-d polynomials
𝜆 = 𝑑!"#$ %§ Conjecture: 𝜆 = 𝑑*! + suffices
§ Random restriction lemma for polynomials over Boolean space?
Local Hyperconcentration Theorem
Thank you!