Chapter 1Divide and ConquerPolynomial Multiplication
Algorithm TheoryWS 2014/15
Fabian Kuhn
Algorithm Theory, WS 2014/15 Fabian Kuhn 2
Representation of PolynomialsCoefficient representation:
• Polynomial ∈ of degree is given by its1 coefficients , … , :
⋯
• Example:3 15 18
• The most typical (and probably most natural) representation of polynomials
Algorithm Theory, WS 2014/15 Fabian Kuhn 3
Representation of PolynomialsPoint‐value representation:
• Polynomial ∈ of degree is given by1 point‐value pairs:
, , , , … , ,
where for .
• Example: The polynomial
3 2 3is uniquely defined by the four point‐value pairs 0,0 , 1,6 , 2,0 , 3,0 .
Algorithm Theory, WS 2014/15 Fabian Kuhn 4
Operations: Coefficient RepresentationDeg.‐ polynomials ⋯ , ⋯
Addition:⋯
• Time:
Multiplication:
⋅ ⋯ , where
• Naive solution: Need to compute product for all 0 ,
• Time:
Algorithm Theory, WS 2014/15 Fabian Kuhn 5
Operations Point‐Value RepresentationDegree‐ polynomials
, , … , , , , , … , ,
• Note: we use the same points , … , for both polynomials
Addition:
, , … , ,
• Time:
Multiplication:
⋅ , ⋅ , … , , ⋅
• Time:
Algorithm Theory, WS 2014/15 Fabian Kuhn 6
Faster Polynomial Multiplication?Multiplication is fast when using the point‐value representation
Idea to compute ⋅ (for polynomials of degree ):
, of degree 1, coefficients
2 2 point‐value pairs , and ,
2 point‐value pairs ,
of degree 2 2, 2 1 coefficients
Evaluation at points , , … ,
Point‐wise multiplication
Interpolation
Algorithm Theory, WS 2014/15 Fabian Kuhn 7
Point‐Value Representation of , • Select points , , … , to evaluate and in a clever way
Consider the powers of the principle th root of unity:
Principle root of unity:
1, 1
Powers of (roots of unity):
1 , , … ,
Note: cos ⋅ sin
Algorithm Theory, WS 2014/15 Fabian Kuhn 8
Discrete Fourier Transform• The values for 0,… , 1 uniquely define a
polynomial of degree .
Discrete Fourier Transform (DFT):• Assume , … , is the coefficient vector of poly.
⋯
DFT ≔ , ,… ,
Algorithm Theory, WS 2014/15 Fabian Kuhn 9
Faster Polynomial Multiplication?
Idea to compute ⋅ (for polynomials of degree ):
, of degree 1, coefficients
2 2 point‐value pairs , and ,
2 point‐value pairs ,
of degree 2 2, 2 1 coefficients
Evaluation at points , , … ,
Point‐wise multiplication
Interpolation
Algorithm Theory, WS 2014/15 Fabian Kuhn 10
Properties of the Roots of Unity• Cancellation Lemma:
For all integers 0, 0, and 0, we have:
• Proof:
Algorithm Theory, WS 2014/15 Fabian Kuhn 11
Divide‐and‐Conquer Approach• Divide of degree 1 ( is even) into 2 polynomials of
degree ⁄ 1 differently than in Karatsuba’s algorithm
• ⋯ ⁄ (even coeff.)⋯ ⁄ (odd coeff.)
Algorithm Theory, WS 2014/15 Fabian Kuhn 12
Discrete Fourier TransformEvaluation for 0,… , 1:
⋅
⁄ ⋅ ⁄ if 2
⁄⁄ ⋅ ⁄
⁄ if 2
For the coefficient vector of :
DFT ⁄ ,… , ⁄⁄ , ⁄ , … , ⁄
⁄
⁄ , … , ⁄⁄⁄ , ⁄
⁄ , … , ⁄⁄
Algorithm Theory, WS 2014/15 Fabian Kuhn 13
ExampleFor the coefficient vector of :
DFT ⁄ ,… , ⁄⁄ , ⁄ , … , ⁄
⁄
⁄ , … , ⁄⁄⁄ , ⁄
⁄ , … , ⁄⁄
4:
Need: , and ,
(DFTs of coefficient vectors of and )
Algorithm Theory, WS 2014/15 Fabian Kuhn 14
Recursive StructureFor simplicity, we abuse notation in the following:
• Poly. ⋯ with coefficient vector Let DFT ≔ DFT
Recursive structure: • For 4:
DFT DFT ⋅ DFT
• General (assume is even): DFT
DFT /
⋅ DFT /
Algorithm Theory, WS 2014/15 Fabian Kuhn 15
Computation of • Divide‐and‐conquer algorithm for DFT :
1. Divide
1: DFT
1: Divide into (even coeff.) and (odd coeff).
2. Conquer
Solve DFT ⁄ and DFT ⁄ recursively
3. Combine
Compute DFT based on DFT ⁄ and DFT ⁄
Algorithm Theory, WS 2014/15 Fabian Kuhn 16
Analysis• : time to compute DFT :
2 2 , 1 1
• As for mergesort, comparing orders, closest pair of points:
⋅ log
Algorithm Theory, WS 2014/15 Fabian Kuhn 17
Small ImprovementPolynomial of degree 1:
⁄ ⋅ ⁄ if 2
⁄⁄ ⋅ ⁄
⁄ if 2
⁄ ⋅ ⁄ if 2
⁄⁄ ⁄ ⋅ ⁄
⁄ if 2
Need to compute ⁄ and ⋅ ⁄ for 0 ⁄ .
Algorithm Theory, WS 2014/15 Fabian Kuhn 18
Example
⋅
⋅
⋅
⋅
Algorithm Theory, WS 2014/15 Fabian Kuhn 19
Fast Fourier Transform (FFT) AlgorithmAlgorithm FFT(a)• Input: Array of length , where is a power of 2• Output: DFT
if 1 then return ; // ≔ FFT , , … , ;≔ FFT , , … , ;≔ ; ≔ 1;
for 0 to ⁄ 1 do //
≔ ⋅ ;≔ ; ⁄ ≔ ;≔ ⋅
end;return , , … , ;
Algorithm Theory, WS 2014/15 Fabian Kuhn 20
Example• 3 15 18 0, 0,18, 15,3
Algorithm Theory, WS 2014/15 Fabian Kuhn 21
Faster Polynomial Multiplication?
Idea to compute ⋅ (for polynomials of degree ):
, of degree 1, coefficients
2 2 point‐value pairs , and ,
2 point‐value pairs ,
of degree 2 2, 2 1 coefficients
Evaluation at , , … , using FFT
Point‐wise multiplication
Interpolation
Algorithm Theory, WS 2014/15 Fabian Kuhn 22
InterpolationConvert point‐value representation into coefficient representation
Input: , , … , , with for
Output: Degree‐ 1 polynomial with coefficients , … , such that
⋯ ⋯
⋮⋮⋯
linear system of equations for , … ,
Algorithm Theory, WS 2014/15 Fabian Kuhn 23
InterpolationMatrix Notation:
1 ⋯1 ⋯⋮ ⋮ ⋱ ⋮1 ⋯
⋅ ⋮ ⋮
• System of equations solvable iff for all
Special Case :
1 1 1 ⋯ 11 ⋯1 ⋯⋮ ⋮ ⋮ ⋱ ⋮1 ⋯
⋅⋮ ⋮
Algorithm Theory, WS 2014/15 Fabian Kuhn 24
Interpolation• Linear system:
⋅ ⟹ ⋅
, , ⋮ , ⋮
Claim:
Proof: Need to show that
Algorithm Theory, WS 2014/15 Fabian Kuhn 25
DFT Matrix Inverse
⋯ 1
⋯ ⋮ ⋯
⋅
⋯ 1 ⋯⋯ ⋯⋯ ⋯ ⋮ ⋯ ⋯
Algorithm Theory, WS 2014/15 Fabian Kuhn 26
DFT Matrix Inverse
,
ℓ
ℓ
Need to show ,1if0if
Case :
Algorithm Theory, WS 2014/15 Fabian Kuhn 27
DFT Matrix Inverse
,
ℓ
ℓCase :
Algorithm Theory, WS 2014/15 Fabian Kuhn 28
Inverse DFT
•
⋯
⋯ ⋮ ⋯
• We get ⋅ and therefore
1⋯ ⋅ ⋮
1⋅ ⋅
Algorithm Theory, WS 2014/15 Fabian Kuhn 29
DFT and Inverse DFTInverse DFT:
1⋅ ⋅
• Define polynomial ⋯ :
1⋅
DFT:• Polynomial ⋯ :
Algorithm Theory, WS 2014/15 Fabian Kuhn 30
DFT and Inverse DFT
⋯ , ⋅ :
• Therefore:, , … ,
1⋅ , , , … ,
1⋅ , , , … ,
• Recall:DFT , , , … ,
⋅ , , , … , ,
Algorithm Theory, WS 2014/15 Fabian Kuhn 31
DFT and Inverse DFT• We have DFT ⋅ , , , … , , :
1⋅ DFT if 0
1⋅ DFT if 0
• DFT and inverse DFT can both be computed using FFT algorithm in log time.
• 2 polynomials of degr. can be multiplied in time log .
Algorithm Theory, WS 2014/15 Fabian Kuhn 32
Faster Polynomial Multiplication?
Idea to compute ⋅ (for polynomials of degree ):
, of degree 1, coefficients
2 2 point‐value pairs , and ,
2 point‐value pairs ,
of degree 2 2, 2 1 coefficients
Evaluation at , , … , using FFT
Point‐wise multiplication
Interpolation using FFT
Algorithm Theory, WS 2014/15 Fabian Kuhn 33
Convolution• More generally, the polynomial multiplication algorithm
computes the convolution of two vectors:
, , … ,, , … ,
∗ , , … , ,
where, :
,
• is exactly the coefficient of in the product polynomial of the polynomials defined by the coefficient vectors and
Algorithm Theory, WS 2014/15 Fabian Kuhn 34
More Applications of ConvolutionsSignal Processing Example:• Assume , … , represents a sequence of
measurements over time• Measurements might be noise and have to be smoothed out• Replace by weighted average of nearby last and next
measurements (e.g., Gaussian smoothing):
1⋅
• New vector ′ is the convolution of and the weight vector1⋅ , , … , , 1, , … , ,
• Might need to take care of boundary points…
Algorithm Theory, WS 2014/15 Fabian Kuhn 35
More Applications of ConvolutionsCombining Histograms:• Vectors and represent two histograms• E.g., annual income of all men & annual income of all women
• Goal: Get new histogram representing combined income of all possible pairs of men and women:
∗
Also, the DFT (and thus the FFT alg.) has many other applications!
Top Related