The Fourier & wavelet transforms in practiceApplication to the (non) detection of periodic signals

Frédéric Auchère frederic.auchere@ias.u-psud.frInstitut d’Astrophysique Spatiale, CNRS/ Université Paris-Sud, France

ESAC Data Analysis & Statistics Worshop – October 8, 2018, Villafranca, Spain

262

Level, requirements

o Who are you?

Master *********

PhD *********************

Phd+5 ********

PhD+10 **

PhD+++ ****

Lecture will be PhD level although there will be nuggets for the more advanced listener

o RequirementsBasic math, no prior knowledge of Fourier / wavelet analysis

Hands on sessions will be IDL but no knowledge required

362

Jean Baptiste Joseph Fourier(1768-1830) Fourier’s transform

‘‘The in-depth study of Nature isthe most fecund source of

discoveries in mathematics’’

462

Topics covered

1. Motivations2. Fourier analysis

2.1 The Fourier transform: definitions & basic properties2.2 The measurement process2.3 The continuous signal and its transform2.4 The discrete signal and its transform2.5 The effects of sampling, aliasing2.6 The discrete transform of the discrete signal2.7 How to treat the low frequencies2.8 The FFT2.9 The case of unenvenly space data

3. Wavelet analysis3.1 The wavelet transform: definitions & basic properties

4. Detection of periodic signals (Fourier & wavelets)4.1 Fourier transform of noise4.2 Side effects of detrending4.3 Local vs. Global confidence levels4.4 On the proper use of the Torrence & Compo wavelet code

Hands -on

562

Motivation

662

Some references

Many textbooks on Fourier analysis …

Bracewell, R. N. 1999, ‘‘The Fourier transform and its applications’’, 3rd Edition, McGraw Hill

Brault, J. W., & White, O. R. 1971, ‘‘The analysis and restoration of astronomical data via the FastFourier Transform’’, A&A, 13, 169

Torrence, C. & Compo, G. 1998, ‘‘A practical guide to wavelet analysis’’, BAMS, 79, 61

Scargle, J. D. 1982, ‘‘Studies in astronomical time-series analysis. II. Statistical aspects of spectral analysisof unevenly spaced data’’, ApJ, 263, 835

Gabriel, A. H., Baudin, F., Boumier, P. et al. 2002, ‘‘A search for solar g modes in the GOLF data’’, A&A, 390, 1119

Auchère, F., Froment, C., Bocchialini, K. et al. 2016, ‘‘On the Fourier and wavelet analysis of coronal time series’’, ApJ, 825, 110 + ERRATUM!

etc.

762

Fourier transform: definition & notations

o For a function 𝑠𝑠(𝑡𝑡) (𝑠𝑠 and its derivatives are supposed intergable)

o Several other common conventions (e.g. minus signs) and notations, e.g.

F 𝑠𝑠 ≡ F 𝑠𝑠 𝜈𝜈 ≡ F 𝑠𝑠 𝑡𝑡 ≡ ⋯ ≡ �̂�𝑠

Angular frequency : �̂�𝑠 𝜈𝜈 = ∫−∞+∞ 𝑠𝑠(𝑡𝑡)𝑒𝑒−𝑖𝑖𝜔𝜔𝑡𝑡𝑑𝑑𝑡𝑡

Fourier transform: �̂�𝑠 𝜈𝜈 = �−∞

+∞𝑠𝑠(𝑡𝑡)𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡

Inverse transform: 𝑠𝑠 𝑡𝑡 = �−∞

+∞�̂�𝑠 𝜈𝜈 𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝜈𝜈

862

o Units & duality:

o �̂�𝑠 𝜈𝜈 is a complex function (and so is 𝑠𝑠(𝑡𝑡), see above)o 𝐴𝐴 𝜈𝜈 = �̂�𝑠 𝜈𝜈 is the amplitude spectrum

o 𝜙𝜙 𝜈𝜈 = arg �̂�𝑠 𝜈𝜈 is the phase spectrum

o 𝑃𝑃 𝜈𝜈 = �̂�𝑠 𝜈𝜈 𝑖 is the power spectrum

o �̂�𝑠 𝜈𝜈 = Re �̂�𝑠 𝜈𝜈 + Im �̂�𝑠 𝜈𝜈o �̂�𝑠 𝜈𝜈 = �̂�𝑠 𝜈𝜈 𝑒𝑒𝑖𝑖𝑖𝑖(𝑖𝑖)

Fourier transform: definition & notations

Measurement domain Fourier domain

Time Inverse time (frequency)

Length Inverse length (wave number)

unit Inverse unit

Inverse unit unit

962

Fourier transform: basic properties

�−∞

+∞𝑠𝑠 𝑡𝑡 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡 = �

−∞

+∞𝑠𝑠 𝑡𝑡 cos −2𝜋𝜋𝜈𝜈𝑡𝑡 𝑑𝑑𝑡𝑡 + 𝑖𝑖 �

−∞

+∞𝑠𝑠 𝑡𝑡 sin −2𝜋𝜋𝜈𝜈𝑡𝑡 𝑑𝑑𝑡𝑡

If 𝑠𝑠 𝑡𝑡 ∈ ℝ (which is the case for physical signals)

𝑠𝑠 𝑡𝑡 = 𝑜𝑜 𝑡𝑡 + 𝑒𝑒 𝑡𝑡

⇒ �̂�𝑠 𝜈𝜈 = �𝑜𝑜 𝜈𝜈 + �̂�𝑒 𝜈𝜈

If 𝑠𝑠 𝑡𝑡 is even ⇒ �−∞

+∞𝑠𝑠 𝑡𝑡 sin −2𝜋𝜋𝜈𝜈𝑡𝑡 𝑑𝑑𝑡𝑡 = 0 ⇒ �̂�𝑠 𝜈𝜈 ∈ ℝ and even

If 𝑠𝑠 𝑡𝑡 is odd ⇒ �−∞

+∞𝑠𝑠 𝑡𝑡 cos −2𝜋𝜋𝜈𝜈𝑡𝑡 𝑑𝑑𝑡𝑡 = 0 ⇒ �̂�𝑠 𝜈𝜈 ∈ 𝕀𝕀 and odd

∈ ℝ and even ∈ 𝕀𝕀 and odd

If 𝑠𝑠 𝑡𝑡 ∈ ℝ ⇒ �̂�𝑠 𝜈𝜈 = �𝑜𝑜 𝜈𝜈 + �̂�𝑒 𝜈𝜈 = �𝑜𝑜 −𝜈𝜈 − �̂�𝑒 −𝜈𝜈 = �̂�𝑠 −𝜈𝜈 (conjugation symmetry)

1062

Fourier transform: basic properties

o Linearity : 𝑠𝑠 𝑡𝑡 = 𝑎𝑎𝑎𝑎 𝑡𝑡 + 𝑏𝑏𝑏𝑏 𝑡𝑡 ⇒ �̂�𝑠 𝜈𝜈 = 𝑎𝑎𝑎𝑎 𝜈𝜈 + 𝑏𝑏 �𝑏𝑏(𝜈𝜈)

o Translation / time shifting :

o Modulation / frequency shifting :

𝑠𝑠 𝑡𝑡 = 𝑎𝑎 𝑡𝑡 − 𝑡𝑡0 ⇒ �̂�𝑠 𝜈𝜈 = �−∞

+∞𝑎𝑎 𝑡𝑡 − 𝑡𝑡0 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡

= �−∞

+∞𝑎𝑎 𝑡𝑡′ 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖 𝑡𝑡′+𝑡𝑡0 𝑑𝑑𝑡𝑡′

= 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡0𝑎𝑎 𝜈𝜈

𝑠𝑠 𝑡𝑡 = 𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝑖𝑖0𝑡𝑡𝑎𝑎 𝑡𝑡 ⇒ �̂�𝑠 𝜈𝜈 = �−∞

+∞𝑎𝑎 𝑡𝑡 𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝑖𝑖0𝑡𝑡𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡

= 𝑎𝑎 𝜈𝜈 − 𝜈𝜈0

1162

Fourier transform: basic properties

o Time / frequency scaling :

Time reversal: a = −1 ⇒ 𝑠𝑠 𝑡𝑡 = 𝑎𝑎 −𝑡𝑡 ⇒ �̂�𝑠 𝜈𝜈 = 𝑎𝑎(−𝜈𝜈)

o Integration :

�̂�𝑠 0 = �−∞

+∞𝑎𝑎 𝑡𝑡 𝑑𝑑𝑡𝑡

𝑠𝑠 𝑡𝑡 = 𝑎𝑎 𝑎𝑎𝑡𝑡 ⇒ �̂�𝑠 𝜈𝜈 = �−∞

+∞𝑎𝑎 𝑎𝑎𝑡𝑡 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡

= �−+∞

±∞𝑎𝑎 𝑡𝑡′ 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖

𝑡𝑡′𝑎𝑎𝑑𝑑𝑡𝑡′

𝑎𝑎

=1𝑎𝑎 𝑎𝑎

𝜈𝜈𝑎𝑎

Time dilation ⇔ frequency contraction

Time contraction ⇔ frequency dilation

1262

Fourier transform of a derivative

�𝑠𝑠′ 𝜈𝜈 = �−∞

+∞𝑠𝑠′ 𝑡𝑡 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡 = �

−∞

+∞lim

𝑠𝑠 𝑡𝑡 + Δ𝑡𝑡 − 𝑠𝑠(𝑡𝑡)Δ𝑡𝑡

𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡

= limΔ𝑡𝑡→0

∫−∞+∞𝑠𝑠 𝑡𝑡 + Δ𝑡𝑡 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡 − ∫−∞

+∞𝑠𝑠 𝑡𝑡 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡Δ𝑡𝑡

= limΔ𝑡𝑡→0

𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝑖𝑖Δ𝑡𝑡�̂�𝑠 𝜈𝜈 − �̂�𝑠(𝜈𝜈)Δ𝑡𝑡

= �̂�𝑠 𝜈𝜈 limΔ𝑡𝑡→0

𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝑖𝑖Δ𝑡𝑡 − 1Δ𝑡𝑡

= 𝑖𝑖2𝜋𝜋𝜈𝜈�̂�𝑠 𝜈𝜈

⇒ �𝑠𝑠(𝑛𝑛) = 𝑖𝑖2𝜋𝜋𝜈𝜈 𝑛𝑛�̂�𝑠 𝜈𝜈

1362

Convolution theorem

�𝑎𝑎 ∗ 𝑏𝑏= �

−∞

+∞�−∞

+∞𝑎𝑎 𝜏𝜏 𝑏𝑏 𝑡𝑡 − 𝜏𝜏 𝑑𝑑𝜏𝜏 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡

= �−∞

+∞𝑎𝑎 𝜏𝜏 �

−∞

+∞𝑏𝑏 𝑡𝑡 − 𝜏𝜏 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡 𝑑𝑑𝜏𝜏

= �−∞

+∞𝑎𝑎 𝜏𝜏 �

−∞

+∞𝑏𝑏 𝑡𝑡′ 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖(𝑡𝑡′+𝜏𝜏)𝑑𝑑𝑡𝑡′ 𝑑𝑑𝜏𝜏

= �−∞

+∞𝑎𝑎 𝜏𝜏 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝜏𝜏 �𝑏𝑏(𝜈𝜈) 𝑑𝑑𝜏𝜏

= 𝑎𝑎 𝜈𝜈 . �𝑏𝑏(𝜈𝜈)

Likewise, �𝑎𝑎.𝑏𝑏 = 𝑎𝑎 𝜈𝜈 ∗ �𝑏𝑏(𝜈𝜈)

o The Fourier transform translates between multiplication and convolution

o In combination with the linearity and time-shifting properties, the convoutiontheorem can be used to derive the Fourier transform of many usual signals

1462

Parseval’s theorem

𝑠𝑠 𝑡𝑡 ∈ ℝ and 𝑠𝑠 𝑡𝑡 = 𝑎𝑎(𝑡𝑡). 𝑎𝑎(𝑡𝑡)

�̂�𝑠 𝜈𝜈 = �−∞

+∞𝑎𝑎𝑖 𝑡𝑡 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡

= 𝑎𝑎 𝜈𝜈 ∗ 𝑎𝑎(𝜈𝜈) (from the convolution theorem)

= �−∞

+∞𝑎𝑎 𝜈𝜈 𝑎𝑎(𝜈𝜈 − 𝜎𝜎)𝑑𝑑𝜎𝜎

𝜈𝜈 = 0 ⇒ �−∞

+∞𝑎𝑎𝑖 𝑡𝑡 𝑑𝑑𝑡𝑡 = �

−∞

+∞𝑎𝑎 𝜈𝜈 𝑎𝑎(−𝜎𝜎)𝑑𝑑𝜎𝜎

⇒ �−∞

+∞𝑎𝑎𝑖 𝑡𝑡 𝑑𝑑𝑡𝑡 = �

−∞

+∞𝑎𝑎(𝜈𝜈)

𝑖𝑑𝑑𝜈𝜈

o Physics: the average power in the signal is equal to the average power in its transform

o Signal: the variance of the signal is equal to the mean power (more on this later)

1562

Fourier transform of common functions / distributions

𝑠𝑠 𝑡𝑡 = 𝛿𝛿0(𝑡𝑡)

⇒ �̂�𝑠 𝜈𝜈 = �−∞

+∞𝛿𝛿0(𝑡𝑡) 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡

= 1

�̂�𝑠 𝜈𝜈 = 𝛿𝛿0(𝜈𝜈)

⇒ 𝑠𝑠 𝑡𝑡 = 1�−∞

+∞𝛿𝛿0(𝜈𝜈) 𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝜈𝜈

= 1

𝑠𝑠 𝑡𝑡 = 1⇒ �̂�𝑠(𝑡𝑡) = 𝛿𝛿0

https://en.wikipedia.org/wiki/Fourier_transform

1662

Fourier transform of common functions / distributions

𝑠𝑠 𝑡𝑡 = 𝛿𝛿𝑇𝑇(𝑡𝑡)

⇒ �̂�𝑠 𝜈𝜈 = �−∞

+∞𝛿𝛿(𝑡𝑡 − 𝑇𝑇) 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡

= 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑇𝑇

�̂�𝑠 𝜈𝜈 = 𝛿𝛿𝛮𝛮(𝜈𝜈)

⇒ 𝑠𝑠 𝑡𝑡 = �−∞

+∞𝛿𝛿(𝜈𝜈 − 𝛮𝛮) 𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝜈𝜈

= 𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝛮𝛮𝑡𝑡

𝑠𝑠 𝑡𝑡 = 𝑒𝑒𝑖𝑖𝑖𝑖𝑖Ν𝑡𝑡

⇒ �̂�𝑠(𝑡𝑡) = 𝛿𝛿(𝜈𝜈 − 𝛮𝛮)

1762

Fourier transform of common functions / distributions

𝑠𝑠 𝑡𝑡 = 𝛿𝛿𝑡𝑡0(𝑡𝑡)

⇒ �̂�𝑠 𝜈𝜈 = �−∞

+∞𝛿𝛿(𝑡𝑡 − 𝑡𝑡0) 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡

= 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡0

𝑠𝑠 𝑡𝑡 = 𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝑖𝑖0𝑡𝑡

⇒ �̂�𝑠(𝑡𝑡) = 𝛿𝛿(𝜈𝜈 − 𝜈𝜈0)

𝑠𝑠 𝑡𝑡 = cos(2𝜋𝜋𝜈𝜈0𝑡𝑡)

⇒ �̂�𝑠(𝑡𝑡) =12 𝛿𝛿 𝜈𝜈 + 𝜈𝜈0 + 𝛿𝛿(𝜈𝜈 − 𝜈𝜈0)

𝑠𝑠 𝑡𝑡 = sin(2𝜋𝜋𝜈𝜈0𝑡𝑡)

⇒ �̂�𝑠(𝑡𝑡) =12𝑖𝑖 𝛿𝛿 𝜈𝜈 − 𝜈𝜈0 − 𝛿𝛿(𝜈𝜈 + 𝜈𝜈0)

1862

Fourier transform of common functions / distributions

𝑠𝑠 𝑡𝑡 = �1 if 𝑥𝑥 < 𝑇𝑇/20 if 𝑥𝑥 > 𝑇𝑇/2

⇒ �̂�𝑠 𝜈𝜈 = �−𝑇𝑇/𝑖

+𝑇𝑇/𝑖𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡

=𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡

𝑖𝑖2𝜋𝜋𝜈𝜈−𝑇𝑇/𝑖

𝑇𝑇/𝑖

=𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑇𝑇 − 𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝑖𝑇𝑇

𝑖𝑖2𝜋𝜋𝜈𝜈

= 𝑇𝑇sin(𝜋𝜋𝜈𝜈𝑇𝑇)𝜋𝜋𝜈𝜈𝑇𝑇

= 𝑇𝑇sinc(𝜋𝜋𝜈𝜈𝑇𝑇)

1962

Fourier transform of common functions / distributions

𝑠𝑠 𝑡𝑡 = 𝑒𝑒−𝑎𝑎𝑡𝑡2

⇒ �̂�𝑠 𝜈𝜈 = �−∞

+∞𝑒𝑒−𝑎𝑎𝑡𝑡2𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡

= �−∞

+∞𝑒𝑒− 𝑎𝑎𝑡𝑡2+𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡 𝑑𝑑𝑡𝑡

= 𝑒𝑒−𝑖𝑖2𝑖𝑖2𝑎𝑎 �

−∞

+∞𝑒𝑒− 𝑎𝑎𝑡𝑡2+𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡−𝑖𝑖

2𝑖𝑖2𝑎𝑎 𝑑𝑑𝑡𝑡

= 𝑒𝑒−𝑖𝑖2𝑖𝑖2𝑎𝑎 �

−∞

+∞𝑒𝑒− 𝑎𝑎𝑡𝑡+𝑖𝑖𝑖𝑖𝑖𝑖𝑎𝑎

2

𝑑𝑑𝑡𝑡

=𝜋𝜋𝑎𝑎 𝑒𝑒

−𝑖𝑖2𝑖𝑖2𝑎𝑎

2062

Use the basic properties to understand your signal and its transform!

𝑎𝑎 𝑡𝑡 = �𝑚𝑚=0

𝑀𝑀−1

𝑎𝑎𝑚𝑚𝑝𝑝(𝑡𝑡 − 𝑚𝑚𝑇𝑇)

𝜓𝜓 𝜈𝜈 = 𝑃𝑃(𝜈𝜈) 𝑖 𝑀𝑀𝜎𝜎𝑖 + 𝜇𝜇𝑖sin 𝜋𝜋𝜈𝜈𝑇𝑇𝑀𝑀sin 𝜋𝜋𝜈𝜈𝑇𝑇

𝑖

𝑀𝑀 pulses equally spaced by 𝑇𝑇

Random amplitudes

Individual pulse

(variance 𝜎𝜎𝑖, mean 𝜇𝜇)

× {Continuum + discrete harmonics}Individual

pulse power

Expected Fourier spectra for Gaussianand double exponential pulses

o Analytic expressions of Fourier transforms are often available even for complex signalso Example: random pulse train

o Well know result in telecommunications! (see also Auchère et al. 2018, ApJ, 827, 152)

2162

The measurement process

o Physical measurements are in general a sample, discrete and finite in space and time, of an underlying signal that is more extended in time and space.o Light curve of a star between 𝑡𝑡1 and 𝑡𝑡𝑖 every Δ𝑡𝑡o Spectrum of a star between 𝜆𝜆1 and 𝜆𝜆𝑖 every Δ𝜆𝜆o Image of an object between (𝜃𝜃1, 𝜃𝜃𝑖, 𝜙𝜙1, 𝜙𝜙𝑖) at the angular resolution Δ𝜃𝜃, Δ𝜙𝜙o Etc.

o The signal has intrinsic cutoff frequencies in time and space: SDo Characteristic times of the physical processes (e.g. cooling time)o Natural width of spectral lineso Etc.

o The measurement apparatus introduces noise and thus frequencies potentially higher than the intrinsic frequencies

o The measurement apparatus itself has intrinsic cutoff frequencies: SC

o Finite response timeo Finite integration timeo Spatial or spectral resolution (e.g. Airy disk)o Etc.

2262

The continuous signal and its transform

o Let’s assume a continuous signal, e.g. the spectrum of a staro The transform appears ‘noisy’: beats dues to the various components required to

reproduce the signalo The transform is symmetrical (more on this later)o The transform is zero above a given cutoff frequency SD.o In a well-designed experiment, one has SC > SD

o The noise expands the range of frequencies over which the transform is non zero

2362

The discrete signal and its transform

o We now sample the signal regularlyo One value every Δ𝑡𝑡 or one value every Δ𝜃𝜃, Δ𝜙𝜙, etc.o The sampled signal is 𝑠𝑠𝑠𝑠 = 𝑠𝑠𝑐𝑐ШΔ𝑥𝑥 (Ш = «Dirac comb», “sha” Cyrillic letter)

o The continuous transform of this signal is thus:

o The sampling induces a replication of the transform every Δ𝜈𝜈 = 1/Δ𝑥𝑥

o We have �Ш𝑎𝑎 = Ш1/𝑎𝑎 and �𝑎𝑎 .𝑏𝑏 = �𝑎𝑎 ∗ �𝑏𝑏

o Thus �𝑠𝑠𝑠𝑠 = �𝑠𝑠𝑐𝑐 ∗�ШΔ𝑥𝑥 = �𝑠𝑠𝑐𝑐 ∗ Ш1/Δ𝑥𝑥

o The continuous transform of a discrete signal is thus periodic!o One replica (alias) centered on each peak of the Dirac comb.

o Other way: the above expression is that of a Fourier series ! And these represent periodic functions.

𝑠𝑠(𝑥𝑥𝑗𝑗): signalN: number of samples𝑥𝑥𝑗𝑗 = 𝑗𝑗Δ𝑥𝑥

�𝑠𝑠𝑠𝑠 𝜈𝜈 = �𝑗𝑗=0

𝑁𝑁−1

𝑠𝑠𝑐𝑐(𝑥𝑥)𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑥𝑥𝑗𝑗

2462

The sampling theorem

o A bad sampling choice can lead to the superimposition of the replicas (aliases). The transform is then « aliased »

o The frequency in between two replicas is called the Nyquist frequency: SNy = 0.5/Δ𝑥𝑥o The sampling theorem

o 1/Δ𝑥𝑥 must be greater or equal to twice the maximum frequency Sc contained in the signal

o If this condition is verified, the discrete samples contain all the information contained in the continuous signal

2562

Aliasing is back!

o Beware of noise!o The measurement noise is broad-bando If possible a low-pass filter must be included in the instrumento Hard to escape to photon noise

o The measurements are finite in space and timeo Represented by the multiplication of the signal by a rectangular function Π(x)o Produces high frequencies due to the discontinuity between the beginning and end of

the measuremento The transform can be aliased even if the sampling is sufficient wrt sampling theorem!

2662

Aliasing still!

o Due to the finite nature of the measurement, the transform is convolved by a sinc functiono 𝑠𝑠 = 𝑠𝑠0ΠΔ𝑥𝑥 → �̂�𝑠 = �𝑠𝑠0 ∗ �ΠΔ𝑥𝑥 → �̂�𝑠 = �𝑠𝑠0 ∗ sinc1/𝑁𝑁Δ𝑥𝑥

o The practical transform can’t be identical to the transform of the underlying signal even in the limit of a continuous signalo We will see how to deal with that

o The longer the observing time / the FOV extended / etc., the higher the frequency resolution: 𝛿𝛿𝜈𝜈 = 1/𝑁𝑁Δ𝑥𝑥

2762

Aliasing (the end)

o Aliasing: the transform at frequency 𝜈𝜈 is contaminated by frequencies > Nyquisto The Fourier representation of undersampled data is ambiguous:

o The 5 right-hand side components can’t be distinguished from the lefto The problem can’t be solved by calculating the transform at frequencies > Nyquisto There is no additional information beyond Nyquisto Did I say that there is no additional information beyond Nyquist?o One last time: there is no additional information beyond Nyquist

2862

DFT (Discrete Fourier Transform)

o Given the length 𝑁𝑁Δ𝑥𝑥 of the measurement, the frequency resolution is 1/𝑁𝑁Δ𝑥𝑥o We thus evaluate the transform at 𝑁𝑁 discrete frequencies separated by Δ𝜈𝜈 = 1/𝑁𝑁Δ𝑥𝑥

�̂�𝑠 𝜈𝜈𝑘𝑘 = �𝑗𝑗=0

𝑁𝑁−1

𝑠𝑠(𝑥𝑥𝑗𝑗)𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑘𝑘𝑥𝑥𝑗𝑗

o The inverse transform is

𝑠𝑠 𝑥𝑥𝑗𝑗 = �𝑘𝑘=0

𝑁𝑁−1

�̂�𝑠(𝜈𝜈𝑘𝑘)𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑘𝑘𝑥𝑥𝑗𝑗

o The transform of a discrete signal is periodic ↔ the underlying signal itself is periodic !

2962

Symmetry properties

o The N measured values are realo The transform is composed of 𝑁𝑁 complex valueso Symmetry with respect to the origin and Nyquist

o �̂�𝑠 𝜈𝜈 = �̂�𝑠 −𝜈𝜈 ∗

o �̂�𝑠 𝜈𝜈𝑁𝑁𝑁𝑁 + Δ𝜈𝜈 = �̂�𝑠 𝜈𝜈𝑁𝑁𝑁𝑁 − Δ𝜈𝜈 ∗

o The imaginary part of the coefficients at the origin and at Nyquist is zero

o There are 𝑁𝑁 independent Fourier coefficientso 𝑁𝑁/2 + 1 realo 𝑁𝑁/2 − 1 imaginary

3062

Dealing with low frequencies (if necessary!)

o Subtract the mean of the signal first!

o The mean is simply the value of the transform at the origin

o Using zero-mean signal simplifies the treatment of the edges

o Low frequency trends produce a peak near the origin. Its intensity is redistributed by the window function (convolution by sinc for a square window)

o Possible to de-trend the data prior to Fourier analysiso But beware of nasty side effects !o More on this later …

3162

Treating the edges

o The discontinuities at the edged create aliasingo The is minimized by forcing a smooth transition at the edgeso Apodization of the signal with a window functiono Equivalent to replace the square window (finite length of the measurements) by a window

transitioning smoothly to zero: 𝑠𝑠 𝑡𝑡 = 𝑠𝑠0 𝑡𝑡 ∗ 𝑊𝑊(𝑡𝑡) → �̂�𝑠 𝜈𝜈 = �̂�𝑠0 𝜈𝜈 ∗ �𝑊𝑊 𝜈𝜈o There is a large number of possible windows, each with advantages and drawbacks

o Generally, a wide window will have a good frequency resolution but high lobes,o and vice versa

3262

Treating the edges

o Hann

o Tukey

3362

Summary

o Transform �̂�𝑠 𝜈𝜈𝑘𝑘 = ∑𝑗𝑗=0𝑁𝑁−1 𝑠𝑠(𝑥𝑥)𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑘𝑘𝑥𝑥𝑗𝑗 and its inverse 𝑠𝑠 𝑥𝑥𝑗𝑗 = ∑𝑘𝑘=0𝑁𝑁−1 �̂�𝑠(𝜈𝜈𝑘𝑘)𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑘𝑘𝑥𝑥𝑗𝑗

o The Fourier transform is periodic and symmetric with respect to the origin and Nyquist

o The measurement is one period of an infinite, periodic signal

o If possible the instrument must have a low pass filter and the signal must be sampled at high enough frequency to avoid aliasing: 1/Δx > 2𝑆𝑆𝑐𝑐

o The mean should be subtracted. Low frequency trends can be subtracted (if the application requires it, see later)

o If necessary, a window can be applied to minimize the impact of discontinuities at the edges

3462

What about unenvenly sampled data?

o The definition of the discrete Fourier transform still holds for unevenly spaced data

�̂�𝑠 𝜈𝜈𝑘𝑘 = �𝑗𝑗=0

𝑁𝑁−1

𝑠𝑠(𝑥𝑥𝑗𝑗)𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑘𝑘𝑥𝑥𝑗𝑗

o The periodogram is the power spectrum of this evevenly spaced data (Scargle, 1981, 1982)

o Please READ the papers before using ito The periodogram is not a magical tool that will let you violate information theoryo You can’t beat Shannon (no information beyond Nyquist)o The smoother peaks contain no additional information (merely interpolations)o Choice of optimal frequencies will depend on the observation window

o The Scargle periodogram has the same statistical properties as for evenly spaced data

o If the sampling is not pathological, then the detection statistics presented below is valid

3562

The Fast Fourier Transform

o FFT = Fast Fourier Transform

o The FFT computes exactly and efficiently the DFTo The acronym FFT is often used (wrongly) as synonymous of FTo Directly computing the DFT requires O(N2) operationso The FFT does the same in O(N log N) operationso The gain is of the order of N / log(N): N=10 gain 10, N=106 gain >105, etc.o The FFT is also more accurate than the direct evaluation of the DFT

o Several FFT algorithms existo Cooley-Tukey, Prime-factor, Bruun, Rader, etc.o Dedicated algorithmes for real data, and / or symmetric

o Cooley-Tukeyo Published by J. W. Cooley and J. W. Tukey in 1965o In fact already known by Gauss in 1805 …o Maximum error O(ε log N) instead of O(εN3/2) for straight DFTo RMS error in O(ε √log N) for Cooley–Tukey and O(ε √N) for the DFT

3662

Applications: filtering, convolutio, deconvolution, compression, etc. etc. etc. etc.

o Frequency filtering is very easy to implement in the Fourier space:𝑠𝑠 𝑡𝑡 = 𝑠𝑠0 𝑡𝑡 𝐹𝐹 → �̂�𝑠 𝜈𝜈 = �𝑠𝑠0 𝜈𝜈 ∗ �𝐹𝐹 𝜈𝜈See convolution in the Hands on session

o Visualizing of the power spectrum is a good way to choose the filter

3762

Wavelet analysis

o Will use the code by Torrence, C. & Compo, G. P. 1998, BAMS, 79(1), 61-78o Reasons for success (2602 citations, >0.5 cite/day)

o Practical guide to wavelet analysis (very good paper, READ it!)o Provides quantitative confidence levels for wavelet spectrao Ready to use IDL code (can be used as a black box: dangerous!)o Produces compelling graphics

o But use it properly!o Global confidence levelso Proper background noise model

3862

Wavelet transform

o If the signal is non stationary, how to have information on both the frequency and the location?o Crude approach: run Fourier transforms over a

running segment and vary the segment length →Windowed Fourier Transform

o If resolution in time no resolution in frequency

o Need for a scale independent method → wavelets:

o Ψ0 𝜂𝜂 = 𝜋𝜋−1/4𝑒𝑒𝑖𝑖𝜔𝜔0𝜂𝜂𝑒𝑒−𝜂𝜂2/𝑖 (Morlet)𝜂𝜂 = 𝑡𝑡/𝑠𝑠 (dimensionless ‘‘time’’)

o Ψ0 𝜂𝜂 must have zero mean and be bounded in time and frequency

o Wavelet can be real or complex functions

3962

Wavelet transform

o The continuous wavelet transform of a discrete signal (TC98 notations), defined as

𝑊𝑊𝑛𝑛 𝑠𝑠 = �𝑛𝑛′=0

𝑁𝑁−1

𝑥𝑥𝑛𝑛′ Ψ∗ 𝑛𝑛′ − 𝑛𝑛 𝛿𝛿𝑡𝑡𝑠𝑠

is the convolution of 𝑥𝑥𝑛𝑛 with a scaled and translated (and normalized) version of Ψ0 𝜂𝜂

o Convolution theorem → the wavelet transform can be easily computed in Fourier space

𝑊𝑊𝑛𝑛 𝑠𝑠 = �𝑘𝑘=0

𝑁𝑁−1

�𝑥𝑥𝑘𝑘 �Ψ∗ 𝑠𝑠𝜔𝜔𝑘𝑘 𝑒𝑒𝑖𝑖𝜔𝜔𝑘𝑘𝑛𝑛𝑛𝑛𝑡𝑡

𝜔𝜔𝑘𝑘 = ± 𝑖𝑖𝑖𝑘𝑘𝑁𝑁𝑛𝑛𝑡𝑡

𝑘𝑘 ≤ 𝑁𝑁𝑖

, 𝑘𝑘 > 𝑁𝑁𝑖

o Wavelet power spectrumo The wavelet trasnform is imaginary and by analogy with the Fourier transform one

defines real part, imaginary part, amplitude, phase, and power

𝑃𝑃𝑛𝑛 𝑠𝑠 = 𝑊𝑊𝑛𝑛(𝑠𝑠) 𝑖

4062

Which one to chose? Depends on the application!

4162

Power spectrum normalization (Fourier or wavelets)

o It is useful to be able to compare different power spectrao Remember Parseval in Fourier

�−∞

+∞𝑎𝑎𝑖 𝑡𝑡 𝑑𝑑𝑡𝑡 =�

−∞

+∞𝑎𝑎(𝜈𝜈)

𝑖𝑑𝑑𝜈𝜈

o Discrete equivalent for a zero-mean signal of variance 𝜎𝜎0𝑖:

1𝑁𝑁�

𝑖𝑖=0

𝑁𝑁−1

𝑠𝑠 𝑥𝑥𝑖𝑖 𝑖 = 𝜎𝜎0𝑖 =1𝑁𝑁�

𝑖𝑖=0

𝑁𝑁−1

�̂�𝑠(𝜈𝜈𝑖𝑖) 𝑖

o We have equivalent expressions for wavelets (the total energy is conserved under wavelet transform)

o Since the expectation value of white noise is 𝜎𝜎0𝑖at all frequencies (and at all scales for wavelets), we normalize all power spectra to 𝜎𝜎0𝑖 (i.e. multiply the IDL Fourier spectrum by 𝑁𝑁/𝜎𝜎0𝑖, but check the definition of YOUR Fourier transform). This gives a measure of power relative to white noise. This is the convention used in TC98.

4262

o A Fourier or wavelet spectrum shows a bunch of peakso Which ones are significant?

o How to define confidence levels?

o Robust quantitative detection is importanto Many oscillatory phenomena have been reported in the astrophysics literature

o In the solar corona: periods from sub-second to tens of hours, in all types of structures

o Several authors have questioned the soundness of some of these detections

o Cenko et al. 2010, Gruber et al. 2011, Inglis et al. 2015, Ireland et al. 2015, Auchère et al. 2016, Threlfall et al. 2017

o The methods and / or assumptions used in several papers are flawed/wrong

o Beware of your brain / eyes: ‘‘I can see the pattern’’ is not a statistical proof!

o Preconceptions (“there must be periods in my data”)

o Unconscious biases

o Optical illusions

o It’s OK not to find periods !o A negative result often tells more than a positive one

Search for periodic components in a noisy signal

4362

cf. Stephen Jay Gould (1991) on the glow-worms in the Waitomo cave of New Zealand

To find a periodic signal is to find a pattern in randomness. Which one is random?

4462

Fourier transform of noise (stationary process, constant variance)

o The Fourier transform of Gaussian noise is white (same expected value at all frequencies)o The PSD of noise is distributed as 𝜒𝜒𝑖𝑖 at each frequency:

o Both real and imaginary parts are Gaussian distributedo If 𝑍𝑍1,𝑍𝑍𝑖 … ,𝑍𝑍𝑘𝑘 are 𝑘𝑘 independent, standard, normal random variables, then:

𝑄𝑄 = ∑𝑖𝑖=1𝑘𝑘 𝑍𝑍𝑖𝑖𝑖 ~𝜒𝜒𝑘𝑘𝑖

o Taking into account normalization factors: 𝑃𝑃 𝑝𝑝 = 𝑒𝑒− 𝑝𝑝𝜎𝜎02

o Most probable value is 0 while the mean is 𝜎𝜎0𝑖

4562

Confidence levels: Local vs. Global (e.g. Scargle 1982, Gabriel et al. 2002)

Red noiseWhite noise

N =

256

N =

819

2

Local confidence levels𝑃𝑃(𝑚𝑚) = 𝑒𝑒−𝑚𝑚𝑚𝑚 = − ln 𝑃𝑃95% confidence level

𝑃𝑃 = 0.05𝑚𝑚 = 3.0

Global confidence levels𝑃𝑃𝑔𝑔(𝑚𝑚) = 1 − 1 − 𝑃𝑃(𝑚𝑚) ⁄𝑁𝑁 𝑖

𝑃𝑃𝑔𝑔(𝑚𝑚) = 1 − 1 − 𝑒𝑒−𝑚𝑚 ⁄𝑁𝑁 𝑖

𝑚𝑚 = − ln 1 − 1 − 𝑃𝑃𝑔𝑔⁄𝑖 𝑁𝑁

95% confidence level:𝑃𝑃 = 0.05𝑁𝑁 𝑚𝑚

256 7.8

8192 11.3

4662

How to find periods in any time series in 3 easy steps with the TC98 code

True El Nino time series Fake El Nino time series

1. Take a signal that has a power law Fourier spectrum (basically any Solar time series)2. De-trend using a running boxcar filter3. Apply blindly the (very good) Torrence & Compo wavelet package

or more generally do not estimate properly the background poweror trust your eyes/brain

What happens?

4762

Important observation

Irel

and,

J. e

t al.

2015

, ApJ

, 798

, 1

Power laws (or similar) are everywhere: −1.3 < α < −4.

4862 1st problem: combined with a power law, detrending produces a colored signal

𝑠𝑠𝑑𝑑 =𝑆𝑆0(𝑡𝑡)

𝑆𝑆0(𝑡𝑡) ∗ 𝑎𝑎(𝑡𝑡) − 1 ≈𝛿𝛿𝑆𝑆0 𝑡𝑡 − 𝛿𝛿𝑆𝑆0 𝑡𝑡 ∗ 𝑎𝑎(𝑡𝑡)

𝑆𝑆0(𝑡𝑡)

Effects of detrending

Foul

lon

et a

l. 20

009,

ApJ

, 700

, 165

8

4962

Detrending of a time series whose PSD is a power-law

1∆𝑇𝑇

4 + 𝑠𝑠2Maximum ~

Ψ 𝜐𝜐 = 𝑎𝑎𝜐𝜐𝑠𝑠 1 −sin 𝜋𝜋𝜐𝜐Δ𝑡𝑡𝜋𝜋𝜐𝜐Δ𝑡𝑡

𝑖

𝑠𝑠𝑑𝑑 =𝑆𝑆0(𝑡𝑡)

𝑆𝑆0(𝑡𝑡) ∗ 𝑎𝑎(𝑡𝑡)− 1 ≈

𝛿𝛿𝑆𝑆0 𝑡𝑡 − 𝛿𝛿𝑆𝑆0 𝑡𝑡 ∗ 𝑎𝑎(𝑡𝑡)𝑆𝑆0(𝑡𝑡)

5062

On simulated data, α=-2

Obvious pulsations, right?

5162

The detrended signal IS narrowband (while the original IS NOT!)

Obvious pulsations, right ?

2nd problem: white noise is NOT a pertinent background model

5262

Periods tend to be found close to the cutoff of the detrending filter…Object Analysis Noise model Data Detrending Filter width Period

EUV filament Wavelets (TC) red EIT195 running boxcar (M) 20h 10-20h

Full disk EUV Wavelets (TC) red SEM304 running boxcar (M) 14h 12-15h

EUV filament Fourier white EIT195 running boxcar (M) 14h 12h

Full disk EUV Fourier white SEM304 running boxcar (M) 14h 12h

EUV filament Fourier white EIT195 running boxcar (M) 50h 28-40h

Full disk EUV Fourier white SEM304 running boxcar (M) 50h 28-40h

EUV filament Wavelets (TC) red EIT195 running boxcar (M) 14h 8-16h

EUV filament Fourier white EIT195 running boxcar (M) 14h 12h

QPP Fourier white NoRH17 running boxcar (S) 20s 16s

QPP Fourier white SXT running boxcar (S) 20s 16s

none Fourier white Red noise simulations running boxcar (S) 50s 33-100s

QPP Visual running boxcar (S) 20s 1-30s

Coronal bright points Wavelets (TC) white ? TRACE1216 & EIT195 smoothed running difference 3.5m 8-64m

Coronal bright points Wavelets (TC) white? SUMER SVI933 running boxcar (S) 1500s 491s

Coronal hole Wavelets (TC) white EIS HeII Fe XI Fe XII runnning boxcar ? 600s 394-860s

Magnetic flux asymmetry Wavelets (TC), Fourier white NOAA magnetograms running boxcar ?

None ? 1yr? 1.5-3.6yr

Loops, QS BP Wavelets (TC) white? EIS Fe XIII I, v running boxcar (S) 10 min 4.5-13.8 min

Loops Wavelets (TC) white EIS Fe XII v running boxcar (S?) 10 min 4-6 min

XBP Wavelets (TC) ?

QPPs Fourier white XRT ? 28-60 min

3th problem: AR(1) noise model of TC98 is not a good model either

5362 4th problem: used confidence levels are usually local

AR(1)𝑥𝑥𝑛𝑛= 𝛼𝛼𝑥𝑥𝑛𝑛−1 + 𝑧𝑧𝑛𝑛

PSD1 − 𝛼𝛼𝑖

1 + 𝛼𝛼𝑖 − 2𝛼𝛼cos 2𝜋𝜋𝜐𝜐𝛿𝛿𝑡𝑡

𝛼𝛼 determined from the data

If 𝛼𝛼 ≈ 1 and 𝜐𝜐 ≪ 1

⇒ 𝑃𝑃𝑆𝑆𝑃𝑃 ≈1 − 𝛼𝛼

2 𝜋𝜋𝜐𝜐𝛿𝛿𝑡𝑡 𝑖

But TC98 cade CAN be used with any background noise model! (see hands-on)

TC98 red noise model on simulated data, α=-3

5462

Local confidence levels (as done in TC98)

o In the wavelet spectrumo The power at each point is 𝜒𝜒𝑖𝑖 distributedo Monte Carlo simulations of white & AR(1) red noise

5562

Local confidence levels (as done in TC98)

o In the time-averaged wavelet spectrumo The power at each point is not 𝜒𝜒𝑖𝑖 distributedo Sum of a number of correlated 𝜒𝜒𝑖𝑖 distributed variableso The number and correlation depends on the scaleo Number of degrees of freedom determined from Monte-Carlo simulations

𝜈𝜈 = 2 1 + 𝑛𝑛𝑎𝑎𝑛𝑛𝑡𝑡𝛾𝛾𝑠𝑠

𝑖

5662

How significant are extended peaks of wavelet power?

o For an event or probability 𝑝𝑝, the probability to get 𝑘𝑘 successes in 𝑙𝑙 trials is

𝑃𝑃𝑏𝑏 𝑘𝑘; 𝑙𝑙, 𝑝𝑝 =𝑙𝑙!

𝑘𝑘! 𝑙𝑙 − 𝑘𝑘 !𝑝𝑝𝑘𝑘 1 − 𝑝𝑝 𝑙𝑙−𝑘𝑘

o Example with 95% confidence levels (𝑝𝑝 = 0.05)o TC98 give the degree of freedom at each scale: 𝑙𝑙 = 4.5o Length of the structure 𝑘𝑘 = 1.8o → 𝑃𝑃𝑏𝑏~2 × 10−𝑖(upper boundary due to binomial distribution approximation)o Local probability in the time-averaged spectrum 10−3

5762

Global confidence levels: wavelet spectrum

o Generalization of the Fourier global confidence levels

o The bins of the wavelet spectrum are notindependento Intuitively, 𝑛𝑛 should be the number of degrees

of freedom in the spectrum, outside the COI, normalized by the scale resolution 𝛿𝛿𝑗𝑗

o 𝑎𝑎 should be close to 1o 𝑎𝑎 and 𝑛𝑛 are empirically derived

o Monte-Carlo simulationso 100 000 white noise time-serieso For each couple 𝑁𝑁, 𝛿𝛿𝑗𝑗:

o For each local probability 𝑃𝑃 𝑚𝑚 ,compute the global probability 𝑃𝑃𝑔𝑔(𝑚𝑚)

o Fit the resulting data (Morlet)𝑎𝑎 = 0.801 𝑁𝑁𝑜𝑜𝑜𝑜𝑡𝑡𝛿𝛿𝑗𝑗 0.01

𝑛𝑛 = 0.491 𝑁𝑁𝑜𝑜𝑜𝑜𝑡𝑡𝛿𝛿𝑗𝑗 0.9𝑖6

𝑃𝑃𝑔𝑔 𝑚𝑚 = 1 − 1 − 𝑃𝑃 𝑚𝑚𝑁𝑁𝑖

→ 𝑃𝑃𝑔𝑔(𝑚𝑚) = 1 − 1 − 𝑃𝑃(𝑚𝑚)𝑎𝑎 𝑛𝑛

5862

Global confidence levels: time-averaged wavelet spectrum

o Same principle for the time-averaged wavelet spectrumo Number of degrees of freedom a function of scaleo But probability is relative to the expected power at each scale

𝑃𝑃𝑔𝑔(𝑚𝑚) = 1 − 1 − 𝑃𝑃(𝑚𝑚)𝑎𝑎 𝑛𝑛

o Monte-Carlo simulationso Compute 𝑃𝑃𝑔𝑔(𝑚𝑚) for varying 𝑃𝑃(𝑚𝑚)o Fit the resulting data (Morlet)

𝑎𝑎 = 0.805 + 0.45 × 2−𝑆𝑆𝑜𝑜𝑜𝑜𝑜𝑜𝑛𝑛𝑗𝑗𝑛𝑛 = 1.136 𝑆𝑆𝑜𝑜𝑜𝑜𝑡𝑡𝛿𝛿𝑗𝑗 1.𝑖

5962

All this in practice (no detrending + custom noise model + global confidence levels)

𝜎𝜎 𝜈𝜈 = 𝑎𝑎𝜐𝜐𝑠𝑠 + Κ𝜌𝜌 𝜐𝜐 + 𝐶𝐶

Demonstration code: https://idoc.ias.usud.fr/MEDOC/wavelets_tc98

+ photon noise+ pulseBackground activity

The period seems evident in that case, but much can be learned from the understanding of the spectral shape → train of random pulses, not a wave!

6062

Are these confidence levels too harsh?

o The short answer: no (even if they can be disappointing)

o Bias of the TC98 code (higher peak if the signal is low frequency, see their website)o Automatically taken care of by the Monte-Carlo derived confidence levels.

o Peaks are blobs, not single pixels, and correlation higher at low frequency, thus more likely to have many bins above a threshold at low frequency; thus global wavelet levels should be scale dependento No, a “global detection” is defined as a whole spectrum with at least one bin above a

threshold. Does not matter if a blob or a bin.o Automatically taken care of by Monte-Carlo

6162

Analysis of an extended field of view

o Compare only normalized power values! (i.e. whitened, or divided by the expected power)o With non-normalized spectra, if the power in one pixel is higher it may simply be because the

variance of the signal is higher there (the power is higher at all frequencies, more activity !)

o The principle behind 𝑃𝑃𝑔𝑔 𝑚𝑚 = 1 − 1 − 𝑃𝑃 𝑚𝑚⁄𝑁𝑁 𝑖

can in principle be generalized to 3Do Nearby pixels are not independento Coherence length function of Fourier frequency (and wavelength)

6262

Advices

o Don’t be afraid not to find anything in your data

o Look at your data and at its spectrum (log-log plots!)

o Estimate what to expect in the power spectrum o Analytic solutions to many signals existo Decompose the signal into simpler components and use the basic theorems

(shift, convolution, etc.) to compute the Fourier transform of the signal you think may be in there

o If the signal that you expect is periodic, use Fourier. If there is nothing in Fourier, wavelets won’t help for the detection (it is not “more powerful” because “fancier”).

o Do not use the global wavelet levels for detection of periodic signals. Use the time-averaged spectrum (if you must) or Fourier.

o Wavelets will however give you the extra information on when (or where) there is something significant if the signal is intermittent.