Optimized hierarchical continuous-wave searches

Badri KrishnanBadri KrishnanCurt Cutler, Iraj GholamiCurt Cutler, Iraj Gholami

AEI, GolmAEI, Golm

G040155-00-Z

LSC meeting, March 2004 ASIS session

Motivation

Full coherent searches for unknown pulsars Full coherent searches for unknown pulsars not computationally feasiblenot computationally feasible

Require incoherent, sub-optimal methodsRequire incoherent, sub-optimal methods Illustrative example: The stack slide searchIllustrative example: The stack slide search

5obsp TN

Example: Searching for young, fast pulsars over the whole sky and including two spin-down parameters for 10 days data requires a 1017 Flops computer

P.Brady and T.Creighton, PRD 61, 082001 (2000)

The Stack-Slide method Basic idea: Take the Fourier transform of each segment and track the Basic idea: Take the Fourier transform of each segment and track the

Doppler shift by adding power in the frequency domain (Stack and Doppler shift by adding power in the frequency domain (Stack and Slide)Slide)

First step : Break up data into N segmentsFirst step : Break up data into N segments

Calculate power spectrum or DeFT for each segmentCalculate power spectrum or DeFT for each segment

NTT obs

obsT

Frequency

Tim

eAdd power after frequency bins are shifted according to the time-frequency pattern

General hierarchical pipeline Break data into segments

Analyze eachsegment coherently

Combine segments incoherently

Select candidates

Analyze candidatescoherently using all available data

Acquire more data

Detection orupper limit

Incoherent stepcan be either stack-slide orHough

The search parameters

Ni : Number of stacks Ti : Time-baseline of each stack i : Mismatch in signal power Xi: Threshold on summed power

Variables for each incoherent stage Variables for final coherent stage

Tobs : Total observation time coh : Mismatch in signal power

Given : • Available computer power C0

• Data of a certain time duration Tobs

• Weakest signal strength we wish to detect h0

• Desired confidence level

Number of incoherent stages : n

We want to know:• Optimal values of the search parameters

Minimize computational cost subject to constraints

Two different search modes

Take fresh data in each stageTake fresh data in each stage

Re-use old data Re-use old data

time

Ist stage IInd stage IIIrd stage

Ist stage

IInd stage

IIIrd stage

False alarm rate for ith stage

Determines number of candidates for next stage to analyze

False dismissal rate for ith stage

Determines weakest signal that can be detected

StatisticsSummed power follows chi-square distribution with 2N d.o.f

N

k k0

X

X

dhP

dhP

0)|(

)0|(

False alarm and false dismissal rates :

Total false alarm rate always determined by final coherent stage

Choose thresholds for each stage by fixing false dismissal rate

),,,,(),,,,( 0)0()0(

0 hTNXXXhTN iiiiiiiiiii

Template countingParameter space metric calculated by Brady & Creighton

)(

2/

1111

)2(4

!3.01max

2/

2/

2/)1(min

2/max

2/)3(2max

222max

2max

0

min1

{0,1,2,3}s

NGss

sTfN

CBAs

fM

MckrGNMN

ss

s

sss

ss

s

s

s

k s

kk

sssp

63

32

1

0

00243.0)(

0708.0)(

524.0)(1)(

NNG

NNG

NNGNG

35

2

118.0

1046.0

014.0

NdayTC

dayTB

A

Number of points in coarse grid : Number of points in fine grid : ),,(

)1,,(

max

max

NTNN

NTNN

ppf

ppc

Computational costs

Computational cost of each stage is essentially cost of calculating FFTplus cost of summing the power

For each point in parameter space, number of floating point operations for first, intermediate and coherent stages:

)2(log6

)1(2log3

2

612log6

max2max)()(

)()()(

)1()()(

)()()(

max2max)1(

)()()()1()(

)1()1(

)1()1(

max2)1()1()1(

max)1(

obsobsnpf

nobs

cohpfobsncoh

iipc

i

iipfi

ipf

ipc

iiii

pcpfpc

TfTfNTNT

FC

TNNNN

TffN

NTNFC

NNNNTfNTNfC

F(i) = Number of candidates which survive the ith stage

Optimization strategy

• Basic strategy is to minimize Total computational cost subject to constraint that amount of analyzed data is lesser than available data

• False alarm rate is not really a constraint because false alarm rate is set by final coherent step

• Computationally limited searches can only see strong signals and when we do see them, it is usually easy to build up confidence • Want to analyze data in (roughly) real time• Function to be optimized is

obsa TT

n

coh

SThNFA

20)( exp

power nalComputatio )( obsa TTSf

S(x) = 1 if 0 < x < 1 and very large otherwise

Preliminary resultsExample of search criteria :

• Total observation time : 1 year• Signal strength we wish to detect : • Allowed false dismissal rate for each stage : 1%• Mismatch in coherent stage : 0.10• Don’t reuse old data• Max number of spindowns included : 3• All-sky search• Largest frequency searched over : fmax = 1000 Hz• Smallest spindown age : min = 40 yr

15/20 nobs STh

Optimization carried out by simulated annealing and amoeba method

Two stage search with coherent follow-up: Length of each stack : 0.37 days ; 0.25 days Number of stacks : 199 ; 956 Observation time : 73 days ; 240 days Power mismatch : 0.51 ; 0.23 Computational requirement : 8.8 x 1013 Flops

Three stage search with coherent follow-up: Length of each stack : 0.17 days ; 0.15 days ; 0.48 days Number of stacks : 511; 1028 ; 261 Observation time : 84 days ; 149 days ; 125 days Power mismatch : 0.48 ; 0.33 ; 0.02 Computational requirement : 7.8 x 1012 Flops

Single stage search with coherent follow-up: Length of each stack : 1.9 days Number of stacks : 39 Observation time : 74 days Power mismatch : 0.53 Computational requirement : 6.3 x 1016 Flops

Conclusions• Optimization scheme for hierarchical stack-slide search presented• Tells us what the search pipeline parameters must be• Expect similar results for hierarchical Hough search also• Does not consider cost of Monte Carlo simulations or memory issues• Shows that hierarchical schemes are absolutely essential for large parameter space blind searches