Pierre Daoust Statistics Canada CES, Bonn (Germany)

Prioritizing Follow-up of Non-Prioritizing Follow-up of Non-Respondents Using Scores for the Respondents Using Scores for the

Canadian Quarterly Survey of Canadian Quarterly Survey of Financial Statistics for EnterprisesFinancial Statistics for Enterprises

Pierre Daoust Pierre Daoust Statistics CanadaStatistics Canada

CES, Bonn (Germany)CES, Bonn (Germany)

Statistique StatisticsCanada Canada

Description of the Quarterly Survey Description of the Quarterly Survey of Financial Statistics for of Financial Statistics for

Enterprises (QFS)Enterprises (QFS)

Provides information on corporate income Provides information on corporate income statements (elements of revenues, statements (elements of revenues, expenses and profits) and balance sheets expenses and profits) and balance sheets (assets, liabilities and equity)(assets, liabilities and equity)

Selected for Statistics Canada 2003 Selected for Statistics Canada 2003 Strategic Streamlining Initiative (SSI)Strategic Streamlining Initiative (SSI)

Description of QFSDescription of QFS

Preliminary estimates available 58 days after the Preliminary estimates available 58 days after the end of the reference quarterend of the reference quarterRevisions made Revisions made – the following quarterthe following quarter– at the end of a calendar yearat the end of a calendar year– once QFS estimates can be benchmarked to annual once QFS estimates can be benchmarked to annual

data compiled from the Canada Revenue Agency data compiled from the Canada Revenue Agency taxation statements. taxation statements.

Emphasis in producing quality estimates for Emphasis in producing quality estimates for preliminary release and first revision to minimize preliminary release and first revision to minimize magnitude of revisions.magnitude of revisions.


Sample survey of 5000 large businesses Sample survey of 5000 large businesses from eighty industry groups:from eighty industry groups:– Must units (400)Must units (400)– Reports by mail, electronically (EDR), or Reports by mail, electronically (EDR), or

through special arrangements (e.g. banks)through special arrangements (e.g. banks)

Administrative data used for small Administrative data used for small businessesbusinesses


BLAISE is used to monitor response status and BLAISE is used to monitor response status and to schedule follow-upsto schedule follow-upsCritical two-week window where most follow-ups Critical two-week window where most follow-ups occuroccurFollow-up of Must units done by a special teamFollow-up of Must units done by a special teamOut-of-scope, chronic non-respondents, special Out-of-scope, chronic non-respondents, special cases not scheduled for follow-up in BLAISEcases not scheduled for follow-up in BLAISEBLAISE uses scores to prioritize follow-upsBLAISE uses scores to prioritize follow-upsImputing for non-response prior to producing Imputing for non-response prior to producing estimatesestimates

QFS Score FunctionQFS Score Function

Scores produced dynamically in productionScores produced dynamically in production– Static version to set up BLAISEStatic version to set up BLAISE- Dynamic runs prior to each of the critical two weeks Dynamic runs prior to each of the critical two weeks

for follow-upsfor follow-ups- Each dynamic runs takes into account the up-to-date Each dynamic runs takes into account the up-to-date

collection status of sampled unitscollection status of sampled units

Inspired by the score function used for Statistics Inspired by the score function used for Statistics Canada’s Unified Enterprise Survey (UES) Canada’s Unified Enterprise Survey (UES)


UES scores derived based on coverage UES scores derived based on coverage targets for total revenuetargets for total revenue

QFS scores derived based on: QFS scores derived based on: – coverage targets for total revenue and total coverage targets for total revenue and total

assets for non-financial industriesassets for non-financial industries– coverage targets for total assets for financial coverage targets for total assets for financial

industriesindustries


Study was conducted in 2004 to set coverage Study was conducted in 2004 to set coverage targetstargetsUnits allocated to the three priority groups Units allocated to the three priority groups – 1 (Must units) 1 (Must units) – 3 (Other follow-ups) 3 (Other follow-ups) – 5 (No follow-up)5 (No follow-up)

Ranks are calculated for each priority group Ranks are calculated for each priority group using a multi-step approachusing a multi-step approach– Progress toward achieving coverage targets needs to Progress toward achieving coverage targets needs to

occur uniformly across industry groups.occur uniformly across industry groups.


Multi-step strategy to derive ranksMulti-step strategy to derive ranks– In each industry groupIn each industry group

First derive independently the ranks for total revenue and First derive independently the ranks for total revenue and total assetstotal assets

Combine information from the two independent sources (sum Combine information from the two independent sources (sum of ranks)of ranks)

– Across industry groups to prioritize unitsAcross industry groups to prioritize unitsCategorize enterprises based on their importance to Categorize enterprises based on their importance to coverage targets coverage targets

Integrate the industry and category groups in the ranks by Integrate the industry and category groups in the ranks by properly ordering and scaling unitsproperly ordering and scaling units

QFS Score Function QFS Score Function Rank Rank and high priority calculations for total revenue in an industry and high priority calculations for total revenue in an industry

group (Revenue coverage target = 80%).group (Revenue coverage target = 80%).

Unit Priority Weighted Revenue

Cumul-ative

Weighted Revenue

Cumul-ative

Percent Revenue

%

Prelim Revenue

High Priority

RevenueRank

Q022 1 20 20 22.99 Yes 1 Q020 1 15 35 40.23 Yes 2 Q011 3 10 45 51.72 Yes 3 Q018 3 10 55 63.22 Yes 4 Q004 3 7.5 62.5 71.84 Yes 5 Q015 3 7 69.5 79.89 Yes 6 Q009 3 6.5 76 87.36 Yes 7 Q003 3 6 82 94.25 No 8 Q016 3 3 85 97.70 No 9 Q005 3 2 87 100.00 No 10

QFS Score Function QFS Score Function RankRank and high priority calculations for total assets in an industry group and high priority calculations for total assets in an industry group

(Assets coverage target = 70%).(Assets coverage target = 70%).

Unit Priority Weighted Assets

Cumul-ative

Weighted Assets

Cumul-ative

Percent Assets

%

Prelim Assets High

Priority

Assets Rank

Q020 1 50 50 31.25 Yes 1 Q022 1 6 56 35.00 Yes 2 Q005 3 30 86 53.75 Yes 3 Q018 3 20 106 66.25 Yes 4 Q004 3 20 126 78.75 Yes 5 Q011 3 10 136 85.00 No 6 Q016 3 10 146 91.25 No 7 Q009 3 5 151 94.38 No 8 Q003 3 5 156 97.50 No 9 Q015 3 4 160 100.00 No 10

QFS Score Function QFS Score Function Category groupings within industry/priority groupsCategory groupings within industry/priority groups

Category For Non-Financial Industry Groups

For Financial Industry Groups

2 Units important for both variables

Units important for total assets

3 Units important for only one variable

Not applicable

4 Units not needed to achieve target coverages

QFS Score FunctionQFS Score Function Categorizing and ordering units in an industry groupCategorizing and ordering units in an industry group

Unit Priority Category Revenue Rank

Assets Rank

Sum Ranks

New Rank

Q020 1 2 2 1 3 1 Q022 1 2 1 2 3 2 Q018 3 2 4 4 8 1 Q004 3 2 5 5 10 2 Q011 3 3 3 6 9 3 Q005 3 3 10 3 13 4 Q009 3 3 7 8 15 5 Q015 3 3 6 10 16 6 Q016 3 4 9 7 16 7 Q003 3 4 8 9 17 8

QFS Score Function QFS Score Function Adjusting the industry/priority/category groups Adjusting the industry/priority/category groups

Revenue coverage target = 80%,Assets coverage target = 70%Revenue coverage target = 80%,Assets coverage target = 70%

Unit Weighted Revenue

New Cum. Per. Reve-nue

Weighted Assets

New Cum. Per. Assets

New Revenue High Priority

New Assets High Priority

Prio-rity

Cate-gory

Q020 15 17.24 50 31.25 Yes Yes 1 2 Q022 20 40.23 6 35.00 Yes Yes 1 2 Q018 10 51.72 20 47.5 Yes Yes 3 2 Q004 7.5 60.34 20 60.00 Yes Yes 3 2 Q011 10 71.84 10 66.25 Yes Yes 3 2 Q005 2 74.14 30 85.00 Yes Yes 3 2 Q009 6.5 81.61 5 88.13 Yes No 3 3 Q015 7 89.66 4 90.63 No No 3 4 Q016 3 93.10 10 96.88 No No 3 4 Q003 6 100 5 100 No No 3 4

QFS Score FunctionQFS Score Function Scaling the ranksScaling the ranks

Two issues:Two issues:– Only priority groups (1,3,5) and ranks are provided to BLAISEOnly priority groups (1,3,5) and ranks are provided to BLAISE– Number of units in a priority/category group varies by industry Number of units in a priority/category group varies by industry

Need to integrate the industry and category groups in the Need to integrate the industry and category groups in the ranksranksScale the ranks in each industry/priority/category groupScale the ranks in each industry/priority/category group– Same range of ranks for a priority/category group across Same range of ranks for a priority/category group across

industry groups industry groups – Non-overlapping sequential ranges of ranks for priority/category Non-overlapping sequential ranges of ranks for priority/category

groupsgroups– Ranks are distributed uniformly in each industry/priority/category Ranks are distributed uniformly in each industry/priority/category

groupgroup

QFS Score FunctionQFS Score Function Scaling variables for each priority/category groupScaling variables for each priority/category group

Priority Category

(# Units) of this Industry/ Priority/ Category

Largest (Max) # of Units in this Priority/ Category

Scaling Variable where Scaling Constant=10

1 2 2 26 10x26/2=130.0 3 2 4 111 10x111/4=277.5 3 3 1 120 10x120/1=1200.0 3 4 3 121 10x121/3=403.33

QFS Score FunctionQFS Score Function Calculating final ranks for an industry groupCalculating final ranks for an industry group

Priority Category Scaling

Variable Units Scaled Rank

Final (rounded) Rank

Q020 =130 130 1 2 130.0

Q022 =130+130=260 260 Q018 =277.5 278 Q004 =277.5+277.5=555 555 Q011 =555+277.5=832.5 833

3 2 277.5

Q005 =832.5+277.5=570 1110 3 3 1200.0 Q009 =1110+1200.0=2310.0 2310

Q015 =2310+403.33 =2713.33

2713

Q016 =2713.33+403.33 =3116.67

3117 3 4 403.33

Q003 =3116.67+403.33 =3520.0

3520

QFS Score FunctionQFS Score Function Range of final ranksRange of final ranks

Priority Category Minimum

Rank Maximum

Rank

1 2 10 260

3 2 10 1110

3 3 1120 2310

3 4 2320 3520


A small number of units are reassigned to not A small number of units are reassigned to not get called for follow-upget called for follow-up– Chosen amongst the highest ranks (lowest priorities) Chosen amongst the highest ranks (lowest priorities)

from those not needed to achieve the coverage from those not needed to achieve the coverage targetstargets

For the dynamic run, cumulative percentages For the dynamic run, cumulative percentages are based on only the enterprises that should are based on only the enterprises that should still be followed-up still be followed-up

Unweighted Responses Rates (current quarter)

15

20

25

30

35

40

45

50

55

60

65

70

75

80

1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1

02 02 02 02 03 03 03 03 04 04 04 04 05 05 05 05 06

All enterprises Needed for targets Not needed for targets

Weighted Coverage Rates Total Revenue - (current quarter)

15

20

25

30

35

40

45

50

55

60

65

70

75

80

1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1

02 02 02 02 03 03 03 03 04 04 04 04 05 05 05 05 06


Weighted Coverage Rates Total Assets - (current quarter)

15

20

25

30

35

40

45

50

55

60

65

70

75

80

1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1

02 02 02 02 03 03 03 03 04 04 04 04 05 05 05 05 06


Unweighted Response Rates (Revision next quarter)

15

20

25

30

35

40

45

50

55

60

65

70

75

80

1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1

02 02 02 02 03 03 03 03 04 04 04 04 05 05 05 05 06


Weighted Coverage Rates Total Revenue -(Revision next quarter)

15

20

25

30

35

40

45

50

55

60

65

70

75

80

1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1

02 02 02 02 03 03 03 03 04 04 04 04 05 05 05 05 06


Weighted Coverage Rates Total Assets -(Revision next quarter)

15

20

25

30

35

40

45

50

55

60

65

70

75

80

1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1

02 02 02 02 03 03 03 03 04 04 04 04 05 05 05 05 06


Future PlansFuture Plans

Review current approach for follow-ups, Review current approach for follow-ups, could extend follow-up period and have could extend follow-up period and have more runs of dynamic scoresmore runs of dynamic scoresLimit to be imposed automatically on Limit to be imposed automatically on number of attemptsnumber of attemptsExtend scores to editing processExtend scores to editing processContinue to improve scores as we get Continue to improve scores as we get more results and experiencemore results and experience

Prioritizing Follow-up of Non-Prioritizing Follow-up of Non-Respondents Using Scores for the Respondents Using Scores for the

Canadian Quarterly Survey of Canadian Quarterly Survey of Financial Statistics for EnterprisesFinancial Statistics for Enterprises

For more information, please contact:

Pour de plus amples informations veuillez contacter…

Courriel / Email: [email protected]

Statistique StatisticsCanada Canada Pierre Daoust

Pierre Daoust Statistics Canada CES, Bonn (Germany)

Documents

Transcript of Pierre Daoust Statistics Canada CES, Bonn (Germany)