Detecting Deceptive Discussions in Conference Calls · More serious restatement is more likely to...

Detecting Deceptive Discussionsin Conference Calls

David Larcker and Anastasia Zakolyukina

Linguist 287

Larcker and Zakolyukina (Stanford GSB) Detecting Deceptive Discussions November 2, 2010 1 / 25

Deception Overview

Why Conference Calls?

1. Observed restatements of financials2. Q&A section is relatively unscripted3. Poor performance of accounting models

Deception Overview

Psychological Perspectives

1. Emotions: guilt, fear2. Cognitive effort: hard to lie3. Attempted control: caution vs. extensive rehearsal4. Lack of embracement: lack conviction, no personal experience

Deception Overview

Research Design

Bag-of-wordsLIWC + self-constructed categoriesLogistic model: deceptive vs. non-deceptiveUnbalanced classesOut-of-sample evaluation with 10-fold cross-validation 10 times

Deception Overview

LIWC+ self-constructed categories

1. Reference to general knowledge, e.g. you know, you would agree2. Shareholders value, e.g. shareholder value, value for investors3. Value creation, e.g. create value, delivering value4. Extreme positive emotions, e.g. fantastic, superb5. Extreme negative emotions, e.g. absurd, insane

Deception Overview

Word Categories: deception

Less(-) More(+)wc

References

I wethey ipron genknlrefPositives/Negatives

assent posemone negate anx anger swearnegemoextr

posemoextrCognitive process

certain tentatOther cues

hesit shvalue valueLarcker and Zakolyukina (Stanford GSB) Detecting Deceptive Discussions November 2, 2010 6 / 25

Deception Overview

Quarterly Earnings Conference Calls for U.S. firms 2003 - 2007FactSet Research Systems Inc.: answers of CFO/CEO from Q&AGlass, Lewis & Co. 2003-2009: restatementsPoint-In-Time: originally reported and restated figuresCompustat, CRSP: replication of Correia (2009)

Deception Overview

Deception Criteria

Key assumptionExecutive knows that false figures are reported

→More serious restatement is more likely to have deception cues

Serious Restatement = Deceptive Instance1. Material weakness disclosure or2. Late filing or3. Auditor change or4. Form 8-K filing

Deception Overview

Deception Criteria

Size of Restatement: aggregate

Company Filing date Restated quartersBally Total Fitness Holding Corp. 2004-03-30 Q1 2000 - Q4 2002Bally Total Fitness Holding Corp. 2005-11-30 Q1 2000 - Q1 2004Bally Total Fitness Holding Corp. 2006-06-27 Q1 2005 - Q3 2005

⇒ Q3 2005 : ndNIQ_SUM = 26%

CriteriaNT, AS25 (0.06%), AS50 (0.21%), PS25 (0.06%), PS50 (0.24%)

Classifier Evaluation

Contingency table

Predicted Class (PC)Deceptive(B) Truthful(W)

Actual Class (AC)Deceptive(B) True Positives False NegativesTruthful(W) False Positives True Negatives

Accuracy =TP + TN

TP + TN + FP + FN, Precision =

TPTP + FP

TPR =TP

TP + FN,FPR =

FPFP + TN

ROC Curve: Simple Example (1)

Cutoff = 0.3

W W B W B0.1 0.4 0.5 0.7 0.8

ACB 2 0W 2 1

TPR =22, FPR =

0.0 0.2 0.4 0.6 0.8 1.0

ROC curve

False positive rate

ROC Curve: Simple Example (2)

Cutoff = 0.45

W W B W B0.1 0.4 0.5 0.7 0.8

ACB 2 0W 1 2

TPR =22, FPR =

●●

0.0 0.2 0.4 0.6 0.8 1.0

ROC curve

False positive rate

ROC Curve: Simple Example

Cutoff = 0.6

W W B W B0.1 0.4 0.5 0.7 0.8

ACB 1 1W 1 2

TPR =12, FPR =

●●

0.0 0.2 0.4 0.6 0.8 1.0

ROC curve

False positive rate

ROC Curve

●●●

●●

0.0 0.2 0.4 0.6 0.8 1.0

ROC curve

False positive rate

Cross-validation: K folds

1. Randomly split the data into K folds2. Fix k th fold3. Estimate classifier using K-1 fold4. Apply/evaluate classifier using k th fold5. Rotate folds

10-fold cross-validation 10 times → 100 out-of-sample performances

ROC for CEOs of financial firms (NT)

ROC for the sample of CEOs of financial firms (NT)

Average false positive rate

0.0 0.2 0.4 0.6 0.8 1.0

Financial Firms: NT

Panel A: CEO Panel B: CFOSample composition

Total firm-quarters 1533 1265Deceptive firm-quarters 160 130Deceptive firm-quarters(%) 10.44 10.28

AUC, Precision, and Accuracy in %AUC (t-test vs. 50 %) 58.93 (20.26) 53.56 (6.37)Precision 13.76 11.57Accuracy 64.27 62.61

CEOs: NT, AS50

Sample compositionNT AS50

CFOs: NT, AS50

Sample compositionNT AS50

Deception Pattern

CEO sample (N = 16,577)(1)

NT AS25 AS50Word count

wc‡ +/− 1.10∗

ReferencesI − 0.93∗∗

we + 0.94∗∗∗ 1.05∗

they +/− 1.24∗∗∗

ipron +/− 1.08∗∗∗

genknlref +/− 1.49∗∗∗ 1.23∗∗∗

Deception Pattern

CEO sample (N = 16,577)(2)

NT AS25 AS50Positives/Negatives

assent − 1.30∗

posemone − 0.93∗∗

posemoextr +/− 1.23∗∗∗

anx + 0.67∗ 0.47∗∗ 0.43∗∗

swear† + 0.86∗∗ 0.78∗∗

negemoextr + 0.76∗∗ 0.67∗∗ 0.63∗

Cognitive mechanismcertain − 0.85∗∗

Other cueshesit† +/− 0.88∗

shvalue† +/− 0.84∗

value† +/− 0.75∗∗ 0.65∗∗∗ 0.67∗∗

Deception Pattern

CFO sample

NT, AS25, AS50

Less(-) More(+)wc

Referencesthey genknlref

Positives/Negatives

posemone anx assentCognitive process

certain tentatOther cues

shvalue value

Deception Pattern

Accounting vs. Linguistics: AUC

mean % t-test vs 50% CFO DACCEO 55.92 18.65 0.80 6.40CFO 55.59 18.10 5.95DAC 52.91 9.01DAC +CEO 56.47 19.52DAC +CFO 55.73 19.17DAC +CEO+CFO 57.86 24.30

Deception Pattern

Unadjusted vs. Adjusted: NT

Panel A: CEOUnadjusted Adjusted t-test

AUC (t-test vs. 50%) 60.20(15.30) 56.99(11.52) 4.27Precision 2.96 2.74 3.03Accuracy 61.68 61.22 2.34

Panel B: CFOUnadjusted Adjusted t-test

AUC (t-test vs. 50%) 50.50(0.73) 56.14(7.42) -5.70Precision 2.00 2.26 -3.26Accuracy 54.37 61.86 -28.40

Deception Pattern

Takeaways

Out-of-sample superiority over random by 4%-6% for CEOs/CFOsOut-of-sample accuracy is 50% - 65% for CEOs/CFOsOut-of-sample improvement over an accounting modelDeceptive CEOs vs. Deceptive CFOs

Detecting Deceptive Discussions in Conference Calls · More serious restatement is more likely to...

Documents

Transcript of Detecting Deceptive Discussions in Conference Calls · More serious restatement is more likely to...

IFRS_Restatement restatement

EGTRRA Restatement Deadlines - DATAIR

(Unfair, Deceptive or Abusive Acts and Practices)Unfair, Deceptive ...

When the Restatement is not a Restatement: The Curious ...

Restatement Third, Employment Law Chapter 5: …rja14/shb09/jolls2.pdf · Restatement Third, Employment Law Chapter 5: ... A separate chapter of the Restatement Third, ... 25 26 27

Deceptive Hacking Presentation

Deceptive bends

Deceptive Schedules

Restatement Third of Agency

The Guide to Restatement

Audit Analytics 2008 Restatement Rpt

Restatement Document - Investors

A Third Conflicts Restatement - Indiana Law Journalilj.law.indiana.edu/articles/75/75_2_Juenger.pdf · A Third Conflicts Restatement t ... RESTATEMENT (SECOND) OF CONFLICT OF LAWS

Deceptive Speech

RESTATEMENT FOR ARRANGER LIABILITY NDER …vjel.vermontlaw.edu/files/2013/06/Restatement-for-Arranger... · 2009] Restatement for Arranger Liability Under CERCLA 373 article will

Deceptive practices

RESTATEMENT (SECOND) OF

LAND-POSSESSOR LIABILITY IN THE RESTATEMENT (THIRD… · LAND-POSSESSOR LIABILITY IN THE RESTATEMENT (THIRD) OF TORTS: ... the Restatement (Second) of Torts also addresses land ...

Restatement (Second) of Agency

Deceptive Miscommunication