Construct validity of Bitemark assessments using the ABFO decision tree

Adam Freeman Iain Pretty*

Overview of presentation  

• Background • Scientific approach • Methods • Results • Impact and suggestions

Background  

The decision tree is a means of formalizing the approach to bitemark analysis by taking the assessor through a series of stages and decisions that aim to ensure that the decisions made are consistent with the level of forensic evidence available.

Background – Schematic of tree  

Patterned Injury

Is it a human BM ? Not a human BMSuggestive of a

human BM

Analysis – No comparison

Human Bitemark No analysis, no comparison

What are the characteristics ?

Next Step

Background  

This study examined Step 1 – the evaluation of the injury, is the injury a bitemark and if so, what are the bitemark’s characteristics? Today presenting data on the assessment of the injury as a bitemark only

Scientific approach  Several methods being applied to BM research: •  Mechanistic approach •  Decision making approach

In the absence of truth we are using construct validity – through reliability testing - if its not reliable its not valid.

Methods  250 cases submitted by DABFO – included an orientation shot and a close up with scale Selected 100 cases to represent a wide spread of anatomical location, presentation, evidence quality Presented to DABFO on an online system with anonymity of decisions Asked if there was sufficient evidence to render any opinion, and if so, what is it?

Methods  Data collected Demographics reported Kappa used to measure agreement Descriptive statistics to assess the spread of decisions and understand the reasons for disagreement

Results  38 Diplomates completed the whole study, 44 completed partially. Represents a total of 3924 decisions on bitemark cases Range of experience measured in three ways:

How many cases in past 5 years – 18.58 How many years have you been active – 19.87 How many times have you testified in the past five years? – 2.05

Results – decision spread  Look at the spread of decisions for individual cases.

-­‐2  

3  

8  

13  

18  

23  

28  

78  

68  

81  

86  

65  

70  

52  

85  

99  

15  

100   69  

18  

35   4   43  

98  

60  

94  

56  

73  

44  

66  

36  

83  

22  

55  

62  

89   3   41  

77  

30  

51  

76  

10  

11  

21  

27  

47  

48  

92  

96  

72   8   74   1   50  

19  

25  

This  is  a  human  bite  mark   This  is  sugges;ve  of  a  bite  mark   This  is  not  a  bite  mark  

Results – decision spread  Look at the spread of decisions for individual cases.

0  

5  

10  

15  

20  

25  

30  

35  

40  

34   49   71   90   95   29   97   2   40   5   75   24   39   53   84   91   23   31   37   87   12   17   32   67   93   13   38   46   6   79   82   28   58   59   64   9   14   20   42   45   57   88   16   54   61   80   26   33   63   7  

This  is  a  human  bite  mark   This  is  sugges;ve  of  a  bite  mark   This  is  not  a  bite  mark  

Impact and suggestions  All research has strengths and weaknesses •  Good number of decisions •  A lot of work – not all diplomates completed •  Some argued that not realistic approach

The study suggests level of reliability of injury assessment for bitemarks is not currently satisfactory from the population of assessors studied

The impact of three choices from the decision model has decreased reliability– removal of “suggestive” combined with greater detail on the identification of bitemarks within the decision model should be considered. The use of a simpler, dichotomous decision, should lead to increased reliability – although the decision direction of the “suggestives” is unknown.

Impact and suggestions   We need to undertake further examination of those cases where there was higher levels of agreement to determine how the decision tree can capture these elements to improve reliability, both for bites and non-bites. The first step of BM analysis is determining if the presented injury is a bitemark – the current data suggest that agreement levels observed require significant improvement and means of achieving this have been proposed. A further assessment following the introduction of these changes will be required.

Thank you  

We would like to thank those Diplomates who submitted their cases for inclusion in the study and for those who took the time to complete the exercise. Thank Dr Peter Loomis, and the ABFO, for supporting this work through design, implementation and reporting of findings.