AFTE Response to Seven Questions Related to Forensic Science ...

The Association of Firearm and Tool Mark Examiners (AFTE) December 23, 2015 response to seven questions related to forensic science posed on November 30, 2015 by The President’s Council of Advisors on Science and Technology (PCAST):

Q1 Part 1: What studies have been published in the past 5 years that support the foundational aspects of each of the pattern-based forensic science methods, including (but not limited to) latent print analysis; firearms/toolmarks; shoe/tire prints; bitemark analysis; questioned documents?

The Scientific Working Group for Firearm and Toolmarks (SWGGUN) developed the Admissibility Resource Kit (ARK) in 2005 to assist forensic firearm and tool mark examiners in the preparation for evidence admissibility hearings. When the SWGGUN was defunded in 2013, the AFTE Board of Directors and the past SWGGUN members decided to republish and maintain the ARK on the AFTE website. The ARK contains a collection of resources that represents significant research, legal opinions, challenges, rulings and other issues related to the discipline. The foundational research included on the ARK extends well beyond the past 5 years.

https://afte.org/resources/swggun-ark

The following are literature citations, all published within the last five years, for the more important studies that qualify as material principally concerned with the validity of firearm and toolmark identification. A short summary follows each citation.

Scientific practice demands that possible exceptions be researched and published (efforts to test or falsify), and that a large body of confirmatory evidence from training programs, experimentation, etc., will forever remain unpublished.

Testability of the Scientific Principle

Firearms Identification, Bullets

Hamby, J, et al, “The Identification of Bullets Fired From 10 Consecutively Rifled 9MM RUGER Pistol Barrels – A Research Project Involving 619 Participants from 23 Countries Using Optical Comparison Microscopy and ‘Ballistics’ Imaging Instrumentation with an Analysis of Possible Error Rate Using Bayesian Statistics”, Journal of Forensic Sciences (In Press) 2015

Ten consecutively rifled RUGER P-85 pistol barrels were obtained from the manufacturer and then test fired to produce known test bullets and ‘unknown’ bullets for comparison by firearms examiners from around the world. This study is a continuation of one originally designed and reported on by David Brundage. The original study was primarily limited to examiners from nationally accredited laboratories in the United States and we wanted to expand the study to provide test sets for firearms examiners around the world. The RUGER P-85 pistol and the 10 consecutively rifled barrels were borrowed from the Illinois State Police. Ammunition was obtained from the Winchester Ammunition Company (A Division of Olin), and 240 tests sets produced and distributed to forensic scientists and researchers around the world. A thesis, which involved a total of 201 participants – including the original 67 reported on by Brundage - was published by Hamby in 2001. This paper reports on the final conclusions of the research conducted by Brundage, Hamby and Thorpe over a 15-year period. Recently, 20 additional test

AFTE Response to PCAST Questions Regarding the State of the Firearm & Toolmark Discipline

of 23

sets were manufactured using a 4th type of 9mm Luger ammunition and polymer ‘clone’ sets made as well. These sets – both actual bullets and clone sets – have been distributed for use in forensic laboratories worldwide. (Note- Currently this research project has a total of 653 participants from 31 countries)

Chu, et al., “Automatic Identification of Bullet Signatures Based on Consecutive Matching Striae (CMS) Criteria”, Forensic Science International, Volume 231, 2013, pp. 137-141.

This paper described a study of fired bullet markings from ten consecutively manufactured firearm barrels by an automated 3D signature analytic method. This study used 3D topography image capture technology with acquisitions that were cross-correlated to existing firearm Consecutive Matching Striae (CMS) identification criteria. Results provided a fairly objective test that demonstrated support for these firearm CMS criteria.

Wong, C., “The Inter-Comparison of 1,000 Consecutively-Fired 9mm Luger Bullets and Cartridge Cases from a Ruger P89 Pistol Utilizing both Pattern Matching and Quantitative Consecutive Matching Striae as Criteria for Identification”, AFTE Journal, Volume 45(3), Summer 2013, pp. 267-272.

Previous studies have investigated the effect of consecutive firing of firearms to determine how the wear on barrels and breechfaces would affect the identification of fired bullets and cartridge cases. This study was conducted to determine if the toolmarks on fired bullets and cartridge cases would change significantly after firing 1,000 cartridges through a Ruger P89 9mm Luger semiautomatic pistol, while using both pattern matching and quantitative consecutively matching striae (QCMS) as identification criteria during the comparison process. While there were some differences between the toolmarks on the bullets and cartridge cases throughout the firing sequence, each bullet and cartridge case was successfully identified to the first bullet or cartridge case.

Mikko, D., et al., “Reproducibility of Toolmarks on 20,000 Bullets fired through an M240 Machine Gun Barrel”, AFTE Journal, Volume 44, Number 3, Summer 2012, pp. 248-253.

This article discusses the reproducibility of toolmarks on 7.62mm high velocity bullets fired through a single M240 machine gun barrel. Over the years, there have been several research studies and published articles pertaining to consecutively manufactured rifled barrels and the ability to microscopically identify bullets as having been fired through the same barrel of a firearm; however, to the knowledge of the authors, there has not been any in-depth microscopic study pertaining to 20,000 bullets being fired through a single rifled barrel and subsequently identified to that particular barrel. This study was designed to provide credible evidence in regards to the reproducibility and uniqueness of striations on the bearing surfaces of fired bullets. Despite changes to the reproducibility of some of the individual markings over the course of the study, the authors were able to correctly identify the barrel of origin for each of the collected fired bullets. See subsequent related article: Mikko, D. and Miller, J., “An Empirical Study/Validation Test Pertaining to the Reproducibility of Toolmarks on 20,000 Bullets Fired


of 23

Through M240 Machine Gun Barrels”, AFTE Journal, Volume 45, Number 3, Summer 2013, pp. 290-291.

Mikko, D. and Miller, J., “An Empirical Study/Validation Test Pertaining to the Reproducibility of Toolmarks on 20,000 Bullets Fired Through M240 Machine Gun Barrels”, AFTE Journal, Volume 45, Number 3, Summer 2013, pp. 290-291.

This article is a follow-up to an article that was published in the AFTE Journal-Volume 44, Number 3-Summer 2012, titled “Reproducibility of Toolmarks on 20,000 Bullets fired through an M240 Machine Gun Barrel”. Using a second M240 Machine gun with its original barrel, along with a new spare barrel assembly, thirty (30) additional bullets were test fired through both barrels and subsequently inter-compared blindly by four firearm and toolmark examiners, one of which had just completed his formal two-year training period. Additionally, the recovered (60) test fired bullets from both barrels were also mixed with the 127 bullets recovered during the test firing of 20,000 bullets in the reproducibility study and examined by the four firearm and toolmark examiners in a blind test study, in order to determine whether or not the examiners could correctly identify or eliminate the bullets as being fired through the correct barrel. A total of 164 questioned fired bullets were examined, which resulted in 164 correct answers from the participants in the study (zero percent error rate).

Fadul, T. G., “An Empirical Study to Evaluate the Repeatability and Uniqueness of Striations/Impressions Imparted on Consecutively Manufactured Glock EBIS Gun Barrels”, AFTE Journal, Volume 43, Number 1, Winter 2011, pp. 37-44.

This paper describes an empirical study of ten consecutively manufactured Glock barrels containing the Enhanced Bullet Identification System (EBIS). Study consisted of test sets sent to 238 examiners from 150 laboratories in 44 states and 9 countries that were designed to test the examiner’s ability to correctly identify fired bullets to the barrel that fired them. The results from 183 of these examiners produced an error rate of 0.4%. This study validated the repeatability and uniqueness of striated markings in gun barrels, as well as the ability of a competent examiner to reliably identify fired bullets to the barrels that marked them.

Intelligent Automation, Incorporated, “A Statistical Validation of the Individuality of Guns Using High Resolution Topographical Images of Bullets”, National Institute of Justice Grant #2006-DN-BX-K030, October, 2010

This was a study of marks on fired bullets by a topography based (3D) automated system. This study continued the analysis of a previous 2005 NIJ bullet study and validated the original premise of Firearm/Toolmark ID. This study also concluded that 1) the ability to determine that a given bullet was fired from a specific barrel depends on the individual barrel itself and not only on the brand of its manufacture, and 2) the performance of the automated analysis system used in this study is not representative of that of a trained firearms examiner as humans have a remarkable ability to perform pattern matching that is difficult to be replicated in any automated system.


of 23

Firearms Identification, Cartridge Cases Hamby, J., et al, “Evaluation of GLOCK 9mm Firing Pin Aperture Shear Mark Individuality Based On 1,632 Different Pistols by Traditional Pattern Matching and IBIS Pattern Recognition”, Journal of Forensic Sciences (In Press) 2015. Over a period of 21 years, a number of fired GLOCK cartridge cases have been evaluated. A total of 1,632 GLOCK firearms were used to generate a sample of the same size. Our research hypothesis was that no cartridge cases fired from different 9-mm semi-automatic GLOCK pistols would be mistaken as coming from the same gun. Using optical comparison microscopy, two separate experiments were carried out to test this hypothesis. A sub-sample of 617 test fired cases were subjected to algorithmic comparison by the Integrated Ballistics Identification System (IBIS). The second experiment subjected the full set of 1,632 cases to manual comparisons using traditional pattern matching. None of the cartridge cases were "matched" by either of these two experiments. Using these empirical findings, an established Bayesian probability model was used to estimate the chance that a 9-mm cartridge case could be mistaken as coming from the same firearm when in fact it did not (i.e. the random match probability).

Baldwin, D.P., et al., “A Study of False-Positive and False-Negative Error Rates in Cartridge Case Comparisons”, USDOE Technical Report # IS-5207 (April 7, 2014) This report provides the details for a study designed to measure examiner (not laboratory) error rates for false identifications and false eliminations when comparing an unknown to a collection of three known cartridge cases. Volunteer active examiners with Association of Firearm and Toolmark Examiners (AFTE) membership or working in laboratories that participate in ASCLD were provided with 15 sets of 3 known + 1 unknown cartridge cases fired from a collection of 25 new Ruger SR9 handguns. The ammunition was all Remington 9-mm Luger (manufacturer designation L9MM3) and sets were made up of cartridge cases fired within 100 cartridges of each other for each gun. During the design phase of the experiment, examiners had expressed a concern that known samples should not be separated by a large number of fired cartridges. However, studies published on this effect indicate that several thousands of cartridges could be fired by the same firearm without making the identifying characteristics change enough to prevent identification. [1] Examiners were provided with a background survey, an answer sheet allowing for the AFTE range of conclusions, and return shipping materials. They were also asked to assess how many of the 3 knowns were suitable for comparison, providing a measured rate of how often each firearm used in the study produces useable, quality marks. The participating examiners were provided with known positives and known negatives from independent groups of samples, providing independent measurements of a false-positive rate and independent measurements of a false-negative rate, allowing the study to measure both rates and uncertainties in those rates. Responses were received from 218 participating examiners. The rate of false negatives (estimated as 0.367% from comparisons known to be from the same firearm but reported as eliminations) was quite low with the error distributed across examiners of various backgrounds (state, federal, local, private, etc. as determined from self-reported survey information). The overall rate of false positives (estimated as 1.01% from comparisons known to be from different firearms but reported as identifications) was significantly higher. However, most of the errors


of 23

were reported by a small number of examiners; that is, individual examiners have varying error rates. For most examiners this is quite low while for some it is relatively high. Hence the overall rate is best interpreted as an average of widely varying individual rates. Inconclusive results were not recorded as errors. Rates of poor quality mark production for these handguns varied across the 25 sample handguns. Those rates were 2.3 (±1.4) %.

False-positive and false-negative error rates for individual examiner performance on comparisons were measured. The rates are not uniform across the sample population with a few examiners providing most of the false-positive responses. False-negative rates are low and comparable to or lower than the rate of production of poor quality marks by the firearms used in this study. Laboratory error rates may be significantly lower than these individual rates if quality assurance procedures are applied that can effectively manage to reduce or eliminate the propagation of false positives reported by individuals.

Stroman, A., “Empirically Determined Frequency of Error in Cartridge Case Examinations Using a Declared Double-Blind Format”, AFTE Journal, Vol. 46(2), Spring 2014, pp. 157-175.

This paper describes a no-gun empirical study of fired cartridge cases to determine the frequency of error in firearms identification using a declared double-blind testing format; i.e., a declared test containing blind elements. Seventy-four of seventy-five examiners accurately identified the questioned fired cartridge cases to the respective known specimens with no false positives. This study also demonstrated that examiners were able to accurately evaluate breechface markings avoiding mis-identifications from substantial subclass marks borne by the cartridge cases.

Chu, Tong and Song, “Validation Tests for the Congruent Matching Cells (CMC) Method Using Cartridge Cases Fired with Consecutively Manufactured Pistol Slides”, AFTE Journal, Volume 45(4), Fall 2013, pp. 361-366.

This was a study of ten (10) consecutively manufactured slides using 3D topography technology with correlations of paired breech marking correlation cells to establish firearm identifications. Test results showed significant separation between KM and KNM distributions without any false positive or false negative identification.

Fadul, et al, “An Empirical Study to Improve the Scientific Foundation of Forensic Firearm and Tool Mark Identification Utilizing Ten (10) Consecutively Manufactured Slides”, AFTE Journal, Volume 45(4), Fall 2013, pp. 376-389.

Empirical study of marks produced from 10 consecutively Ruger brand manufactured pistol slides by 217 firearm examiners from 46 states and the District of Columbia. Results of this study established an error rate of less than 0.1%, and validated toolmark durability as these slides maintained their individual signature after multiple firings.

Stowe, A., “The Persistence of Chamber Marks From Two Semiautomatic Pistols on Over 1,440 Sequentially-Fired Cartridge Cases”, AFTE Journal, Vol. 44(4), Fall 2012, pp. 293–308.


of 23

A Browning Hi-Power semiautomatic pistol and a Hi-Point Model C semiautomatic pistol were test fired a total of 1,440 times each, and their chamber marks were examined. Ammunition used included cartridges with cases made of aluminum, brass and nickel-plated brass. Microscopic examination of the chamber marks revealed that they were reproducible and identifiable up to 960 firings and that the metallic composition of the cartridge case does not affect the reproducibility of the chamber marks.

Petraco D. K., et al, “Application of Machine Learning to Toolmarks: Statistically Based Methods for Impression Pattern Comparisons”, NIJ/NCJRS Document #239048, Award #2009-DN-BX-K041, July 2012

This was a statistical study that evaluated 3D quantitative surface topographies of toolmarks, consisting of fired cartridge cases, screwdriver and chisel striations, generated using confocal microscopy. Principal component and canonical variate analysis, as well as support vector machine methodology, was used to objectively associate these toolmarks with the tools that produced them. Estimated toolmark identification error rates were approximately 1% using these algorithmic methods. The findings of this objective and quantitative scientific research support the general conclusions codified in the AFTE Theory of Identification.

Weller, T. J., et al, “Confocal Microscopy Analysis of Breech Face Marks on Fired Cartridge Cases from 10 Consecutively Manufactured Pistol Slides”, Journal of Forensic Sciences, Volume 57(4), July 2012, pp. 912-917.

This was a study of 90 test fired cartridge case specimens from ten consecutively manufactured pistol slides. A total of 8010 comparisons were conducted by using confocal microscopy with a 3D cross-correlation analysis logarithm. The average match scores were 0.82 with the average non-match scores 0.20. There was no overlap of scores between matching and non-matching test scores. This study provided objective data that supports the AFTE Theory of Identification.

Valle, F., et al, “Nanotechnology for Forensic Sciences: Analysis of PDMS Replica of the Head of Spent Cartridge Cases by Optical Microscopy, SEM and AFM for the Ballistic Identification of Individual Characteristics Features of Firearms”, Forensic Science International, Issue 222, 2012, pp. 288-297.

A novel application of replica molding to a forensic problem, viz. the accurate reproduction of the case head of gun and rifle cartridges, prior and after being shot, is presented. The fabrication of an arbitrary number of identical copies of the region hit by the firing pin and the breech face is described. The replicas can be (i) handled without damaging the original evidence, and (ii) distributed to different law enforcement agencies for comparison against other evidence found on crime scenes or ballistics tests of seized firearms, (iii) maintained on a file in the laboratory. A detailed analysis of the morphological features was carried out using a variety of instrumentation.


of 23

Mayland, B. and Tucker, C., “Validation of Obturation Marks in Consecutively Reamed Chambers”, AFTE Journal, Volume 44(2), Spring, 2012, pp. 167-169.

This study of fired cartridge cases from ten consecutively manufactured firearms was conducted to determine the reproducibility and reliability of obturation marks from reamed chambers for identification purposes. Results of this empirical study, which consisted of sixty-four (64) participants from nineteen (19) national laboratory systems, effected a sensitivity rating of 0.927. These results demonstrate that obturation markings imparted on fired cartridge cases can be used as a reliable means of identification to the firearm that marked them.

Saribey, A, and Hannam, A., “Comparison of the Class and Individual Characteristics of Turkish 7.65mm Browning / .32 Automatic Caliber Self-Loading Pistols with Consecutive Serial Numbers”, Journal of Forensic Sciences, Volume 58(1), January 2012, pp. 146-150.

Firearms identification is based on the fundamental principle that it is impossible to manufacture two identical items at the microscopic level. As firearms manufacturing technologies and quality assurance are improving, it is necessary to continually challenge this principle. In this study, two different makes of 7.65mm Browning / .32 caliber self-loading pistols of Turkish manufacture were selected and examined. Ten pistols with consecutive serial numbers were examined and test fired 10 times. The fired cartridge cases were recovered for comparison purposes. It was found that for each make of pistol, the individual characteristics within the firing pin impression, ejector and breech face marks of all 10 pistols were found to be significantly different.

La Porte, D., “An Empirical Validation Study of Breechface Marks on .380 ACP Caliber Cartridge Cases Fired from Ten Consecutively Finished Hi-Point Model C9 Pistols”, AFTE Journal, Volume 43, Number 4, Fall 2011.

An empirical study was conducted using ten (10) consecutively finished Hi-Point model C9 slides and one frame acquired from the Hi-Point Manufacturing Company in Mansfield, Ohio. The ten (10) slides were mounted on the frame and test fired to obtain cartridge cases for comparison. The test fired cartridge cases were microscopically examined, evaluated and compared for class and individual characteristics that resulted from the manufacturing process. Prominent striations were evident on each test-fired cartridge case. These resulted from sanding of the breech face. The variations that occur during the manufacturing process of sanding result in unique, identifiable, individual breech face marks devoid of subclass influence. A limited validation study was conducted after the empirical study. Correct associations were made during this limited study.

Thompson, R., Song, J., Zheng, A., and Yen, J., “Cartridge Case Signature Identification Using Topography Measurements and Correlations: Unification of Microscopy and Objective Statistical Methods”, National Institute of Standards and Technology, Presented at the 18th European Network of Forensic Science Institutes (ENFSI) Conference, Lisbon, Portugal, October, 2011


of 23

A comparison microscope employing the standard optical comparison method and confocal microscopy, with subsequent cross-correlation topography analysis, were used to correctly identify cartridge cases fired from ten consecutively made pistol slides. Subsequent cross correlation function analysis and statistical analysis of match and non-match scores correctly identified the fired cartridge cases back to their respective known slide source in 19 of 20 occasions with one inconclusive result. Results of the mathematical determination of slide source were compared to the validated results from the microscopic comparisons.

Lightstone, L., “The Potential for and Persistence of Subclass Characteristics on the Breech Faces of SW40VE Smith & Wesson Sigma Pistols”, AFTE Journal, Volume 42(4), Fall 2010, pp. 308-322.

An article published in the 2007 AFTE Journal Summer edition discusses a situation in which a high degree of subclass characteristics were found in two firearms during routine casework. Gene Rivera of the Charlotte-Mecklenburg Police Department Crime Laboratory describes how these two firearms came to be discovered through the use of NIBIN, and reemphasizes the importance of the firearms examiner's job to be able to recognize and distinguish subclass characteristics when present. It was this striking case that prompted further research into the propensity and persistence of subclass characteristics in the Sigma Series line, and the potential for individuality to be established on these firearms.

Toolmark Identification

King, E., “Validation Study of Computer Numerical Control (CNC) Consecutively Manufactured Screwdrivers”, AFTE Journal, Volume 47(3), Summer 2015, pp. 171-176.

The purpose of this research was to perform a validation study to determine if screwdrivers that are consecutively manufactured using the computer numerical control (CNC) process can be identified by trained forensic examiners after having their class characteristics reproduced by striated toolmark samples. The results were based on participation from seven members of the Scientific Working Group for Firearms and Toolmarks (SWGGUN) and yielded an error rate of 0.00%. This result provides support of toolmark identification in the scientific community, thus complying with the Daubert standard. These results further demonstrate the CNC-consecutively manufacturing process did not eliminate the individual or class characteristics of the screwdrivers and does not interfere with the ability of examiners to correctly associate tools with the marks they leave on surfaces.

Ekstrand, et al, “Virtual Tool Mark Generation for Efficient Striation Analysis”, Journal of Forensic Sciences, Volume 59(4), 2014, pp. 950-959.

This is a follow-up study on Zhang and Chumbley’s research regarding the development of virtual toolmarks by a 3-D computer simulation that would allow for the development of highly predictable toolmark characterizations. Initial study involved the production of test toolmarks by six screwdriver tips that were then compared by a previously developed statistical algorithm. Preliminary experimental results indicate that the use of a manipulative, virtual tool


of 23

could provide quantitative data for the characterization of tool marked surfaces that would improve the scientific basis of toolmark identification. These results support the present theory and conclusions held in Toolmark Identification.

Zheng, X.A., et al, “2D and 3D Topography Comparisons of Toolmarks Produced from Consecutively Manufactured Chisels and Punches”, AFTE Journal, Vol. 46(2), Spring 2014, pp. 143-147.

This paper described an automated blind study of toolmarks from consecutively made chisel and punches utilizing 2D and 3D topography analysis. These analytical comparative results were expressed as a maximum value of the normalized Cross Correlation Function (CCF). Based on the CCF metric, all of the toolmarks were correctly identified to the tool that produced them. This study provides additional objective scientific support for the validity of Toolmark Identification.

Chumbley, S. and Morris, M., “Significance of Association in Tool Mark Characteristics”, Department of Justice (DOJ) Grant 2009-DN-R-119, Document 243319, August 2013 (Ames Laboratory)

In a recent study of tool marks produced by sequentially made screwdriver tips, the authors developed a computer algorithm that would reliably separate matching tool marks from those that do not match using an analysis based on Mann-Whitney U-statistics applied to data files containing 2-dimensional information obtained using an optical profilometer. These successful results indicate that the significance of association can be accomplished by statistical evaluation of the data file. The work carried out in the present project (and discussed in the report) built upon this success by providing additional statistical information that will increase the relevance of the measurements obtained.

Grieve, T., “Objective Analysis of Toolmarks in Forensics”, Graduate Thesis and Dissertations, Paper 13014, 2013, Iowa State University

Since the 1993 court case of Daubert v. Merrell Dow Pharmaceuticals, Inc. the subjective nature of toolmark comparison has been questioned by attorneys and law enforcement agencies alike. This has led to an increased drive to establish objective techniques with known error rates, much like the DNA analysis is able to provide. This push has created research in which the 3-D surface profile of two different marks are characterized and the marks’ cross sections are run through a comparative statistical algorithm to acquire a value that is intended to indicate the likelihood of a match between the marks. The aforementioned algorithm has been developed and extensively tested through comparison of evenly striated marks made by screwdrivers. However, this algorithm has yet to be applied to quasi-striated marks such as those made by the shear edge of slip-joint pliers. The results of this algorithm’s application to the surface will be presented.

Objective mark comparison also extends to comparison of toolmarks made by firearms. In an effort to create objective comparisons, microstamping of firing pins and breech faces have been introduced. The process involves placing unique alphanumeric identifiers surrounded by a radial


of 23

code on the surface of the firing pins, which transfer to the cartridge’s primer upon firing. Three different guns equipped with micro stamped firing pins were used to fire 3000 cartridges. These cartridges are evaluated based on the clarity of their alphanumeric transfers and the clarity of the radial code surrounding the alphanumerics.

Petraco, N., et al, “Estimation of Striation Pattern Identification Error Rates by Algorithmic Methods”, AFTE Journal, Volume 45(3), Summer 2013, pp. 235-244.

This was a computational study using algorithmic methods of toolmark striation patterns produced by screwdriver tips and firearm firing pin apertures in determining error rates. Multivariate analysis, as well as support vector machine methodology, was used to objectively associate these toolmarks with the tools that produced them. Estimated toolmark identification error rates were approximately 1% using these algorithmic methods. The findings of this objective and quantitative scientific research support the general conclusions codified in the AFTE Theory of Identification.

Petraco, N., et al, “Application of Machine Learning to Toolmarks: Statistically Based Methods for Impression Pattern Comparisons”, NIJ/NCJRS Document #239048, Award #2009-DN-BX-K041, July 2012

This was a statistical study using 3D quantitative surface topographies of toolmarks, consisting of fired cartridge cases, screwdriver and chisel striations, by confocal microscopy. Principal component and canonical variate analysis, as well as support vector machine methodology, was used to objectively associate these toolmarks with the tools that produced them. Estimated toolmark identification error rates were approximately 1% using these algorithmic methods. The findings of this objective and quantitative scientific research support the general conclusions codified in the AFTE Theory of Identification.

Chumbley, L. S., et al, “Validation of Tool Mark Comparisons Obtained Using a Quantitative, Comparative, Statistical Algorithm”, Journal of Forensic Sciences, Volume 55(4), 2010, pp. 953-961.

A statistical analysis and computational algorithm for comparing pairs of toolmarks by profilometry data was conducted. Toolmarks produced by 50 sequentially made screwdrivers, at selected fixed angles, were analyzed both empirically by practicing examiners and by the established computational algorithms. The results of these comparisons, as well as a subsequent blind study with the practicing examiners, showed scores of good agreement between the algorithm and human experts. It was also noted that in some of the examination phases, examiner performance was much better than the algorithm.

Bachrach B., Jain A., Jung S., Koons R.D., “A Statistical Validation of the Individuality and Repeatability of Striated Tool Marks: Screwdrivers and Tongue and Groove Pliers”, Journal of Forensic Sciences, Volume 55(2), 2010, pp. 348-357.


of 23

This study statistically validated the original premise of individuality in Toolmark Identification by analyzing statistical distributions of similar values resulting from the comparison of Known Matches (KM) and Known Non-Matched (KNM) pairs of striated toolmarks. This quantifiable analysis of KM and KNM toolmark similarity distributions showed nearly error-free identifications.

Firearm and Toolmark Identification Theoretical

Kerkhoff, W., et al, “The Likelihood Ratio Approach in Cartridge Case and Bullet Identification”, AFTE Journal, Volume 45(3), Summer 2013, pp. 284-289.

This article summarizes the different aspects of the discussion that led to the implementation of the likelihood ratio approach of firearms identification by the Firearms Section of the Netherlands Forensic Institute (NFI). The authors' (three firearms examiners and a statistician) perspectives on the use of this approach in cartridge case and bullet comparison are shared.

Heikkinen, V., et al, “Quantitative High-Resolution 3D Microscopy Improves Confidence When Determining the Order of Creation of Toolmarks”, AFTE Journal, Volume 45(2), Spring 2013, pp. 150-159.

The authors of this paper address the problem of determining the order of creation of engravures (toolmarks) on spent cartridges and fired bullets. We employ quantitative high resolution large area 3D optical imaging for traceable comparison. This solution is novel in the sense that so far only qualitative 2D imaging has been used to address this issue. Our main result is that we can now determine the order of creation of two different kinds of toolmarks on spent cartridges. The main impact of the result is that this technique improves the investigator's confidence when determining the order of creation of the marks as well as the direction of the engravure. Our work advances the state of the art in the field of forensic toolmark inspection by enabling a new quantitatively measured dimension (2D->3D) to improve the objectivity of the forensic analysis. Our work was carried out on copper that was scratched with a steel stylus in a controlled manner. The method was validated using spent cartridges. In practice this effort could aid inspection work aiming at telling apart marks created by the cartridge manufacturer from those made by the gun that fired the cartridge.

Bolton-King, R., et al., “Numerical Classification of Curvilinear Structures for the Identification of Pistol Barrels”, Forensic Science International, Issue 220, 2012, pp. 197-209.

This paper demonstrates a numerical pattern recognition method applied to curvilinear image structures. These structures are extracted from physical cross-sections of cast internal pistol barrel surfaces. Variations in structure arise from gun design and manufacturing methods providing a basis for discriminations and identification. Binarised curvilinear land transition images are processed with fast Fourier transform on which principal component analysis is performed. The proposed methodology is therefore a promising novel approach for the classification and identification of firearms.


of 23

Petraco, D. K., et al, “Addressing the National Academy of Sciences’ Challenge: A Method for Statistical Pattern Comparison of Striated Tool Marks”, Journal of Forensic Sciences, Volume 57(4), 2012, pp. 900-911.

Toolmark test specimens from nine slotted screwdrivers were encoded into high-dimensional feature vectors and analyzed by multiple statistical pattern recognition methods. The statistical methods used which are widely known and accepted in academic applications, rely on few assumptions of the data’s underlying distribution, can be accompanied by standard confidence levels and are falsifiable. Correct classification rates of at least 97% were achieved.

Fracture Matching

Claytor, D., “Validation of Fracture Matching Through the Microscopic Examination of the Fractured Surfaces of Hacksaw Blades”, AFTE Journal, Vol. 42(4), Fall 2010, pp. 323-334.

Validation of fracture matching method utilizing two consecutively manufactured hacksaw blades fractured eleven times and inter-compared. Two hundred fifty-three topical comparisons were conducted between forty-four fractured edges. Additional fractured hacksaw blade test specimens were produced and sent to examiners around the world yielding three hundred-thirty test results.

Weimar, B., et al., “Physical Match Examination of the Joint Faces of Adhesive PVC-Tapes”, AFTE Journal, Volume 42(3), Summer 2010, pp. 271-277.

A new method is presented for the physical match examination of the joint faces of cut and torn PVC insulation tapes. The combination of heat treatment, casting and comparison-light-microscopy with oblique light from opposite directions lead to results with a high conclusiveness. The method can be applied with the standard equipment in forensic toolmark laboratories

Q1 Part 2: What studies are needed to demonstrate the reliability and validity of these methods?

The reliability of the science of firearm and tool mark identification has been established through numerous validation studies, most of which are cited on the AFTE website under the SWGGUN Admissibility Resource Kit (https://afte.org/resources/swggun-ark). These studies evaluate tools (such as firearms) produced using different manufacturing methods, and have consistently shown that qualified forensic practitioners are able to distinguish between tool marks produced using different tools. Additional validation studies may be appropriate to capture new manufacturing processes, as well as, responses from a larger segment of the forensic firearm and tool mark population.


of 23

Q2 Part 1: Have studies been conducted to establish baseline frequencies of characteristics or features used in these pattern-based matching techniques? If not, how might such studies be conducted?

There are two main types of toolmarks considered by the firearm and toolmark examiner; impressed and striated.

Impressed toolmarks are, as the name implies, created when a harder tool workingsurface strikes, or comes into contact with, a softer surface with sufficient force to createan impression.

Striated toolmarks are created by a sliding motion where a harder tool working surface,like the rifled bore of a firearm, or the edge of a screwdriver, makes contact with a softermaterial, like a fired bullet or edge of a metal door frame. Parallel lines, called striae, ofvarying width, are formed.

Pattern-Matching is the criteria for identification method of toolmark comparison and identification that is utilized by forensic laboratories throughout the US. The Association of Firearm and Toolmark Examiners (AFTE) Theory of Identification (adopted by AFTE in 1993 and slightly revised in May 2011) states the following:

AFTE Theory of Identification as it Relates to Toolmarks 1. The theory of identification as it pertains to the comparison of toolmarks enables opinions ofcommon origin to be made when the unique surface contours of two toolmarks are in “sufficient agreement.” 2. This “sufficient agreement” is related to the significant duplication of random toolmarks asevidenced by the correspondence of a pattern or combination of patterns of surface contours. Significance is determined by the comparative examination of two or more sets of surface contour patterns comprised of individual peaks, ridges and furrows. Specifically, the relative height or depth, width, curvature and spatial relationship of the individual peaks, ridges and furrows within one set of surface contours are defined and compared to the corresponding features in the second set of surface contours. Agreement is significant when the agreement in individual characteristics exceeds the best agreement demonstrated between toolmarks known to have been produced by different tools and is consistent with agreement demonstrated by toolmarks known to have been produced by the same tool. The statement that “sufficient agreement” exists between two toolmarks means that the agreement of individual characteristics is of a quantity and quality that the likelihood another tool could have made the mark is so remote as to be considered a practical impossibility. 3. Currently the interpretation of individualization/identification is subjective in nature, foundedon scientific principles and based on the examiner’s training and experience.

Attempts have been made in establishing a more objective criteria called Quantitative Consecutive Matching Striae (QCMS) which is in use by some firearm and toolmark examiners; however, it is not yet employed universally. QCMS is a way of describing in numerical terms an identification after traditional pattern matching methods have been employed. Once a pattern is found, the striations are tabulated and compared against the QCMS baseline. It should be noted that currently QCMS can only be employed when striated marks are involved and is not yet capable of capturing impressed marks which are routinely encountered by examiners in casework.


of 23

Creating baseline frequency studies is a difficult proposition in the field of Firearms and Toolmarks Examination due to the dynamic nature this type of evidence presents. Given there can be no degree of control over the absence or presence of affected surface areas that may contain baseline marks makes the use of a standard frequency database difficult. However, in recent years research has been and continues to be conducted using computer technology to begin formulating criteria and to assist in creating objective, measurable standards for identification within the field.

The following are literature citations, all published within the last five years, for some of the emerging research which has been performed. A short summary follows each citation.

Lilien, R. et al, “Applied Research and Development of a Three-Dimensional Topography System for Imaging and Analysis of Striated and Impressed Tool Marks for Firearm Identification using GelSight” Department of Justice Award 2013-R2-CX-K005, Document 248962, 2015

In the described work, we investigated and developed a novel, accurate, and low-cost system for structural 3D imaging and comparison of cartridge cases. We demonstrated the system’s potential for increasing the quality and reducing the cost of forensic analyses. Several recent studies have called for improved imaging technology and matching algorithms to support firearm identification. Our project, named Top-Match, combines the recently developed GelSight high-resolution surface topography imaging system with state-of-the-art algorithms for matching structural features. Compared to competing technologies, our GelSight based system is fast, inexpensive, and not sensitive to the optical properties of the material being measured. This project aims to extend the system to measure and compare striated toolmarks (e.g., aperture shear), to integrate these marks into the scoring function, and to investigate matching algorithms for comparing 3D surface topographies captured using different imaging modalities (e.g. GelSight vs. confocal microscopy). The research work was completed by Cadre Research Labs, a scientific computing contract research organization, working in collaboration with GelSight Inc., a company formed by the MIT researchers who developed the GelSight surface topography imaging technology. The two companies collaborate closely with Todd Weller, a firearms identification specialist and Criminalist in the Oakland Police Department. We also worked with colleagues at NIST and at the International Forensic Science Laboratory & Training Centre in Indianapolis (Dr. James Hamby). We continue to work with Andy Smith (San Francisco PD), Chris Coleman (Contra Costa County Office of the Sheriff), and Karl Larsen (U. Illinois at Chicago). These collaborators continue to be excellent partners and provide both scans and constructive feedback. The results described below made use of a large set of new and previously collected test fires.

McClarin, D., “Adding an Objective Component to Routine Casework: Use of Confocal Microscopy for the Analysis of 9mm Caliber Bullets”, AFTE Journal, Volume 47(3), Summer 2015, pp. 161-170.

The Alabama Department of Forensic Sciences (ADFS) procured a confocal microscope for the purpose of incorporating three-dimensional (3D) topographical analysis into routine casework. The purpose of employing such a technique was to assist the firearm and toolmark examiner by


of 23

complementing routine analysis with an independent objective analysis. This article covered the research procedures conducted using confocal microscopy at the ADFS.

Spotts, R., et al., “Angular Determination of Toolmarks Using a Computer-Generated Virtual Tool”. Journal of Forensic Sciences, Volume 60(4), 2015, pp. 878-893.

A blind study to determine whether virtual toolmarks created using a computer could be used to identify and characterize angle of incidence of physical toolmarks was conducted. Six sequentially manufactured tips and one random screwdriver were used to create toolmarks at different angles. An apparatus controlled tool angle. Resultant toolmarks were randomly coded and sent to the researchers who scanned both tips and toolmarks using an optical profilometer to obtain 3D topography data. Developed software was used to create virtual marks based on the tool topography data. Virtual marks generated at angles from 30 to 85 degrees (5 degree increments) were compared to physical toolmarks using a statistical algorithm. Twenty of twenty toolmarks were correctly identified by the algorithm. On average the algorithm estimated the correct angle of incidence by -6.12 degrees. This study presents the results, their significance, and offers reasons for the average misidentifications.

Spotts, R., and Chumbley, S., “Objective Analysis of Impressed Chisel Toolmarks”, Journal of Forensic Sciences, Volume 60(6), 2015, pp. 1436-1440.

Historical and recent challenges to the practice of forensic examination have created a driving force for the formation of objective methods for toolmark identification. In this study, fifty sequentially manufactured chisels were used to create impression toolmarks in lead (500 toolmarks total). An algorithm previously used to statistically separate known matching and nonmatching striated screwdriver marks and quasi-striated plier marks was used to evaluate the chisel marks. Impression evidence, a more complex form of toolmark, poses a more difficult test for the algorithm that was originally designed for striated toolmarks. Results show in this instance that the algorithm can separate matching and nonmatching impression marks, providing further validation of the assumption that toolmarks are identifiably unique.

Riva, F. and Champod, C., “Automatic Comparison and Evaluation of Impressions Left by a Firearm on Fired Cartridge Cases”, Journal of Forensic Sciences, Vol. 59(3), May 2014, pp. 637-647.

This paper reported on an automated study of marks contained on fired cartridge cases from seventy-nine (79) 9mm Luger caliber pistols were conducted using 3D surface topography analysis and coupled to a bivariate evaluative model to assign likelihood ratios. The purpose of this analytic system was to conduct an objective comparative analysis with a robust statistical evaluation basis to the results. The system reflected a very high discriminating ability between the known and non-known specimens. This study also reflected very low rates of misleading evidence depending on the firearm considered.


of 23

Yammen, S., and Muneesawang, P., “Cartridge Case Image Matching using Effective Correlation Area Based Method”, Forensic Science International, Issue 229, 2013, pp. 27-42.

A firearm leaves a unique impression on fired cartridge cases. The cross-correlation function plays an important role in matching the characteristic features on the cartridge case found at the crime scene with a specific firearm, for accurate firearm identification. This paper proposes that the computational forensic techniques of alignment and effective correlation area-based approaches to image matching are essential to firearm identification. Specifically, the reference and the corresponding cartridge cases are aligned according to the phase-correlation criterion on the transform domain. The informative segments of the breech face marks are identified by a cross-covariance coefficient using the coefficient value in a window located locally in the image space. The segments are then passed to the measurement of edge density for computing effective correlation areas. Experimental results on a new dataset show that the correlation system can make use of the best properties of alignment and effective correlation area-based approaches, and can attain significant improvement of image-correlation results, compared with the traditional image-matching methods for firearm identification, which employ cartridge-case samples. An analysis of image-alignment score matrices suggests that all translation and scaling parameters are estimated correctly, and contribute to the successful extraction of effective correlation areas. It was found that the proposed method has a high discriminant power, compared with the conventional correlator. This paper advocates that this method will enable forensic science to compile a large-scale image database to perform correlation of cartridge case bases, in order to identify firearms that involve pairwise alignments and comparisons.

Zhang, S. and Chumbley, L.S., “Manipulative Virtual Tools for Tool Mark Characterization”, NCJRS Document #241443, Award # 2009-DN-R-119, March 2013.

This paper describes research on the development of virtual toolmarks by a 3-D computer simulation that would allow for the development of highly predictable toolmark characterizations. Initial study involved the production of test toolmarks by six screwdriver tips that were then compared by a previously developed statistical algorithm.

Preliminary experimental results indicated that the use of a manipulative, virtual tool could provide quantitative data for the characterization of tool marked surfaces that would improve the scientific basis of toolmark identification.

Song, J., et al., “Development of Ballistics Identification- from Image Comparison to Topography Measurement in Surface Metrology”, Measurement Science and Technology, Volume 23, Number 054010, March, 2012.

This was a systematic study of direct measurement and correlation of surface topography on fired bullet markings. Based on this on this system, a prototype for bullet signature measurement and correlation was developed that has demonstrated superior correlation results for bullet signature identifications.


of 23

Chu, W., et al., “Selecting Valid Correlation Areas for Automated Bullet Identification System Based on Striation Detection”, Journal of Research of the National Institute of Standards and Technology, Volume 116, Number 3, May-June 2011. This paper detailed a study on fired bullet markings using automated bullet identification systems that employ an edge detection algorithm and selection process that locates the edge points of significant toolmark features was conducted. Results of this study validated the differentiation ability of individual characteristics if a proper striation threshold length could be established.

Weavers, G., et al, “A Comprehensive Statistical Analysis of Striated Tool Mark Examinations, Part 2: Comparing Known Matches and Known Non- Matches using Likelihood Ratios”, AFTE Journal, Volume 43(2), Spring 2011, pp. 137-145. A potential model for increasing the objectivity in the interpretation of toolmarks is explored using consecutively matching striae (CMS) and Bayesian inference. Given the nature of the data, standard statistical thinking suggests that Bayesian inference is likely to be the most powerful method of interpretation. The unavoidable paucity of data for high CMS runs for the known non-match condition is handled using a small advance in modelling. The resulting likelihood ratios show some, but incomplete separation between the known match and known non-match conditions. Although promising, the resulting incomplete separation between known match and known non-match is thought to represent limitations of the CMS summary of the complete pattern and limitations of the modelling used.

Baldwin, et al, “Statistical Tools for Forensic Analysis of Toolmarks”, Ames Laboratory, Iowa State University, Report IS-5160, 2011 Recovery and comparison of toolmarks, footprint impressions, and fractured surfaces connected to a crime scene are of great importance in forensic science. The purpose of this project is to provide statistical tools for the validation of the proposition that particular manufacturing processes produce marks on the work-product (or tool) that are substantially different from tool to tool. The approach to validation involves the collection of digital images of toolmarks produced by various tool manufacturing methods on produced work-products and the development of statistical methods for data reduction and analysis of the images. The developed statistical methods provide a means to objectively calculate a "degree of association" between matches of similarity produced toolmarks. The basis for statistical method development relies on "discriminating criteria" that examiners use to identify features and spatial relationships in their analysis of forensic samples. The developed data reduction algorithms utilize the same rules used by examiners for classification and association of toolmarks. Q2 Part 2: What publicly accessible databases exist that could support such studies? What closed databases exist? Where such databases exist, how are they controlled and curated? Databases designed to establish the baseline frequencies of characteristics or features used to establish identity for forensic firearm and toolmark comparisons currently do not exist.


of 23

Q2 Part 3: If studies have not been conducted, what conclusions can and cannot be stated about the relationship between the crime scene evidence and a known suspect or tool (e.g., firearm)?

The conclusions that can be rendered between two toolmarks are Identification, Elimination, Inconclusive and Unsuitable, and are defined below:

AFTE Range of Conclusions Identification: Agreement of all discernible class characteristics and sufficient agreement of a combination of individual characteristics where the extent of agreement exceeds that which can occur in the comparison of toolmarks made by different tools and is consistent with the agreement demonstrated by toolmarks known to have been produced by the same tool.

Inconclusive: A. Agreement of all discernible class characteristics and some agreement of individual characteristics, but insufficient for an identification. B. Agreement of all discernible class characteristics without agreement or disagreement of individual characteristics due to an absence, insufficiency, or lack of reproducibility. C. Agreement of all discernable class characteristics and disagreement of individual characteristics, but insufficient for an elimination.

Elimination: Significant disagreement of discernible class characteristics and/or individual characteristics.

Unsuitable: Unsuitable for examination.

Q3: How is performance testing (testing designed to determine the frequency with which individual examiners obtain correct answers) currently used in forensic laboratories? Are performance tests conducted in a blind manner? How could well-designed performance testing be used more systematically for the above pattern-based techniques to establish baseline error rates for individual examiners? What are the opportunities and challenges for developing and employing blind performance testing? What studies have been published in this area?

Many forensic laboratories require competency testing prior to authorization for a forensic practitioner to independently evaluate evidence.

Proficiency testing is a valuable component to measure the performance of individual examiners and the procedures, methods and practices utilized by the laboratory. Forensic laboratory accreditation bodies generally require each laboratory participate annually in proficiency tests provided by an external vendor, if available. Currently, the requirements do not mandate that each examiner participates in an external proficiency test, though most forensic laboratories exceed this standard and require that each examiner participates in an externally provided proficiency test. There are currently two (2) vendors that provide external proficiency tests in the area of Firearms and Toolmark Identification. One of the vendors does not provide, report or publish a statistical evaluation of the compiled results submitted at this time; however, laboratories can review the test summary provided for a particular test to extrapolate this


of 23

information. The other vendor is offering a proficiency testing scheme with calculations of statistics relevant to the forensic science and legal communities to include false positive and false negative error rates, as well as sensitivity and specificity for each test.

Angela Stroman, in the “Declared vs. Blind Testing” section of her recent research paper entitled “Empirically Determined Frequency of Error in Cartridge Case Examinations Using a Declared Double-Blind Format” AFTE Journal, 46(2), Spring 2014, pp. 157-175, did an especially cogent job of describing the current status of proficiency testing in firearm and toolmark identification, and for that reason, it is attached here in its entirety.

Attachment (Click on icon to open document):

Q4: What are the most promising new scientific techniques that are currently under development or could be developed in the next decade that would be most useful for forensic applications? Examples could include hair analysis by mass spectrometry, advances in digital forensics, and phenotypic DNA profiling.

There are currently no quantitative criteria widely utilized for the identification of toolmarks; however, within the past 5 years, there has been significant progress in this area through research in the optical topographical analysis of toolmarks. This is the most promising new technique in the area of firearm and toolmark identification.

The extent of progress in the optical topographical analysis of toolmarks was brought into sharp focus recently with the formation, by RTI International Forensic Technology Center of Excellence, in partnership with the National Institute of Justice (NIJ) and the National Institute of Standards and Technology (NIST), of the “Forensic Optical Topography Working Group”. The final report, dated April 17, 2015, on their March 17-18, 2015 meeting, is attached. In the “Overview” portion of this report, it is stated that “this working group seeks to establish the applicability and validity of optical topography to forensic investigations and to produce publications or training materials that can be accessed by the entire forensic community and that will provide guidance to practitioners on applications and recommendations for further research, development, and capacity assistance. Primarily, the working group will examine optical topography instruments, methods, data systems, and analysis from a practical perspective for ballistic and tool mark identification”.


Q5: What standards of validity and reliability should new forensic methods be required to meet before they are introduced in court?

Validation is the process by which the scientific community acquires the necessary information to (a) assess the ability of a procedure to obtain reliable results, (b) determine the conditions under which such results can be obtained, and (c) define the limitations of the procedure. New forensic methods which have not been scientifically validated or has been validated but not


of 23

adopted for use in the field of forensic science should undergo a developmental validation process before they are introduced in court. Developmental Validation should include:

1. Literature references: Review of publications, academic materials, etc. involving thetechnique or procedure being validated.

2. Simulated casework samples which are representative of the samples routinely analyzedusing the technique or procedure.

3. Accuracy/Precision Studies: The results must demonstrate that the method is capable ofdelivering the level of accuracy and precision required for the particular application ofthe method. The accuracy (proximity to accepted values) and precision (acceptable levelof variability) must be demonstrated to be acceptable for forensic casework.

4. Reproducibility: The test must be reproducible by another individual using the originaltest documentation.

5. Specificity: Where applicable, the method should be demonstrated to yield results whichare specific to the items analyzed.

6. Sensitivity Studies: The sensitivity of the method should be demonstrated when relevantto the validation process.

A new technique or method requires more thought and subsequent testing to properly satisfy validity and reliability issues. By way of an example, recent and rapid developments have taken place in the field or digital imaging of fired bullets and cartridge cases. A comparison of images of these items taken through a traditional optical microscope with digital images of the same objects generated with this ‘new’ technology are visually striking. [See Figure 1 and Figure 2] So much more detail becomes visible in the toolmarks on these ballistic items. Moreover, previous problems with specular reflections (“hot spots”) with traditional illumination of shiny surfaces are totally obviated with these digital imaging systems. Conversely, areas that are dark under normal illumination are easily seen as gray scale images with these same digital systems. The two attached figures show a cartridge case comparison and a bullet comparison with a traditional optical comparison microscope and one of the current digital scanning systems. One might argue that the substantially superior nature of the images generated by the digital scanning system are self-evident or self-authenticating, and that a court should easily be able to see the improvement offered by such a digital scanning system. But lacking expertise in firearms and toolmark examination on the part of a judge, an alternate and more appropriate procedure for validity and reliability, suitable for peer review using this example of a ‘new’ technique, would be as follows:

1. Select a polygonally-rifled firearm such as a Glock or H/K P2000, and ensure (through asubsequent bore cast) that the bore is unique by minimally lapping it with fine grain SiCin a liquid base. [Note: this type of barrel is chosen because it is often very difficult toimpossible to match test-fired bullets under the conventional optical comparisonmicroscope]

The lapping process will produce micro-imperfections in the bore in a random mannerthereby rendering the barrel unique.

2. Prepare indexed, test-fired bullets after multiple shots (5-10 shots) to assure that the“settling in” process is complete.


of 23

3. Verify that these bullets cannot be definitively matched using a state-of-the-art opticalcomparison microscope.

4. Prepare photomicrographs showing the best (if any) areas of marginal agreement on thesetest-fired bullets.

5. Scan and re-examine all test-fired bullets using one of the state-of-the-art digital imagingsystems such as Evofinder, IBIS Trax-HD3D, or LUCIA Bal-Scan.

6. Record the best matches with digital imaging system.

7. Prepare side-by-side comparisons between the results for the same areas with the opticalcomparison microscope and the digital imaging system.

8. Repeat the experiment with other barrels producing difficult to impossible to match test-fired bullets.

Validity and reliability in this example are established with the repeated success of the digital imaging system with its demonstrated ability to make visible unique striae patterns not discernible with the traditional optical comparison microscope. Subsequent peer review by the relevant scientific community would also represent an important consideration if, and when, critics raise a legal challenge to the use of this new technology.

FIGURE 1: CARTRIDGE CASE COMPARISON


of 23

FIGURE 2: BULLET COMPARISON

Note the dark, soot-stained surface of these two bullets when viewed and photographed under the optical comparison microscope. This dark material presents no problem for the digital imaging system employed here. Moreover, a much better comparison appears in the digital image on the right.

Q6 Part 1: Are there scientific and technology disciplines other than the traditional forensic science disciplines that could usefully contribute to and/or enhance the scientific, technical and/or societal aspects of forensic science?

For many years the Firearm and Toolmark community has been left to their own intrigue and dedication to investigate unanswered questions within the discipline as the primary source of research. However, as will be seen in the literature that is cited in this response, one will see that collaboration with Universities and research scientists has become more prevalent. Iowa State University, John Jay College, University of California at Davis are just a few of those universities that have taken up specific research in the field of Firearm and Toolmark Examination. NIST researchers have also contributed significantly to this research effort.

In the most recent history of research within the discipline, our profession has begun collaboration with computer scientists utilizing machine learning algorithms. Machine learning is a sub discipline of computer science that seeks to teach computers how to recognize (and compare) patterns. Since the comparison of toolmarks is the comparison of patterns, the collaboration between firearms and toolmark examiners and machine learning computer scientists is a collaboration that has started to produce interesting research papers.


of 23

Metrology is a second discipline that has enhanced the science of firearm and toolmark identification. Metrology is the science of measurement. In order to use computer machine learning algorithms to compare toolmarks, the toolmarks must be accurately measured. This is where the metrology scientists have (and will) help the forensic community evaluate and implement the best technology for the task at hand.

Q6 Part 2: What mechanisms could be employed to encourage further collaboration between these disciplines and the forensic science community?

The Organization of Scientific Area Committees, established by the National Institute of Standards and Technology (NIST) has as a primary goal to answer this very question. The majority of forensic science disciplines have now been brought together within one entity with a purpose of establishing scientifically sound standards of practice within each discipline. The ability to share knowledge and research and to collaborate between like disciplines is now a greater possibility, which will only serve to enhance the technical and societal impacts of forensic science.

Q7: Please share any additional comments.

On June 14, 2011, AFTE submitted a 94 page response to 25 foundational questions on firearm and toolmark examination submitted by the Subcommittee on Forensic Science (SoFS), Research, Development, Testing, & Evaluation Interagency Working Group (RDT&E IWG). This response consisted of a compilation of numerous references, with abstracts, that AFTE felt provided the scientific underpinnings of forensic firearm and toolmark identification. The entire document can be accessed by going to the AFTE website and looking under the “Resources” tab and then “AFTE Position Documents”.

The SoFS RDT&E IWG felt that if a forensic specialty, like firearm and toolmark identification, could respond to their 25 questions by providing sound, peer-reviewed, references that they probably rested on firm scientific underpinnings. AFTE was one of the first, if not the first, to provide an underpinning compilation list to the RDT&E IWG.

The SoFS RDT&E IWG intended to have someone evaluate these articles to determine whether or not they actually did provide a firm scientific underpinning. However, despite good intentions, they were not able to have this evaluation done prior to the expiration of their charter.

In late 2014 or early 2015, however, it was announced that the American Association for the Advancement of Science (AAAS) had been funded to conduct a quality and gap analysis of the underpinning compilations submitted to the SoFS RDT&E IWG by ten forensic disciplines, including firearms and toolmarks. To date, there has been no public announcement regarding the state of these evaluations by AAAS.

We have attached the letter written to AAAS, a copy of the cover letter the entire compilation provided by AFTE to SoFS/RDT&E IWG.


Image courtesy of Contra Costa County, CA forensic laboratory.

National Institute of Justice

Forensic Optical Topography Working Group Meeting

FINAL REPORT

Meeting Date: March 17–18, 2015

Report Date: April 17, 2015

National Institute of Justice Office of Investigative and Forensic Sciences 810 Seventh Street, N.W. Washington, D.C. 20531 FY2011 Award #2011-DN-BX-K564

FINAL REPORT: Forensic Optical Topography Working Group

NIJ FTCoE (2011-DN-BX-K564) ii | P a g e

Forensic Technology Center of Excellence

The Forensic Technology Center of Excellence (FTCoE) is a collaborative partnership of RTI International and its FEPAC [Forensic Science Education Programs Accreditation Commission]–accredited academic partners: Duquesne University, Virginia Commonwealth University, and the University of North Texas Health Science Center. In addition to supporting the National Institute of Justice’s (NIJ’s) research and development (R&D) programs, the FTCoE provides testing, evaluation, technology transition assistance, and other services for use by crime laboratories, forensic service providers, law enforcement, and other

criminal justice agencies whose mission is to combat crime. NIJ funds the FTCoE to transition forensic science and technology to practice (Award Number 2011- DN-BX-K564).

The FTCoE is led by RTI, a global research institute dedicated to improving the human condition by turning knowledge into practice. With a staff of more than 3,700 providing research and technical services to governments and businesses

in more than 75 countries, RTI brings a global perspective. The FTCoE builds on RTI’s expertise in forensic science, innovation, technology application, economics, data analytics, statistics, program evaluation, public health, and information science.


NIJ FTCoE (2011-DN-BX-K564) iii | P a g e

Contents

1. Overview ............................................................................................................................................... 1

2. Goals and Objectives ............................................................................................................................ 1

3. Topics Discussed ................................................................................................................................... 2

3.1 Elements of Firearm and Tool Mark Identification and Relation to 3D Imaging ........................ 3

3.2 Considerations for Optical Topography in Forensic Science .................................................... 5 3.2.1 General Observations in Conventional Optical Microscopy vs Topographical

Microscopy .............................................................................................................. 5

3.2.2 Classification of Surface Topography (3D) Measurement Methods, ............................. 6

3.2.3 Methods Relevant to Firearm and Tool mark Identification ......................................... 6

3.2.4 Calibration Issues and Standards ............................................................................... 9

3.3 Ballistic and Tool Mark Identification Reference Data and Collection ...................................... 9

3.4 NIST Ballistics Tool Mark Database .................................................................................... 10

3.5 Applications of Optical Topography ................................................................................... 11

3.6 Tool Mark Examination ..................................................................................................... 12

3.7 Effect of Instrumental Variability on Analysis Algorithms ..................................................... 13 3.8 Considerations for the Field, Instrumentation: Types and Costs, Statistical Methods,

Training, Integration with Crime Laboratory Operations ...................................................... 16

3.8.1 Panel Discussions ................................................................................................... 16

3.9 State of the Industry: Overview of Available Instrumentation ............................................. 19

3.10 Challenge of Subclass Characteristics ................................................................................. 19

3.11 Research Needs in the Application of Confocal Microscopy to Ballistic Imaging .................... 20

3.11.1 Research to Date .................................................................................................... 20

3.11.2 Research Needs ..................................................................................................... 21

3.11.3 Other Discussions .................................................................................................. 22 4. Summary ............................................................................................................................................. 22

Bibliography ................................................................................................................................................ 23


NIJ FTCoE (2011-DN-BX-K564) iv | P a g e

Tables

1. Types and names of measurement standards .................................................................................. 9 2. Early Firearm and Tool Mark Research ........................................................................................... 20

Figures

1. Topographical image of breech face and firing pin impressions ...................................................... 7 2. Topographical image of a pair of fired bullets .................................................................................. 8 3. Firing pin impression on cartridge case measured with focus variation. Overlay of

reflectance and topography images. ............................................................................................... 8 4. Four components of surface texture ............................................................................................. 11 5. Specific source distribution ............................................................................................................ 15 6. Common source distribution ......................................................................................................... 15


NIJ FTCoE (2011-DN-BX-K564) 1 | P a g e

1. OVERVIEW The Forensic Technology Center of Excellence (FTCoE) at RTI International, in partnership with

the National Institute of Justice (NIJ) and the National Institute of Standards and Technology (NIST), convened the Forensic Optical Topography Working Group Meeting on March 17 and 18, 2015 at the NIST campus in Gaithersburg, MD. The meeting included researchers and practitioners with a wide range of experience and knowledge regarding forensic applications of microscopy.

This working group seeks to establish the applicability and validity of optical topography to forensic investigations and to produce publications or training materials that can be accessed by the entire forensic community and that will provide guidance to practitioners on applications and recommendations for further research, development, and capacity assistance. Primarily, the working group will examine optical topography instruments, methods, data systems, and analysis from a practical perspective for ballistic and tool mark identification.

The meeting participants considered current technologies for optical topography, including the requirements for systems that may be deployed in crime laboratories. The extension of current ballistic identification methods to topographic methods was also examined. Participants noted the value of the comparison microscope to identification decisions in current practice and determined that it was unlikely that optical topography would supplant the comparison microscope as the primary tool for the forensic examiner in the near term. Instead, optical topography is likely to be a confirmatory tool or a method to examine very difficult comparison cases.

Participants reviewed current and past efforts to implement optical topography in the crime laboratory, including the application of confocal microscopy. These systems demonstrated that optical topography may permit the examiner to distinguish among consecutively manufactured firearms, although a great deal of work remains to establish accepted examination protocols. For example, data filters and matching algorithms will play a key role in the application of optical topography, but the validation of these analytical tools remains an issue.

NIST and the Federal Bureau of Investigations (FBI) have begun to collect reference data from test fires relevant to a wide range of firearms, including various combinations of research data, crime laboratory data, and instrument types. They have developed an XML data standard under the OpenFMC framework that could be used for interoperable sharing of ballistic identification data on a national basis. Many issues remain, including commercial acceptance, practitioner acceptance, the ability of laboratories to handle the large data files and computational load associated with topographic systems, and the collection of sufficient data to inform the development of analytical models.

The working group agreed that it would be advisable to hold a practical review of examination methods at the FBI Laboratory, which houses several optical topography instruments. The review would seek to establish current consensus concerning the application of optical topography to ballistic identification with respect to examiner practices, instrument requirements, training, and analysis. A small subgroup will meet at the FBI Laboratory at a date to be determined to complete this task.

2. GOALS AND OBJECTIVES This working group will review the various technologies associated with the collection of optical

topographic data, i.e., three dimensional (3D) data from surfaces using optical means, including confocal microscopy, interferometry, and focus variation. The working group confines its work to firearm identification and tool mark comparison, although it will briefly review other forensic applications.



The goals of the working group are as follows:

• Determine how to improve practitioner access to optical topography instrumentation and methods.

• Determine how optical topography may improve the ability to individualize firearms and tool marks and provide a more objective and reliable basis for forensic comparisons. These comparisons are expert-based. Although we do not yet have a way to develop a probabilistic framework at this time, we can try to determine how these methods can be used to improve the ability of the individual examiner to quantitate their findings.

• Stimulate future research and development in optical topography, and identify gaps in the research portfolio.

The National Institute of Justice (NIJ) has been funding research in optical topography for over 6 years. Nonetheless, few state and local laboratories have implemented the technology. The working group will consider specific issues relevant to the gap between research and practice, including

• Is this an area that NIJ should continue to fund? Research has shown that this technology can be important, but is there a practical application?

• What are the barriers that prevent transition of optical topography from research to forensic practice?

• What information and resources are needed by labs to implement these systems?

• What investment is needed at the laboratory to make the transition happen?

• What are the specifications of current systems? Is the technology mature enough to be used in practical applications?

• Do we have enough knowledge of the accuracy of the techniques?

• Is the technology ready for casework?

• Will the instrumentation cost decrease as the technology becomes more mature?

What is the current state of research in the area of optical topography? Where will the technology be in 3 to 4 years? The group will also examine the extension of current procedures for comparisons based on two-dimensional (2D) image data to 3D topographic images. As part of this latter task, the group will develop process maps that capture current and proposed comparison methods, including aspects related to data interpretation, such as baseline correction. In addition, the group will examine ways to improve the ability of NIST and affiliated organizations to collect reference data that may be used to validate mathematical approaches, although mathematical and statistical analysis will not be emphasized in the current phase of the working group.

3. TOPICS DISCUSSED Topics discussed by the Forensic Optical Topography Working Group are presented in the

following subsections. Presentations for some of these topics can be found at the following links:

• Elements of Firearm and Tool Mark Identification and Relation to 3D Imaging

• Considerations for Optical Topography in Forensic Science

• Firearm and Tool Mark Identification Reference Data and Collection

https://rti.connectsolutions.com/p3lf8cu8hl0/

https://rti.connectsolutions.com/p7brmjwqrn2/

https://rti.connectsolutions.com/p633qya5ewy/



• Applications of Optical Topography

• Tool Mark Examination

• Effect of Instrumental Variability on Analysis Algorithms

3.1 Elements of Firearm and Tool Mark Identification and Relation to 3D Imaging Firearms identification is the forensic science discipline that identifies bullets, cartridge cases,

or other ammunition components as having been fired from, or worked through the action of, a particular firearm to the practical exclusion of other firearms. A gun is a type of tool that produces marks on bullets and cartridge cases because the metals in bullets and cartridge cases are necessarily softer than those in the firearm. Firearms identification is based on the principle that most of the tool marks that firearms produce on bullets and cases are characteristic of the individual firearm. For example, tool marks are left on the bullet from the barrel rifling, resulting in striations in the land and groove impressions.

Rifling impressions are an indication of the source of the barrel. In part, the type of barrel may be identified because of variations among manufacturers with respect to the design of firearms.

Also, each barrel may be different from others of the same type and, therefore, may be individualized based on imperfections left in the barrel during the manufacturing process and visible at the microscopic scale. The firearm examiner may use these tool marks to identify class characteristics—for example, the type of firearm that fired a bullet. The examiner may also identify marks that permit the individualization to a particular firearm. Tool marks are generally present as either striated or impressed tool marks.

Striated tool marks are scratches or scrape marks that appear as parallel lines, or striae, along the bullet, cartridge, or cartridge case. Impressed tool marks are formed when the cartridge or bullet are forcefully pressed against another surface and give the appearance of being stamped into the metal. Striae arise from rifling marks on bullets, while impressions are left by the impact of the breech face and firing pin on a cartridge case. Forensic examiners use comparison microscopes to identify impressed and striated tool marks.

The comparison microscope was a major development in forensic firearm identification and consists of two microscopes joined together by an optical bridge. The comparison microscope is a minimum requirement to perform firearm and tool mark identification. The decision concerning whether a particular impression or striae is a match to another impression or striae is a subjective judgment based on the human examiner’s pattern-matching ability and experience. This subjective identification can lead to questions, such as the following:

• How much similarity is enough for identification?

• How small a shard can you get to match with practical certainty?

• How does human cognition factor in?

• How many points of comparisons are required to confirm an identification?

• Should examiners report an identification based on a sufficient number of points of comparisons?

• What is the minimum criteria required to confirm identification, etc.?

https://rti.connectsolutions.com/p5pmtzr3q6h/

https://rti.connectsolutions.com/p64vjx524u3/

https://rti.connectsolutions.com/p76g7nc8boj/





In 1992, the Association of Firearm and Tool Mark Examiners (AFTE) adopted the “Theory of Identification,” which was later updated in 2011 and reads1

1. The theory of identification as it pertains to the comparison of tool marks enables opinions of common origin to be made when the unique surface contours of two tool marks are in “sufficient agreement.”

2. This “sufficient agreement” is related to the significant duplication of random tool marks as evidenced by a pattern or combination of patterns of surface contours. Significance is determined by the comparative examination of two or more sets of surface contour patterns comprised of individual peaks, ridges, and furrows. Specifically, the relative height or depth, width, curvature, and spatial relationship of the individual peaks, ridges, and furrows within one set of surface contours are defined and compared to the corresponding features in the second set of surface contours.

3. Agreement is significant when the agreement in individual characteristics exceeds the best agreement demonstrated between tool marks known to have been produced by different tools and is consistent with agreement demonstrated by tool marks known to have been produced by the same tool. The statement that “sufficient agreement” exists between two tool marks means that the agreement of individual characteristics is of a quantity and quality that the likelihood of another tool making the mark is so remote as to be considered a practical impossibility.

Currently, the interpretation of individualization/identification is subjective in nature, founded on scientific principles and based on the examiner’s training and experience. Traditionally, firearms examiners have used their inherent cognitive ability to recognize and compare patterns of consecutive matching striae (CMS) between known and unknown tool mark specimens and determine if sufficient agreement for identification exists based on their recollection of the best known non-matching agreement they ever observed (from their training or other experience). Using this “pattern matching” approach, if the striated agreement exceeds this non-quantified threshold in the mind’s eye of the examiner, then, assuming the absence of subclass influences, the examiner may conclude there is sufficient agreement for identification of the two marks to a single source. Although this method is subjective, the identification criteria used in pattern matching can be very accurate, although error rates are not well understood. In 1997, conservative quantitative consecutive matching striae (QCMS) criteria for the identification of striated tool marks was proposed by Biasotti and Murdock. Using QCMS, sufficient agreement is defined as two (2) runs of three (3x) CMS or one (1) run of six (6x) CMS for 3D marks and two (2) runs of five (5x) or one (1) run of eight (8x) CMS for 2D marks, assuming the absence of subclass influences (Modern Scientific Evidence: The Law and Science of Expert Testimony [West Group, 1997 & Supps., 1999-2001]). In actual empirical research, in 3D tool marks, one (1) group of four (4x) CMS is the highest CMS run that has been observed. In practice, examiners have observed that these thresholds exceed the best agreement observed in known non-matching tool marks. Thus, some examiners consider the QCMS thresholds to provide objective criteria for their discipline. The QCMS thresholds were designed to be conservative based on empirical research, which is why the criteria is higher for shallow (2D) or other difficult marks. Various statistical scientists are examining the firearms identification process. This work may benefit from an examination of QCMS criteria. For example, whether striae are considered a match by a human examiner or an algorithm derived from optical topographic data, QCMS may be used to derive a statistical representation of a match. Also, optical

1 Association of Firearm and Tool Mark Examiners, “ Theory of Identification as it Relates to Tool Marks: Revised”, AFTE Journal, Vol. 43, No. 4 (Fall 2011), p. 287.



topography may be used to improve the ability of examiners to match striae, thus improving their ability to make identifications in difficult cases. As adopted in 1992, the range of conclusions was preceded by: “The examiner is encouraged to report the objective observations that support the findings of tool mark examinations. The examiner should be conservative when reporting the significance of these observations.”2 The conclusions of identity are based on the comparison of individual characteristics made after eliminating the possibility of subclass influence. In other words, the presence of some CMS observed between two firearm-produced tool marks may be due to the type of firearm and not indicative of the individuality of one firearm versus another of the same type. The ranges of conclusions are identification, exclusion, inconclusive, and no value, as described below:

• Identification: Agreement of a combination of individual characteristics and all discernible class characteristics, where the extent of agreement exceeds that which can occur in the comparison of tool marks made by different tools and is consistent with the agreement demonstrated by tool marks known to have been produced by the same tool.

• Inconclusive:

o Some agreement of individual characteristics and all discernible class characteristics, but insufficient for an identification.

o Agreement of all discernible class characteristics without agreement or disagreement of individual characteristics due to an absence, insufficiency, or lack of reproducibility.

o Agreement of all discernable class characteristics and disagreement of individual characteristics, but insufficient for an elimination.

• Elimination: Significant disagreement of discernible class characteristics and/or individual characteristics.

• Unsuitable: Unsuitable for examination.

It is important to note that the word “inconclusive” was never intended to be used alone without an explanation in a report of examination, as outlined in a, b, or c, above.

3.2 Considerations for Optical Topography in Forensic Science

3.2.1 General Observations in Conventional Optical Microscopy vs Topographical Microscopy

The implementation of optical topography may present difficulties to crime laboratories. In one case of a laboratory that wanted to purchase a confocal microscope, the facility was located on the 4th floor, where there were issues with vibration from a nearby highway that would have made the system unusable. The crime lab could not obtain access to the ground floor, so they developed a plan to prepare a room to be vibration-free, but the cost of implementing the plan would have been $100,000. Thus, the full cost of deploying a confocal microscope included the system itself, training, and substantial facility costs. These ancillary costs are a significant barrier to the broad adoption of optical topography in forensic science.

2 AFTE Committee for the Advancement of the Science of Firearm and Tool mark Identification. (2011, June 14). AFTE Response to the 25 Questions related to firearms and tool mark examinations promulgated by the RDT&E IWG. Retrieved March 27, 2015, from www.AFTE.org: http://afte.org/downloads/RDT&E%20IWG%2025%20Questions%206.14.11%20- %20AFTE%20Response%20w%20cov%20let.pdf

https://rti.connectsolutions.com/p7brmjwqrn2/

http://www.afte.org/

http://afte.org/downloads/RDT%26E%20IWG%2025%20Questions%206.14.11%20-%20AFTE%20Response%20w%20cov%20let.pdf





In conventional optical microscopy, the optical image contrast is measured two dimensionally as I (x,y) and is predominantly a function of slope, shadowing, multiple reflections, optical properties, directional illumination, and, indirectly, the local height variations. In topographical microscopy methods such as interferometric and confocal microscopy, the instrument can measure variations in height, i.e., Z (x,y), directly and independent from illumination and shadowing effects. However the data can be subject to signal-to-noise issues, distortions, and data dropouts.

3.2.2 Classification of Surface Topography (3D) Measurement Methods3,4

General considerations may be found for the selection of surface topographic methods in the literature, including ISO Standard 25178-6.3,4 Surface topography measurement methods include Line Profiling, Areal Topography, and Area Integration. Line profiling methods include contact stylus scanning, phase shifting interferometry, circular interferometric profiling, and optical differential profiling. Areal topography includes contact stylus scanning, phase shifting interferometry, coherence scanning interferometry, confocal microscopy, confocal chromatic microscopy, structured light projection, focus variation microscopy, digital holography microscopy, angle resolved scanning electron microscopy (SEM), SEM stereoscopy, scanning tunneling microscopy, atomic force microscopy, optical differential profiling, point autofocus profiling, and photometric stereo. Area integrating methods of surface texture measurement include total integrated scatter, angle resolved scatter, parallel plate capacitance, and pneumatic area integration.

Some of the key limitations for profiling instruments are the spatial resolution, lateral range bandwidth limits as well as the range of vertical and horizontal resolution. Some instruments are limited by their ability to discern very steep slopes, which may show up as artifacts in a 3D image. Therefore, the maximum measureable slope is a critical requirement for surface topographical systems. Similarly, systems must minimize dropouts and outliers and permit the quantitation of these artifacts.

There are standards that should be used by the examiner to ensure the calibration and traceability of their measurements, including NIST sinusoidal reference standards 2073a, 2074, and 2075.

For firearms identification, the relevant field of view ranges from a few microns to a few millimeters. In general, meeting participants believe that the relevant horizontal length scale for optical topography in firearms identification is one micron, although much reference data are based on a horizontal resolution of 3.25 microns, including NIST’s own breech face data. This resolution is based on optical limitations.

The choice of a system must include consideration of cost and speed. In most cases, surface topography systems require some type of scanning in the z-direction (depth), which slows data collection considerably in comparison to traditional, 2D microscopy.

3.2.3 Methods Relevant to Firearm and Tool mark Identification

Excellent and detailed discussion of optical topography methods are available, including (insert Vorburger presentation from the web).

3 ISO Standard 25178-6 (2010), Classification of Methods for Measuring Surface Texture. 4 T.V. Vorburger et al., Int. J. Adv. Manuf. Technol. 33, 110 (2007).



Coherence Scanning Interferometry (CSI),5 also called vertical scanning interferometry or scanning white light interferometry, measures changes in interference signal strength across a surface. In CSI, the object of interest has height features (h) that vary according to the surface of the object. The object is then scanned mechanically, which provides a continuous, smooth scan of the interference objective along the optical axis in the z direction. While the object is scanned, the intensity data, I, is recorded for each image point. As an object is scanned vertically along the optical axis, the interference varies and surface heights are inferred by observing where the interference effect is the strongest. A strength of CSI is vertical resolution of approximately 3 nm with lateral resolution that is comparable to confocal microscopy at approximately 1µm; however, CSI is limited in its ability to optically resolve steep, sloped surfaces. One application of CSI in firearms research is ALIAS, from Pyramidal Technology in cooperation with Heliotis (Switzerland), where CSI was coupled with a high-speed camera to capture topographical images.

Figure 1. Topographical image of breech face and firing pin impressions6

Confocal microscopy7 includes disc scanning confocal microscopy, laser scanning confocal microscopy, and programmable array confocal microscopy. Confocal microscopy works by assembling a series of thin slices of the cross section of the object of interest, taken along the vertical axis. Once assembled, these thin sections can build a very detailed 3D image of the object of interest. The strength of confocal microscopy lies in the vertical resolution of approximately 3 nm and lateral resolutions of approximately 1µm, both similar to CSI. The limitation of this method is that the signal decreases and can become unreliable for surfaces with steep slopes, with dropouts and outliers present at 15º.

5 ISO Draft International Standard (DIS) 25178-604, Geometrical product specification (GPS) – Surface texture: Areal – Part 604: Nominal characteristics of non-contact. 6 www.pyramidaltechnologies.com 7 ISO New Work Item 25178-607, ISO/Technical Committee 213/ Working Group 16 , Geometrical product specification (GPS) – Surface texture: Areal – Part 607: Nominal characteristics of non-contact (confocal microscopy) instruments.

http://www.pyramidaltechnologies.com/



Figure 2. Topographical image of a pair of fired bullets8

Focus Variation Microscopy9 is specifically designed for surface metrology and can be used to characterize surface texture. Focus variation works by moving the shallow depth of focus of the optical system over the object of interest while continuously scanning the surface to produce a 3D model. As the optical lens changes in distance from the object, the variation in sharpness or focus is measured and used to determine depth. The advantages of Focus Variation are that the image is produced in true color and that steep sloped surfaces can be measured. The limitations of focus variation are the vertical resolution of approximately 100 nm and the lateral resolution of several pixels.

Figure 3. Firing pin impression on cartridge case measured with focus variation.

Overlay of reflectance and topography images.

8 From P. Murphy et al., Three-Dimensional Virtual Comparison Microscope for Bullets, http://www.forensictechnology.com/Portals/71705/docs /technote_3dvcmbullets_20100429.pdf (May, 2010). 9 ISO Final Draft Int. Std. (FDIS) 25178-606 Geometrical product specification (GPS) — Surface texture: Areal —Part 606: Nominal characteristics of non-contact (focus variation).

http://www.forensictechnology.com/Portals/71705/docs/technote_3dvcmbullets_20100429.pdf



Some other novel 3D imaging techniques with forensic application include photometric stereo10 and chromatic confocal microscopy.11

3.2.4 Calibration Issues and Standards

Table 1. Types and names of measurement standards

Type Name

A Depth measure standard B Tip condition measurement standard C Spacing measurement standard D Roughness measurement standard E Profile coordinate measure standard

At what length level at x,y,z is it relevant to calibrate these instruments? At what level would they have to be the same to say they truly match? Calibration should correspond to the length scale of the features that one wishes to image. As outlined in Table 1, Current Standard Reference Materials (SRM) were developed for the calibration of the Integrated Ballistic Identification System (IBIS), which is used for database searches of unknowns. These SRMs are for two automated systems, not comparative systems such as the comparison microscope or optical topography. They are relevant to checks on methods, not instruments. Of course, the level of calibration is also limited by the type of optical microscope that is used.

Daubert challenges may be aimed toward questioning the subjective aspects of the comparisons and conclusions rendered by the firearm examiner; however, such subjective assessments may be sufficient and even superior to the quantitation that could be provided by optical topography. In any case, calibration and instrumentation issues remain open questions in this regard.

Overall, the important properties of topographical microscopes for firearm and tool mark analysis are vertical and lateral resolution; maximum measurable slope (important for measuring firing pin impressions); minimization and quantification of dropouts and outliers; cost; and speed.

3.3 Ballistic and Tool Mark Identification Reference Data and Collection Instruments for confocal microscopy, focus variation microscopy, photometric stereo

microscopy, coherence scanning interferometry, and stylus profilometer all give x,y,z data (µm). However, all instrument manufacturers save the data in their own proprietary format, which prevents interoperability, interlaboratory comparison, and certain types of analysis. To improve interoperability, NIST has created an Open Forensic Measurement Consortium (OpenFMC) with the goal of establishing a standard file format for the exchange of 3D forensic topography measurements. The primary point of contact for the OpenFMC is Alan Zheng of NIST. OpenFMC has standardized the use of the XML 3D Surface Profile (X3P) format defined in the ISO 25178-72 standard (OpenGPS). Currently, Sensofar and Cadre Research are the only companies participating in the OpenFMC.

10 M.K. Johnson et al., “Microgeometry Capture using an Elastomeric Sensor”, ACM Trans. Graph. 30, 4, Article 46 (July 2011), DOI = 10.1145/1964921.1964941. 11 ISO 25178-602: 2010 Geometrical product specifications (GPS) — Surface texture: Areal — Part 602: Nominal characteristics of non-contact (confocal chromatic probe) instruments.

https://rti.connectsolutions.com/p633qya5ewy/



OpenFMC a binary file format that contains four records:

1. Header, data types, and axes definition,

2. Metadata regarding the instrument and user,

3. Profile data (x,y,z), and

4. Checksum of the xml-document.

Metadata can include any type of information, including user-defined fields that would enable the use of X3P in crime laboratory information systems. The consortium has developed open-source read/write function converters to enable the adoption of the standard by commercial vendors. Currently, file metadata include information on the firearm, ammunition, measurement conditions, and other elements useful for the development of a reference collection relevant to scientific and statistical study. The XP3 format contains undefined fields that allow for any additional information to be collected in a fifth record. The XP3 format exists on most instruments, so many laboratories can utilize it already, though adoption is not universal.

X3P is compressed data to minimize the storage size. For example, an ASCII file would be about 50 MB, whereas X3P files are 25 MB. With X3P, cross-modality matching the same test fire measured on two different systems can be compared. To test the interoperability of this file system, Cadre research successfully imported NIST Nanofocus confocal data into their TopMatch matching correlation software and were able to correctly identify all test fires using their TopMatch GelSight system. Thus, interoperability between collection methods and laboratories is possible, even when the optical topography instruments are very different.

3.4 NIST Ballistics Tool Mark Database NIST has established a reference collection of optical data for firearms at

www.nist.gov/forensics/ballisticsDB. This database is an open–access, research database containing reflectance microscopy images as well as 3D surface topography tool mark data. This database was created to improve the transition of 3D surface topography from research to application and to improve method development and validation, as well as to allow for development of uncertainty estimates for objective ballistic identification. This database is also being used to validate new algorithms. The horizontal resolution in breech face impression images is 3.25 microns in the database, which is limited by the optical methods used.

The database contains a vast array of test fires collected from consecutively manufactured slides, consecutively manufactured barrels, and persistence/decay studies, as well as collected test fires from different firearms and ammunition. The FBI actively collaborates with NIST in the development of the reference collection. The FBI is conducting an extensive collection of data to add to the database, including test fires from each of the firearms in its collection. This includes 1,038 different models. The FBI has a broad set of optical collection systems (Focus Variation, Confocal, Interferometry [PSI, VSI] Holography, Fringe Projections), all of which will be used in its collection activities. NIST would like to add more crime laboratory sets and firearm types to its collection. NIST also intends to create a limited access database that can be used to validate algorithms for interpretation of firearms identification, such as data search algorithms.

The primary point of contact for the NIST database is Dr. Alan Zheng, who also provides leadership for the OpenFMC.

http://www.nist.gov/forensics/ballisticsDB

http://www.nist.gov/forensics/ballisticsDB



3.5 Applications of Optical Topography 3D imaging technologies have been utilized in many industries, including semiconductor,

materials, paper, energy, optics, forensics, and other fields that are concerned with surface texture analysis. Surface texture consists of micro-roughness, roughness, waviness, and form:

• Micro-roughness is the finest component of the surface texture and is defined as the set of high frequencies or smallest wavelengths (2.5µm, -8 µm) resulting from either sampling noise or the microscopic relief of the structure of the material.

• Roughness is defined as wavelengths ranging from 20 µm to 500 µm and can vary rapidly, depending on the horizontal position. The roughness is decisive for the texture and gives an indication of the nature of the materials, production process, and the machining methods used. However, roughness is not unique; it’s the extraction of the roughness and the characterization of those features that gives the object uniqueness.

• Waviness is defined as wavelengths ranging from 0.5 to 2.5 mm and varies slowly depending on the horizontal position. Waviness generally results from vibrations between the work piece and the machining tool. The roughness is superimposed on the waviness.

• Form has the longest wavelength, similar to the wavelength of the object, so it must be removed to analyze the surface texture of the object of interest.

In order to examine the area of interest or the part of the surface to be visualized, it becomes important to filter out the finer irregularities (roughness) and noise (micro-roughness). The proper use of filtering permits the exploitation of areas of interest. There are two types of filtering: low-pass (waviness) filtering and high-pass (roughness) filtering. The most common form of filtering consists of separating data frequencies (or wavelengths) into two parts, the first one encompassing the long wavelengths or low frequencies (waviness), the other one encompassing the short wavelengths or high frequencies (roughness). The waviness and roughness phenomena are separated mathematically.

Algorithms include thresholds (“cut-off”) on each end of the scale of feature wavelength to determine the amount to which roughness or waviness is removed from a set of data. The quality of the separation depends both on the type of filter and the cut-off value.

Figure 4. Four components of surface texture12

12 Image from Mountains Map Help Topics.



The FBI utilizes a filter template that does all the calculations automatically after the examiner selects an area of interest. Once the examiner selects an area of interest, they then pick an area between the shoulders of a striae or impression to remove calculation artifacts that would arise from including the shoulder of the feature under examination. The goal is that the area should include 18 peaks and valleys in the calculation. After the noise (micro-roughness) is removed, then the form is removed through the use of a quadratic polynomial filter. In these images, one will observe a background “arc” that can be removed through this quadratic polynomial filter. Higher-order polynomials may be needed if the background arc is more complex. For example, a cubic spline may fit well to the background and permit improved background subtraction. After background correction, roughness is filtered. The examiner then applies the same filters to both bullets under comparison, the software picks the best fit for the two final surfaces, and the profile is extracted.

The state crime laboratory in Alabama conducted a series of studies of confocal microscopy using the Sensofar system with the intent to apply confocal microscopy in case work. They did not apply the technique in actual cases, but they did complete extensive studies demonstrating the effectiveness of confocal microscopy to firearm identification, including the following:

• 10 Ruger Barrel Study – 10 consecutively manufactured barrels - a little overlap because this study did not focus on the best area, and it was representative of only one land striae.

• Adjudicated casework – this was a much larger population with a slight overlap again indicative of only one land.

• Speed comparison (SRM 2460) – evaluated the difference between 1x scanning speed and 4x speed using a 20X objective; this results in the use of 4 micron slices and resulted in a good correlation factor (did 10 scans); up to 12x were attempted but higher speeds were not as effective; a 4x speed corresponds to a 2-minute scan.

• Angle difference: striae left – evaluation of how can positioning of bullet under the microscope affect the result. Even shifted, the results are still good. Software gives a good visual so you can also see that the shift has occurred, and they are not aligned. This is different than rotation, but even with rotations the results are good and the visual can show you that there is a shift, so it’s readily visible that this is a bad scan.

• Also evaluated Nose Up and Nose Down shifts in angle rotation. Optical topography systems and evaluation algorithms are highly sensitive to sample tilt.

• Razor blade study with University of Central Oklahoma.

Additional studies include evaluating subclass carryover and the use of waviness as an exclusion characteristic to distinguish barrels. The data are preliminary right now, but show potential as exclusion characteristic instead of a match criteria.

3.6 Tool Mark Examination The Ames Laboratory/Iowa State University (AL/ISU) research team currently utilizes an optical

system manufactured by Alicona, which was selected over other instrumentation (e.g., stylus profilometer, laser confocal) because it allows a 90-degree scanning angle. The system is very portable and offers 4 µm resolution in x and y, and 1 µm resolution in the z direction. AL/ISU has conducted several studies, including comparison of striated tool marks created by screwdrivers and comparison of quasi-striated tool marks with pliers. The research team is also using computer simulation methods, including portable prototype development and virtual tool mark creation, to provide examiners with new capabilities. The portable prototype is based on the Alicona optical profilometer and will allow

https://rti.connectsolutions.com/p64vjx524u3/



everything from simple visual comparisons to comparisons that include statistical information. Everything is written in open- source software and designed to be interoperable with researchers utilizing different analytical algorithms. The system looks like a simple plate-to-plate visual comparison window, but each side can be manipulated independently and different files can be loaded into each window. The topography is graphed, and correlation can be calculated between the two windows. The system also allows the user to load screwdriver tip files to generate a virtual mark that can be compared to the physical mark seen in the window. Then, the program can calculate a correlation value to statistically verify the match. These marks can be manipulated to change the angle so that they can be calculated over any angular range and statistically compared to the real mark.

In a blind angular prediction study involving 20 tool marks, all tips correctly related to their marks, with 14 out of 20 within 5 degrees, and the remainder within 10 degrees.

Future research at AL/ISU will focus on the ability to generate a better virtual mark, developing advanced algorithms to address ever more complex tool marks, and continuing to develop and enhance the prototype by adding additional algorithms and capabilities.

3.7 Effect of Instrumental Variability on Analysis Algorithms Currently, ballistic imaging techniques are based on comparative methods. Moving forward, the

goal will be quantification of the probative value of firearm and tool mark evidence with the ability to apply weight to the evidence, such as likelihood ratios (LRs) or other techniques such as reporting the p value.

Probative value depends on the different sources of variability, the set of circumstances considered, statistical method, and assumptions. The sources of variability include natural variability from the tool, (type brand make, manufacturer, manufacturing process, raw material), evolution of the tool over time, the use of the tool and the variability within the tool mark, as well as the analytical variability from the make and model of the analytical instrument, instrument settings, calibration, and the operator. Sources of variability can also include differences between statistical models (philosophy, design, and assumptions), hypotheses tested by the model, the sample size, and the computational methods used to estimate the parameters of the model. For example, is the statistical model a test against a family of firearms?; just Beretta firearms?; or a specific type or model of Beretta?, etc. These are just some of the many sources of variability that affect the weight of the evidence.

When making a characterization of tool mark features, different methods of acquisition may measure different features, including the width and angle of the land and grooves. This directly impacts the ability of the examiner to match striae or impressions and to make comparison decisions. In other words, as the examiner moves from the comparison microscope to data-based representations of optical topography, the types of distortion will also change. The comparison microscope introduces instrumental variation and subjective bias from the human examiner. The optical topographic microscope introduces distortions associated with the acquisition methods and post-processing. During acquisition post-processing, the aim is to measure raw data with the highest resolution possible, least instrumental interpretation, and the least distortion.

Hypotheses can be common source or specific source. In common source, multiple objects are tested to determine if the objects originate from a common source. The source may be unknown or unavailable for resampling. A typical example is a determination of whether or not two bullets found at one or more crime scenes were fired by the same firearm when you have two traces but no donor or source of the tool marks. This tests the level of similarity, but not the specificity of the features. Specific source analysis is a test of whether or not multiple objects originate from a single specified source. This

https://rti.connectsolutions.com/p76g7nc8boj/



would occur when the source is available for resampling or can be entirely characterized based on the available samples. A typical example of specific source is a test to determine whether or not a bullet originates from a seized firearm. This tests both the level of similarity and the specificity of those features. When a sample from the putative source entirely characterizes that source, both the common source and the specific source approaches result in the same probative value. An example of this would be the standard DNA profile; however, this is not true for most evidence types even when variability of the control samples is relatively reduced (e.g., fingerprints, firearms)

The aim of integrating statistics to quantify probative value is to quantify the “true probative value” of a trace. Assuming that we can completely define and characterize the different sources of variability of the evidence, there exists a likelihood ratio that can help one select between two alternative hypotheses. The question becomes: is there a metric for achieving the likelihood ratio? One can’t quantify the real likelihood ratio because of the computational assumptions that are necessary to simplify the likelihood ratio calculation, therefore the divergence is calculated to account for all the sources of variability. The likelihood ratio can be calculated based on the ratio of two likelihoods or by a plug-in estimation. The ratio of two likelihoods is based on the unrealistic implication that the likelihood structures are known under both alternatives. Plug-in estimates are the most common approach. The plug-in estimate approach makes assumptions on the stochastic process that gave rise to the objects and uses plug-in estimates for the parameters.

Another statistical approach to quantify the probative value of the evidence is the Bayes Factor. The Bayes Factor is a statistically rigorous summary of the value of the evidence, and is the gold standard if calculated properly. The convergence properties of typical computational methods used to calculate marginal likelihoods for the Bayes Factor are unstable unless there is a very large sample size. Plug-in likelihood ratios and Bayes Factors are very difficult and often impossible to calculate for complex evidence forms such as pattern evidence. The high dimensionality of the random vectors, heterogeneity of the random vectors, and unknown likelihood structures make it more difficult to calculate. These difficulties have led several researchers to attempt to simplify the problem by taking advantage of biometric technology to use scores to estimate the likelihood ratio. Different score-based likelihood ratios have been proposed in the literature, including the non-anchored approach, trace-anchored approach, print-anchored approach, and the asymmetrical approach. However, it is important to understand that score-based methods quantify the probative value of the score, not the probative value of the trace. The area of interest is the probative value of the trace and how the probative value of the score and the probative value of the trace relate to each other. The mathematical calculation is not everything; how that information is reported, documented, and verbally testified to in court is a major issue as well. Is it possible to test the convergence of different types of score-based LRs if there is trace and control material that are normally distributed with known distributions.

In Figures 5 and 6, for various examples of both common and specific source LR analysis, the x-axis value is the real likelihood ratio and the y-axis is the score-based likelihood ratio. The line indicates the path the points should follow if the two likelihood ratios were the same. In some cases, these likelihood ratios will roughly correspond. In every case, there is random variation between the two calculations because information is lost when the original data sets are reduced to raw score numbers. The figures depict situations that vary between specific and common sources and difficulty of examination/comparison.



Figure 5. Specific source distribution

In short, it is important to measure and analyze raw data as much as possible to reduce the possibility of deviation. Staying in the feature space maintains the probative value of the evidence. Current NIJ-funded research is examining the effect of various parameters on the likelihood ratio calculation, including the effect of the physical system, instrumental/analytical systems, and statistical computation approach. Each type of variable affects the “ideal” likelihood ratio.

Figure 6. Common source distribution



The common source distribution in Figure 6 shows the difference between the suspect anchored to the trace anchored at the bottom where the scatter increases. The rarer the source becomes as it moves to the right, the more scatter that is present. If the source is rare the result can be an underestimation of the value of the evidence, and if it is common the value of the evidence can be overestimated.

Unfortunately, it is difficult to know which direction it is moving. Research has tested this empirically with handwriting and found that the probative value of the score is not the probative value of the trace. This can be conservative in some circumstances, but not in all, so research is needed to evaluate when this results in a conservative circumstance versus when the power of the evidence is overestimated. This is also complicated because the less sample that is present and available for testing, the more this relationship degrades.

While there may be a true likelihood ratio for any piece of evidence, that likelihood ratio is only an estimation, and different approximations of the likelihood ratio will provide different probative values. Current NIJ research projects address the convergence. Research has shown that score-based methods do not converge and cannot be used to quantify probative value. There are fields (e.g. AFIS scores) that are currently using score-based methods, although these methods should be limited in their application. However, these could be used in a binary decision engine or database search, as long as it is understood that the score does not imply the probative value of the source.

Other demonstrated methods exist that use scores to approximate likelihood ratios:

• Neumann et al., 2012, Quantifying the weight of fingerprint evidence, RSS Series A

• Guharay et al. 2012, Algorithm for spectroscopic data analysis and outlier detection, DTRA/NSF/NGA Algorithm Workshop, San Diego CA.

• Chumbley et al. 2013, Final Report for NIJ Award 2009-DN-R-119

• Saunders et al., 2015, Final report for NIJ Award 2009-DN-BX-K234.

Quantifying the weight of the evidence can only be done in the feature space of the evidence and needs to rely on untransformed raw data.

3.8 Considerations for the Field, Instrumentation: Types and Costs, Statistical Methods, Training, Integration with Crime Laboratory Operations

3.8.1 Panel Discussions

If a score-based method causes the distortions in the results shown, it is possible that any applied filter could distort the data, and would then effect the probative value of the evidence. A score-based method will distort the evidence and undermine the probative value. The score is any quantity that is calculated between two objects of interest using an algorithm that calculates relative value between the objects. Scores have been used to search and retrieve information from a database, but they cannot be used for probative value in court. Every filtering algorithm will introduce distortions, so it is important to understand the effect of the filtering on the power of any post-filter comparison. A filter may be acceptable if it has been demonstrated that it does not affect the comparison, particularly if it can be reduced to a likelihood ratio analysis.

Courtrooms and judges ask for and want statistics; numbers that do not currently exist. The Judge is interested in probative value of the trace and that value is not currently calculated; therefore, the way examiners testify has to be very specific. The way the field currently works versus how we want



the field to work is very different. The answer may be to quantify the measurement process and guide the examination process but abandon the match score. An exception may be cold case investigation, which might need a score approach. In difficult cases, there may be more differences than there are similarities, and a match score would be unreflective of the value of the evidence. In other words, the examiner attempts to find consecutive matching striae, but it is expected that there is variation from one tool mark to the next, even with identical tool and evidence pairs. Therefore, there will be differences between matching evidence. A comparison is based on examination of positively-matching CMS or impressions without regard for non-matching marks. The field is based on the assumption that non-matching features are unimportant, though this assumption has not been fully tested through research. It is possible that the NIST reference collection could serve as a basis for this type of testing, as well as an examination of the statistical power of QCMS criteria for various types of situations (e.g., firearm manufacturing method, type of round, etc.).

During the discussion, two distinct views were expressed over how optical topography should be used. There are several ways to apply optical topography in current practice:

1. Replace current practice completely with optical topography, with comparison done in the computer. This option is not possible at this time, given the current state of technology, analysis, and understanding of methods. Practitioners rely on the comparison microscope, which has been proven an effective tool for making comparison judgments.

2. Optical topography may supplement current practice by giving the examiner a method to look more closely at striae and impressions that may be difficult to compare under the comparison microscope. This would allow for the examination of difficult comparisons, particularly in cases in which emerging manufacturing methods are making it more difficult to distinguish consecutively manufactured barrels. It would be critical to establish methods to ensure that practice conforms to standards concerning the use and incorporation of optical topographic examination in particular cases, especially because of the extended time it requires to apply the method currently.

3. Optical topography could be used as a confirmatory tool in which the examiner conducts a traditional analysis, finds a match or non-match, and then uses optical topography as a check on the result. The system could be used to produce a match score or likelihood ratio, and the examiner’s comparison microscope-based decision would have no effect on the score. This approach would require validation and approval by ASCLD/LAB, at a minimum, though it is anticipated that the results would be used as a basis of court testimony. Again, the laboratory would need to run all data or a predefined subset in the optical topography system to provide an objective analysis of the examiner’s decision. The examiner would testify that, based on his or her knowledge, training, skill, and ability, he or she believes that this gun fired this bullet, and would present pictures to prove that and indicate that an independent instrument confirmed his or her conclusion.

Optical topography could be applied on an experimental basis only so that methods and experience could be developed to inform one or more of the above scenarios.

Some members of the working group expressed concern over any process of examining a specific source and then using a common source score to make a decision, and then using a statistic to confirm. This process might boil down to having the examiner make a decision and then the information would be put it in the machine to get a number to support decisions that were considered favorable. They felt that the examiner must rely on the tool to make a decision instead of making a



decision then using the tool. Theoretically, this concern could be alleviated if there were specific conditions under which optical topography and statistical matching would be applied.

How does DNA differ? DNA relies on human beings to do the mixture interpretation and then the numbers go into a machine but the quantitative part is much simpler because a 5 allele is a 5 allele. DNA can stay in the space of individual characteristics without worry about class characteristics in the same way.

Mathematical algorithms would give the examiner numbers to show the uncertainty of their testimony. That’s good from a research view but not practical in a crime laboratory setting. The biggest issue may be when an examiner comes to an inconclusive decision, but the machine finds a match. Perhaps the solution is to use error rates as opposed to a likelihood ratio. This would be changing the score to be a binary score between x and y so it’s above a threshold or not. Once you use the score to make a decision, one would not report the score but only the error rate of the score.

From the perspective of the practitioner what’s in it for the examiner?

1. Definitive answers in inconclusive comparisons

2. Get ID comparison results in a more objective way – we work with objective information but the interpretation is a subjective process based on objective criteria. It is the visual comparative examination of topographical features of two different tool marks. The use of CMS is an important element. Tool mark examination is a skill—it depends on the examiner’s cognitive ability and training to build awareness of uniqueness—and science—the validated premise that tool marks can have unique, reproducible striae patterns that can, in most instances, be identified to the tool that created them. This is where we are now, but all examiners would recommend a way to make more sound identifications.

3. Help increase the percentage of correct identifications. There are false positive for striated and impressed tool marks, with a variable error rate based on examiner proficiency. Usually, errors arise from poor assumptions about known, non-matching commonalities. Thus, optical topography may provide an objective basis to improve the rate of correct identifications. Further, optical topography is a powerful research tool to improve the field’s understanding of class characteristics and commonalities that are not related to firearm-to-firearm variability.

The field needs to improve proficiency testing. Retention of information is an issue. At the end of training, an examiner may be proficient but 3 months later would they start making mistakes again. AFTE standards have only recently been improved to ensure that those who fail proficiency testing are removed from the list of certified examiners.

Improved proficiency testing may be based on extracting from examiners what makes them say that there is a match. We could then use that to determine a metric to apply on a comparison to determine the difficulty of that comparison. Where is the line where an examiner should always be able to get the right answer? The solution might be to use replicas like they do in Europe. They represent the full range from easy to very difficult. However, CTS error rate would climb and AFTE doesn’t want that. Europe has higher error rates. Europe stumbled when subclasses where introduced. They stumbled but they learned. CTS provides a proficiency test with some blind aspects but they are not truly blind tests. Anyone can take a CTS exam, regardless of training or experience. Certification of examiners is better now because you have to notify AFTE if you fail a proficiency test and certification can be suspended immediately. Ideally, proficiency tests would reflect validation testing that meets ASCLD-LAB and ISO requirements.



What gives good firearm tool marks? The best tool marks from the range of guns in the middle range of quality. High quality guns are difficult to distinguish because of subclass characteristics and poorly made guns are less likely to leave reproducible marks. It is unknown how optical topography may address these gaps. The FBI is collecting test fires from all the firearms that they have. In addition, NIST is also adding low quality test shots, firing shots with no maintenance (dirty guns) and then cleaning them and test firing. All of this information will be represented in the NIST reference collection.

3.9 State of the Industry: Overview of Available Instrumentation Presentations were given at the meeting on the available instrumentation by for firearm and

tool mark identification. The following bullets provide links to those presentations for review:

• Gelsight TopMatch–GS 3D

• Alicona InfiniteFocusG5

• Leica (Multiple Models)

• IBISTrax HD3D.

3.10 Challenge of Subclass Characteristics During the working group meeting, John Murdoch presented an example of the extreme

difficulties that can arise due to subclass characteristics based on an examination of consecutively manufactured Ruger P95 and LC9 barrels from the Ruger plant in Prescott, AZ. Initial, blind examination of test fires from these barrels demonstrated that traditional comparison-microscope-based methods will not reliably individualize in certain cases, as there were no discernable differences among three of the barrels.

Subclass characteristics are manufactured tool marks that repeat virtually unchanged on a series of consecutively produced items that have been made by the same tool. The definition of subclass characteristics, as outlines in the AFTE glossary, is as follows: “Discernible surface features of a manufactured item that are more restrictive than class characteristics in that they:

1. Are produced incidental to manufacturer

2. Relate to a smaller group source (a subset of a class to which they belong) and

3. Can arise from a source that changes over time.”

When these tool marks are present on or near the working surfaces of tools, the tool marks they produce can be mistaken for individual working surface features. Therefore, subclass influences must be recognized to ensure that the tool marks they produce will not be used for identification purposes. It is important to note that although subclass tool marks maybe present near the working surface of the tool, they may either have no influence on the individuality of tool marks made by this working surface or edge because of their position or the manner in which the tool is used. Subclass characteristics are not always present on manufactured tool working surfaces.

Subclass features are those that carry on through the entire length of the rifled bore and may be characteristic of other members of the subclass, such as consecutively manufactured barrels. In order to evaluate subclass, the examiner should make a cast of the barrel bore and compare breech end to the muzzle end to find the tool marks that carry all the way through

https://rti.connectsolutions.com/p4f75f9fv8u/

https://rti.connectsolutions.com/p8ftm1bdbn3/

https://rti.connectsolutions.com/p5zwweqd48a/



We do not know how many sequentially manufactured guns can carry the same subclass. Sometimes, tool marks are present at the crown that can override the subclass characteristics. The best marks for identification are often on the heel of copper-jacketed bullets, in the land impressions. NIST does have some bullets with extensive subclass carryover and will begin working on them and evaluating if filters could be used to eliminate the subclass features. If a gun is fired with a lot of lead bullets, the rifled bore can lead up. Typically when a laboratory receives a gun, they fire it as they receive it. Then, the laboratory cleans the gun and fire progressive test shots after cleaning to see if there is any change in the quality or quantity of the firearm-produced individual tool marks.

In any case, comparison microscopes are very fast tools for sifting through data. Optical topography will not fill that need soon. The FBI feels that confocal microscopy could fill a need as a blind verification tool. Ultimately, optical topography could be a comparison tool when combined with an algorithmic approach.

Other issues include: interoperability, calibration, and interlaboratory comparison. Significant work is required to establish the “math” of optical topography, including error rates, filtering, and other calculations. There is insufficient information that has been established to know how to make an instrument fail and how would know when it fails. In other words, the examiner needs an objective approach to maintenance and calibration issues. For casings, correlation algorithms are still under development. Finally, work flow must be established.

3.11 Research Needs in the Application of Confocal Microscopy to Ballistic Imaging

3.11.1 Research to Date

Some instances of early firearm and tool mark research is shown in Table 2. Sources of more recent research13 and a description of what they reviewed is provided in Table 3.

Table 2. Early Firearm and Tool Mark Research

Type Event Earliest Topographical Analysis and Comparison of Bullet Stria

1958 – John E. Davis Oakland Police Department Crime Laboratory, An Introduction to Tool Marks, Firearms and the Striagraph.

Early Computer Based Bullet Comparisons:

1978 – Geoffrey Garner, Scanning Electron Microscope used to make surface profile measurements on 13 bullets fired from the three 0.38 special caliber revolvers and “signatures” were mathematically compared. 1988 to 1993 – Tsuneo Uchiyama, Bar code like lines were calculated from the image and can be counted by the computer 1998 – De Kinder, et al. Non-optical method that measured the surface topography of the striae in land impressions. The instrument used was an infrared laser surface topography scanner. The data was captured on a digital sensor. This non-optical approach is an early effort to eliminate problems of a surface reflection, surface curvature and illumination differences.

(continued)

13 Additional reference to research related to forensic firearms and tool mark identification can be found in AFTE Committee for the Advancement of the Science of Firearm and Tool mark Identification. (2011, June 14). AFTE Response to the 25 Questions related to firearms and tool mark examinations promulgated by the RDT&E IWG. Retrieved March 27, 2015, from www.AFTE.org: http://afte.org/downloads/RDT&E%20IWG%2025%20Questions%206.14.11%20- %20AFTE%20Response%20w%20cov%20let.pdf






Table 2. Early Firearm and Tool Mark Research (continued)

Type Event Faden, Chumbley et al., 2007

Striated tool marks produced by 44 sequentially produced screwdrivers to examine the profiles that were measured by a stylus profilometer. A mathematical algorithm was used in the comparisons between known match and known non-matched tool marks, including the variability of tip angles.

Bachrach, Koons, et al., 2010

Possibly the first study that used 3D surface profile measurement and comparison of striated tool marks produced by tools under different variable of angle, pressure, and materials. The acquisitions of the surface measurements was through the use of a “non- contact” method; confocal microscopy. The correlation to measure similarity of the profiles used open sourced statistical formulae.

Song, Vorburger, et al., 2012

Application of Cross Correlation Function (CCF) in NIST standard bullet comparison. Describes the production and validation of the Standard Reference Bullet (SRM) using surface “signature profiles” from fired bullets for the production of a virtual profile set. These profiles were used in the production of the SRM bullets. The method and formula used in the qualification of the bullets are the basis for a prototype objective mathematical comparison system for actual fired bullet tool marks.

Weller, Zheng, et al., 2012

Breech face marks on cartridge cases fired from 10 consecutively manufactured slides were measured by confocal microscopy. A total of 8010 comparisons using a 3D arial cross-correlation statistical algorithm which mathematically measures the similarity of know matching and known non-matching cartridge cases. There was no overlapping of scores from the matching and non-matching cases.

Petraco et al., 2012 The research report describes results of using confocal microscopy measurement and methods of mathematical based computer comparisons of striated and impressed tool mark surfaces. Statistical analyses of the methods were compared, and a web-based database was developed for interaction with other researchers.

Petraco et al., 2013 The authors report of the examination and comparison of 3D surface topography measurements from striated tool marks from screwdrivers and cartridge case firing pin aperture shear marks. The measurements were performed using confocal microscopy and the comparisons were performed by multivariate statistical methods; principle component analysis and support vector machine. Using these methods, an estimation of error was determined.

Chu, Thompson, et al., 2013

The concept of consecutive matching striae (CMS) numerical criteria was used as the basis of a matching model derived from tool mark striae from fired bullet land engravings. The bullets were fired from 10 consecutively rifled 9mm caliber barrels, and 15 unknown bullets were compared to the “knowns”. The surface features were measured by confocal microscopy, uninformative features were automatically masked, and the remaining signature detail was compared between all the bullet land combinations totaling almost 13,000 comparisons. Mathematically, the formula is comparing surface profiles between two bullets and measuring the degree of similarity in the “match” position.

3.11.2 Research Needs

During its discussions, the Forensic Optical Topography Working Group identified a number of research needs in which optical topography may be developed or relevant to broader issues, including the following:

1. The field needs an improved understanding of the incidence of class and subclass characteristics in tool mark impressions. The relationship of manufacturing method, history of the tool, materials, type of striae or impression, and other variables can influence the extent to which the examiner can individualize a tool mark impression. Optical topography could be used to produce a wider understanding of the incidence of these characteristics. The NIST database will be a valuable resource in this regard.



2. Examiners rely on assumptions about the power of CMS that should be subject to review with regard to statistical power, relevance within classes and subclasses, and information available in optical topographic data. The examiner should have access to information regarding the statistical power of various numbers of CMS to inform their decision-making.

3. Statistical and data science experts need to provide a framework for the analysis of optical topographic data to provide a basis for its application in the crime laboratory. This should include an understanding of filtering and data analysis algorithms, statistical characterization, error analysis, instrumental analysis, and related issues.

4. Proficiency testing for tool mark examination should be improved to reflect various levels of expertise, including the ability to account for class and subclass characteristics and other confounding variables. Recruitment, training, and proficiency testing standards should be updated to reflect current research concerning tool mark examination and human factors.

5. The Forensic Optical Topography Working Group should develop guidance for the field with respect to the use of optical topography as a confirmatory tool in the forensic laboratory. This guidance should include standards concerning when optical topography might be warranted, the use of optical topography to provide a statistical basis for an examination; the use of optical topography when traditional comparison microscopy cannot provide a match; the standards under which optical topography data is acceptable; guidelines for the procurement and deployment of systems; training; and protocols for the examination process. The working group will establish baseline considerations in this regard during a hands-on workshop at the FBI Laboratory.

3.11.3 Other Discussions

Other topics discussed at the Forensic Optical Topography Working Group Meeting included the following:

• Future work with firearm and tool mark identification will have some challenges (e.g., factory-produced [pre-fired] tool marks on cartridges).

• Consecutively manufactured guns are typical in police officer shootings and military because these organizations purchase large batches of firearms at the same time.

• What needs to be done to implement new methods? Validation procedures and ISO requirements exist, so research needs to be tailored to answer those requirements.

• An interesting type of data would be to tweak IBIS to see how powerful or discriminatory subclass characteristics can be in matching, similar to what is done with DNA.

• Base rates per municipality based on how manufacturers are shipping guns will be needed.

4. SUMMARY This report provides a summary of generalized comments and opinions from a diversified group

of researchers, examiners, commercial providers, and technology experts concerning the current status of forensic optical topography. The Forensic Optical Topography Working Group covered a diverse set of considerations, including the status of tool mark examination, potential impact of optical topography on the field, research needs, considerations in analysis, development of standards and reference data collections, proficiency testing, options in optical topographic technology, data interoperability, and related items. The working group will continue to develop protocols for the application of optical



topography as a confirmation tool to supplement current practice, with the understanding that much work needs to be done to establish optical topography as a primary instrument for tool mark analysis.

BIBLIOGRAPHY AFTE Committee for the Advancement of the Science of Firearm and Toolmark Identification. (2011,

June 14). AFTE Response to the 25 Questions related to firearms and toolmark examinations promulgated by the RDT&E IWG. Retrieved March 27, 2015, from www.AFTE.org: http://afte.org/downloads/RDT&E%20IWG%2025%20Questions%206.14.11%20-%20AFTE%20Response%20w%20cov%20let.pdf

Association of Firearm and Toolmark Examiners. (2011, Fall Volume 43 Number 4). Theory of Identification as it Relates to Toolmarks: Revised. AFTE Journal, p. 287.

Braga, A. (2011). Reconsidering the Ballistic Imaging of Crime Bullets in Gun Law Enforcement Operations. Forensic Science Policy & Management: An International Journal, 2:3, 105-117. http://www.tandfonline.com/doi/abs/10.1080/19409044.2011.613444#.VS-zFC6gs80

Braga, A., & Pierce, G. L. (2004, July 49:4). Linking Crime Guns: The Impact of Ballistics Imaging Technology on the Productivity of the Boston Police Department’s Ballistics Unit. Journal of Forensic Science. http://www.researchgate.net/publication/8394737_Linking_crime_guns_the_impact_of_ballisti cs_imaging_technology_on_the_productivity_of_the_Boston_Police_Department%27s_Ballistic s_Unit

Thompson, R. (2010). Firearm Identification In The Forensic Science Laborataory. Alexandria, VA: National District Attorneys Association.

X Zheng, J. S. (2014). Applications of surface metrology in firearm identification. Surface Topography: Metrology and Properties. http://iopscience.iop.org/2051-672X/2/1/014012/


http://afte.org/downloads/RDT%26E%20IWG%2025%20Questions%206.14.11%20-

http://afte.org/downloads/RDT%26E%20IWG%2025%20Questions%206.14.11%20-

http://www.tandfonline.com/doi/abs/10.1080/19409044.2011.613444#.VS-zFC6gs80

http://www.tandfonline.com/doi/abs/10.1080/19409044.2011.613444#.VS-zFC6gs80

http://www.researchgate.net/publication/8394737_Linking_crime_guns_the_impact_of_ballisti

http://www.researchgate.net/publication/8394737_Linking_crime_guns_the_impact_of_ballisti

http://iopscience.iop.org/2051-672X/2/1/014012/



Forensic Optical Topography Working Group Attendees

Gerald Laporte National Institute of Justice [email protected]

Gregory Dutton National Institute of Justice [email protected]

Danielle McLoud-Henning National Institute of Justice [email protected]

John Morgan RTI International [email protected]

Nicole McCleary RTI International [email protected]

Jeri Ropero-Miller RTI International [email protected]

Sue Ballou National Institute of Standards and Technology [email protected]

Robert Thompson National Institute of Standards and Technology [email protected]

Alan Zheng National Institute of Standards and Technology [email protected]

Brian Reneger National Institute of Standards and Technology [email protected]

Hans Soons National Institute of Standards and Technology [email protected]

Ted Vorburger National Institute of Standards and Technology [email protected]

James Yen National Institute of Standards and Technology [email protected]

John Butler National Institute of Standards and Technology [email protected]

John Lu National Institute of Standards and Technology [email protected]

John Song National Institute of Standards and Technology [email protected]

Derrick McClarin Federal Bureau of Investigation [email protected]

Eric Smith Federal Bureau of Investigation [email protected].

John Murdock Contra Costa County (California) Office of the Sheriff, Forensic Services Division [email protected]

Eric Collins Contra Costa County (California) Office of the Sheriff, Forensic Services Division [email protected]

Dan Gunnell Illinios State Police, Joliet Forensic Science Laboratory [email protected]

Ronald Nichols Bureau of Alcohol, Tobacco, Firearms and Explosives [email protected]

Martin Ols Bureau of Alcohol, Tobacco, Firearms and Explosives [email protected]

Chad Macziewski Iowa State [email protected]

Chris Saunders South Dakota State University [email protected]

Cedric Neuman South Dakota State University [email protected]

Martin Baiker Netherlands Forensic Institute Baiker, Martin [email protected]

Jan de Kinder Nationaal Instituut voor Criminalistiek en Criminologie, Belgium [email protected]

John Brad Etter Alicona [email protected]

Tom Calahan Zeiss [email protected]

Gretchen Falter Leica Microsystems [email protected]

Serge Lévesque FTI, Inc. (NIBIN/IBIS system) [email protected]

mailto:[email protected]






mailto:%[email protected]






















mailto:[email protected].

mailto:[email protected].




















157AFTE Journal -- Volume 46 Number 2 -- Spring 2014

Introduction

The field of firearm and tool mark identification has a long history that is grounded in the scientific method [1, 2]. In practice, forensic science is an applied science and may appear to lack the hallmarks of the scientific research on which it is based. Several academic researchers (non-firearm examiners) have alleged that even while firearm and tool mark examiners continue to perform research in their field, they are not doing so in a scientific manner. Several critiques about the pattern and impression disciplines within forensic science have been published following the release of the 2009 National Academy of Sciences (NAS) report on strengthening the forensic sciences. Many of these articles express the criticism that the empirical studies that have been published are: 1) not blind or double-blind; and 2) unreliable [3, 4, 5, 6, 7].

This article is not intended to provide a comprehensive literature review of how or why firearm and tool mark identification should be considered a science in light of its recent criticisms. It will, however, explain what blind studies are and their significance in minimizing sources of bias in research, the application of the scientific method to firearm and tool mark identification, and present the findings of the author’s research, which incorporated these concepts.

Declared vs. Blind Testing

Blind studies are often used in scientific fields in order to obtain information about a test, product, or a theory with as

Empirically Determined Frequency of Error in Cartridge Case Examinations Using a Declared Double-Blind Format

By: Angela Stroman, Criminalist, California Department of Justice, Bureau of Forensic Services, Redding Regional Crime LaboratoryKeywords: bias, breech face marks, cartridge case, blind testing, declared testing, double-blind, empirical study, error, manufacturing processes, null hypothesis, proficiency tests, scientific method, reliability, single-blind, Smith & Wesson 4006TSW pistol, subclass characteristics, validation study

ABSTRACTAlthough the discipline of firearm and tool mark identification has been accepted by courts since the early 20th Century, it has come under serious criticism from scientific and legal experts over the past several years. One of these criticisms is that the forensic science community, and the field of firearm and tool mark examination in particular, has not done enough to control bias in empirical research, such as participating in blind testing. Numerous examples of firearm and tool mark validation studies can be found in the relevant scientific literature. Many of these are single-blind studies involving the use of consecutively manufactured tools. A literature search of the AFTE Journal archives revealed only two studies that involved declared double-blind testing of firearm and tool mark examiners. The goal of this study was to empirically determine the frequency of error in firearms identification results produced by qualified examiners who were presented with realistic samples using a declared double-blind testing format.

little bias, conscious or subconscious, as possible. The basis of a blind study is that the people taking the test are unaware of certain information that might lead them to be biased toward a particular response [8].

The easiest way to describe a blind study is to provide an example: a manufacturer of peanut butter asks a subject to taste test two different brands of peanut butters and rate them. The peanut butters are presented to the tester in plain jars, labeled “A” or “B”, eliminating the possibility of bias due to brand preference.

However, there is not a clear consensus in the scientific literature as to what constitutes a “blind” test. Two noted critics of the manner in which testing is conducted in the forensic sciences, Saks and Koehler, acknowledge that their use of the phrase “blind examination procedures” can have a variety of meanings [9]. For instance, in the first part of a two-part survey on the feasibility of external blind DNA proficiency testing published in 2003, Peterson, et al. describe a test as being “declared” (or “open”) if the subjects know they are being tested, and a “blind” test as one in which the respondents are not aware they are being tested [10]. The 2009 NAS report on forensic science echoes these definitions [11]. The website of the National Institutes of Health (NIH), on the other hand, describes two types of blind clinical trials: 1) “single-blind” studies, in which only the patients are not told what drug they are being given; and 2) “double-blind” studies, in which the patient as well as nearly all of the hospital staff involved in administering the test do not know this information [12]. Yet, in both of these last two situations, the patient still knows they are participating in a test. Throughout Date Received: October 11, 2011

Peer Review Completed: December 20, 2011

AFTE Journal -- Volume 46 Number 2 -- Spring 2014

Stroman -- Empirically Determined Frequency of Error in Cartridge Case Examinations Using... 158

the remainder of this paper, the terms “single-blind” and “double-blind” will be used in the same general context as the NIH descriptions unless otherwise specified. To use a clinical trial as an example, a patient agrees to participate in a test of a new pharmaceutical drug knowing that they will either receive the actual drug or a placebo, but not which one they actually do receive. Because the test is blind in this respect, the doctor is able to evaluate the efficacy of the drug based on the progress of those patients in the group that received the actual drug versus the group which received the placebo (the control group), without concern for any cognitive bias that may have been introduced had the patients been aware of which type of dose they had been given. In a double-blind scenario, the doctors would also not know which patients were in which group until the end of the study. In this model, the fact that the patients know they are participating in a study does not diminish the blind aspect of the test, since the crucial factor is that both the patients and doctors do not know to which treatment group a particular patient is assigned.

Similar to this example, when a forensic examiner participates in a research study or a declared proficiency test, they are aware that they are being tested. The participants are not aware of what the true answers are or how the tests were created, and therefore a certain amount of confirmation bias has been removed. However, single-blind studies still allow the test giver to potentially interject bias either intentionally or unintentionally. For example, a test is made by a supervisor in the forensic lab and then given to an examiner. The examiner reports their results back to the supervisor, who then displays some unintentional visible or verbal queue to the examiner when the reported results were not what the supervisor anticipated; the examiner is now biased as to what the results might be should the examiner be allowed to revise them before officially submitting them. Alternatively, the examiner could present the findings to the supervisor, who then intentionally tells the examiner, “You may want to go back and look at Item 1,” indicating to the examiner that their initial answer was wrong and they need to re-evaluate.

Most proficiencies and research projects consist of single-blind testing. These tests can limit bias if the administrators do not work in the same office as the participants. In the case of externally-administered proficiency tests, such as those provided by Collaborative Testing Services (CTS), the test creators are specifically tasked with making and distributing the tests and never have any, or at least very limited, interaction with the participants.

In a single-blind test, the fact that the participant knows they are being tested can also play a significant role in their responses to the test. The examiner may feel more pressure

to perform accurately and get the correct answer, which may lead the examiner to treat the test in a different manner than real casework. For example, when an examiner knows they are taking a CTS proficiency test, they may try harder to get the right result because they know there is a lot of personal accountability riding on it. Furthermore, in addition to the possible stigma of getting the wrong answer, the examiner also knows that the laboratory will likely use the results of these tests as a standard for how well the examiner is performing their examinations, as often dictated under the laboratory’s quality assurance guidelines. This can be problematic, since CTS considers a “correct” answer to be one that is among the consensus of the responses they receive from participants and not necessarily what the true answer is based on how the test was prepared (which is known by CTS) or the quality of a particular test sample [13]. Therefore, if an examiner submits an answer (e.g. an inconclusive result) that does not meet the consensus, regardless of the reason, the laboratory can view the conflicting result as a quality issue and require the examiner to be retrained. Prosecutors and defense attorneys may also inquire about an examiner’s past CTS results as a matter of practice before or during the examiner’s testimony. If the examiner has any results that are different than the consensus, they may need to explain the inconsistency every time they testify. Potential repercussions such as these may lead examiners to treat declared proficiencies differently than normal casework.

While CTS proficiencies can be considered single-blind tests, they typically do not accurately reflect the full range of difficulty experienced in real-life firearm and tool mark casework, because they consist of actual fired bullets and cartridge cases or other non-firearm tool marks presented in pristine condition for the sake of reproducibility. Damaged test samples or those with borderline microscopic agreement are inherently very difficult to mass produce consistently using live ammunition. CTS readily admits the limitations of their tests, and has accordingly issued a statement discouraging the use of its tests to calculate error rates for the field [14]. However, in the absence of more controlled studies, proficiency test results such as those from CTS, when properly considered in the context of their limitations, currently appear to provide the best source of data from which to project meaningful false-positive error rates for the field for use in defending against court admissibility challenges [15]. Various commentators have debated the merits of using proficiency tests in this manner; these viewpoints have been summarized by Kaye, et al [16]. Still, since it is recognized that declared proficiency tests are neither designed nor very suitable for determining error rates for individual examiners or a profession as a whole, it has been proposed by at least one study that data be considered from several sources, one of



which being realistic, blind proficiency testing [17].

A review of articles published in the AFTE Journal indicates that most of the past research and validation studies performed in regards to firearm and tool mark identification can be considered to be, at the least, single-blind [18, 19, 20, 21]. Despite the potential pitfalls for bias described above, single-blind studies can still give valid scientific results if conducted with those pitfalls in mind. However, an ideal approach to addressing concerns of bias in validation studies would appear to be the double-blind model. True double-blind (or simply “blind”) studies are those in which the test administrators do not know which participants received which samples and the participants are unaware they are being tested. Even though this type of testing has the greatest potential for minimizing sources of bias, it is difficult and costly to implement. To perform such a test in a forensic laboratory would involve submitting the test samples to the laboratory as if they were routine evidence samples, to include misrepresenting who was actually submitting the test so no one in the lab administration, let alone the examiner, would be aware of the test. In this scenario, the laboratory administrators would have contracted with the external test provider prior to the test, but would not know anything about the test or even be aware of when it was being worked on in the laboratory. This type of testing could be implemented in a variety of ways, with either just the target examiner or the entire laboratory being kept unaware of the test. One major advantage that has been claimed of true double-blind testing is that it can be used to test “the whole system”: the processes a laboratory uses from the time it receives the “evidence” until the time the results are reported out. Obviously, the feasibility of implementing a truly double-blind system of proficiency testing on any significant scale is low, as it would need to involve extremely careful test preparation and the participation of local law enforcement agencies to successfully “deceive” the target laboratory and examiner without raising any suspicions that it might be a test. Due to the intricacies of administering such a test, the cost per test would be prohibitively expensive for any but the most well-funded laboratories. For instance, it has been conservatively estimated that the cost to implement a large-scale (150 laboratories), national double-blind proficiency testing program for forensic DNA analysis would run approximately $1,400 to $10,000 per test, depending on the test provider and the variables of the testing model used [22].

A more practical compromise to true double-blind testing is to incorporate as many of the elements of double-blind (i.e. “blind”) testing into a “declared” test, one in which the participants know they are participants. A search of the AFTE Journal archives revealed two studies that deal with declared

double-blind testing of individuals: a 2003 validation study performed by Bunch and Murphy, and a 2009 validation study by Giroux [23, 24].

The design of the Bunch and Murphy study, which measured examiners’ ability to identify cartridge cases from consecutively manufactured Glock slides, incorporated four principles: 1) return of the test was mandatory; 2) the tests could not be traced back to individual examiners, so the participants would not “try harder” on their test samples for fear of reprisals should they commit any errors; 3) the tests were all different, eliminating the possibility that the tested examiners could reveal or obtain the correct answers by discussing the test amongst themselves; and 4) the test was called “double-blind” by the authors in the respect that the participants were unaware of the correct answers, and that the test administrators (Bunch and Murphy) did not know who received which test specimens. As a further bit of caution to reduce bias, the test answers were reportedly kept under lock and key until the test was completed. Unlike a true double-blind test, but similar to the majority of validation studies within the field of firearms identification, the participants in the study were aware of the fact that they were being tested and therefore any potential bias created by this knowledge was still present [25].

In Giroux’s study, the ability of examiners to identify tool marks made by consecutively manufactured screwdrivers was investigated. Giroux followed the experimental design set forth by Bunch and Murphy insofar as: 1) the return of the test was mandatory; 2) the answers to the test were not traceable back to an individual examiner; 3) all of the tests were different from one another and therefore the participants could not collaborate; and 4) the test administrator had no knowledge of which participants had which test kit [26].

Although both Bunch and Murphy’s study and Giroux’s study are referred to as “double-blind” (only Bunch and Murphy actually used this term; Giroux called his a “blind” study but acknowledges that he used the same experimental design as Bunch and Murphy), using the more generally accepted meanings of the terms “declared” and “blind” as described above, they could be more accurately characterized as “declared tests with two or more blind elements.” While this may at first seem to be merely an exercise in semantics, the distinction is an important one to make when describing a test so as to be unambiguous regarding whether or not the participants had knowledge of the test. This description also fits the research project that is the subject of this article, since it follows a similar design.

One major difference between the validation studies cited



above and proficiency tests like those provided by CTS is the fact that in the above studies none of the validation study test kits were exactly the same. This fact alone prevents the participants from being potentially biased by another nearby examiner also participating in the test. While the replicated test kits provided in CTS-like proficiencies and studies allow the researcher to assess variability in examiners’ abilities, it does potentially allow for the inadvertent sharing of answers or outright collusion between examiners participating in the test. For example, if three examiners in a laboratory or laboratory system are taking a CTS test and Examiner 1 and Examiner 2 both identified Item 1 as having been fired in Gun 2, Examiner 3 can most likely conclude that Item 1 in their test kit will have been fired in Gun 2, if he or she somehow learns (either intentionally or unintentionally) of the other examiners’ results.

Yet another factor that can be used in an experimental design to combat cognitive bias in declared firearm and tool mark identification validation and proficiency tests is the use of open test sets. An open set of test unknowns is one which may or may not contain a sample that matches one or more of the known samples provided to the participants. This is in contrast to a closed set, in which each of the unknown samples match one of the exemplars provided. If the participants know they are dealing with a closed set of unknowns, they will likely perform better on the test than if it were an open set because they may be able to use a process of elimination to infer at least a couple of the answers if they were able to identify the rest of the unknowns. Even if a test is designed using a closed set of unknowns, the test administrators can control bias to a significant degree by ensuring that no indication is given to the participants (via the test instructions) that they may have been given a closed set of unknowns [27]. This is what was done in the current study.

The Scientific Method and Hypothesis Testing

A common criticism of firearm and tool mark identification is that it is a subjective analysis that is not based in science or on the use of the scientific method [28, 29, 30, 31]. However, an analysis of the scientific method and its application in the field of forensic firearm and tool mark examination shows that the field is, in fact, well grounded in its use. Forensic firearm and tool mark examiners use the scientific method on a daily basis whether or not their educational background is in a physical science. Every case they work deals with each of the basic steps.

Any scientific inquiry usually starts with a basic question stemming from an observation. Questions asked in firearm identification typically include: “Did the submitted firearm fire

the questioned bullet?” or “Were these two cartridge cases fired in the same (unknown) firearm?” The question then forms the basis of a hypothesis statement that is capable of being tested. This is an important point, because if a hypothesis cannot be tested (e.g., “Does God exist?”), then it is not something for which science can be used to find an answer. “Tested,” in this sense, specifically refers to the capability of the hypothesis to be proven false through direct empirical observation (the null hypothesis). In science, certainty is an elusive goal. A hypothesis cannot be conclusively “proven” through empirical testing; it can only be disproven. If a hypothesis cannot be disproven, even over the course of years of repeated testing, the only implication is that its premise may be true, whereas once a hypothesis is disproven, it is rejected from further consideration. For this reason, a scientific hypothesis is usually stated in the negative form [32]. There is also another good reason for doing so, however. When forensic scientists deliberately state their hypotheses in the negative, they are taking precautions to minimize confirmation bias. In many instances, if a police agency submits a firearm and a fired bullet to the laboratory for an identification examination, the implied answer to the question they are asking of the examiner, “Was the submitted firearm used to fire the questioned bullet?,” is that the submitter believes it did, otherwise the examination would not have been requested in the first place. The forensic firearm examiner can deliberately take a step back from potential sources of bias such as this by testing the negative hypothesis: “The firearm was not used to fire the questioned bullet.” When approaching a case with this hypothesis in mind, the examiner proceeds with their examination in the normal way, but if this negative hypothesis cannot be disproven, then either the firearm was not used to fire the bullet or there is not enough information to disprove it, leading to an inconclusive result.

When a hypothesis is rejected, a new hypothesis is selected based on the results of the empirical testing. The process is then repeated to test the new hypothesis. The proper use of inductive and deductive logic is a critical component of this process. Grzybowski and Murdock have previously illustrated the application of inductive and deductive reasoning to the field of firearm and tool mark examination, as well as delineating the field’s underlying scientific premises based on this reasoning [33]. When a hypothesis cannot be disproven and test after test continues to confirm it, then it can be considered a theory. However, as was true of the initial hypothesis testing, no amount of confirmatory test results can permanently or irrefutably establish the validity of a theory. For instance, throughout years of testing, the AFTE Theory of Identification has yet to be proven false. Yet, continued research and validation testing of the Theory, using the most current technology and philosophies in an effort to falsify



the hypothesis that firearms and tool marks can be reliably identified on the basis of “sufficient agreement” of their microscopic characteristics, is needed in order to further develop and refine it. Limited testing of hypotheses, sometimes to the point of a researcher trying to prove a preconceived result, is one area that has been criticized by Mnookin, et al. in regards to forensic science; however, these authors also temper their criticism with some sound guidelines for forensic researchers:

Claims of knowledge should be taken as provisional and subject to revision in the face of new information. Dogma should be resisted. Research is not one thing, or one study, or once done, never reexamined. Research is an ongoing, incremental process. Research problems should be approached with an open mind. While it is certainly appropriate to have a hypothesis, or preliminary expectation, about what any given research study will show, investigators should follow the data whether or not it supports their original hypothesis, and whether or not it legitimates current practices. Research projects should be designed according to the norms of relevant academic fields. They should not be designed defensively, to produce, or to increase the chances of producing, a particular outcome [34].

Case Influencing This Research

This research project was based on an actual officer-involved shooting case that was assigned to one of the author’s colleagues. The firearms involved were two Smith & Wesson model 4006TSW, .40 S&W caliber semiautomatic pistols. As is common with these types of cases, one of the questions presented to the examiner was, “Which shots were fired from which firearm?”

In a firearm, a number of different parts are interacting with the cartridge case during cycling and firing. Each of these parts can be considered a separate tool, the working surface of which has to be evaluated individually. These different tools can leave either impressed or striated tool marks of varying levels of reproducibility, which is what makes cartridge case identification a unique challenge.

During the examination, the examiner noticed several details about the firearms that were of interest. First, the firing pins in these firearms are not fixed and can rotate during the firing process. If rotation occurs, it can cause the firing pin to present a different portion of itself to the surface of the primer during the creation of the firing pin drag mark. This different surface will create a different tool mark on the primer; therefore, the dragging of the firing pin may not create reproducible marks

from one cartridge case to the next. The free-floating firing pin also made identification of the firing pin impressions more difficult because the impressions may not have been in the same orientation relative to the extractor and ejector marks on the fired cartridge cases from one shot to the next, creating the illusion of differences. In addition, the firing pins had very few surface defects on them and the resultant markings made by the two firing pins had some similarities.

Secondly, the manufacturing marks on the breech faces of the firearms had a linear parallel configuration, apparently from a broaching process, which indicated that subclass influences needed to be considered. This was compounded by the fact that the breech face marks did not impress well into the submitted evidence and ammunition (Remington Golden SaberTM).

Thirdly, similarities in the ejector faces of both firearms were noted, indicating that these markings could again present a possible subclass influence.

The examiner who worked this case was ultimately able to use an ejector cut-out shear mark on the cartridge cases, along with the firing pin impressions, to make the final identification. The examiner indicated that this examination was more difficult than routine casework even though it was a closed set (or “known universe”) in that the cartridge cases were known to have been fired in one of the two guns from the officers. The author felt this case example would be suitable for a declared blind study due to the fact that the surfaces were not marking well and the possible presence of subclass in the breech face marks.

In addition, the fact that the Smith & Wesson model 4006TSW is a relatively common firearm increases the practical value of the test. Not only do California Highway Patrol (CHP) officers carry the Smith & Wesson model 4006TSW, increasing the possibility that a California firearms examiner will have a case involving this type of gun, but it is also a model that has been available since 2000 [35].

This case prompted the question of how the presence of subclass characteristics may affect an examination. Two questions that arose from this case were: 1) In a declared blind study of cartridge case identification using samples (fired cartridge cases) in which subclass is possibly present, would the error rate of the participants be as low as currently estimated [36] or would the potential subclass influence cause an increase in the error rate?; and 2) Would the presence of potential subclass features in the test samples increase the chance of making an erroneous identification?

Taking these questions into consideration, the hypothesis for



this study became: When firearms examiners are presented with a no-gun examination, where subclass features may be present, and marks commonly used for identification transfer minimally to test-fired cartridge cases, they will not be able to determine that a particular cartridge case was fired in a particular firearm.

Current Research

Three Smith & Wesson model 4006TSW firearms were obtained from the local CHP office. These firearms were from a pool of guns that were maintained as temporary replacement firearms for patrol officers in the event that their assigned firearm is sent for repairs or removed from service after an officer-involved shooting. Upon receipt, all of the working surfaces of the firearms that would come into contact with the cartridge cases, except for the chamber areas, were observed and documented using photography.

Breech faces:

On one of the firearms, the breech face had some very light, parallel, linear markings running vertically across the machined breech face. Based on their relatively uniform, uninterrupted appearance, some of these markings could potentially be subclass characteristics. The machined breech faces of the other two firearms also had vertical marks with an overall parallel linear configuration running from top to bottom; however, at the 12 o’clock position above the firing pin aperture the markings were intersected by other linear marks, forming an angular pattern. This angular pattern indicates that subclass should be less of an issue here because the positioning and relative spatial relationships of the intersecting markings appeared to be random. In addition, the linear marks on these breech faces also displayed an apparent random pattern of start and stop locations across the surface, which further minimizes the presence of subclass influence in these marks. Figures 1 through 3 show silicone casts of the breech face markings of the firearms used in this study. The general reproducibility of the firearms used in this study can be seen in Figures 10 through 12, which show comparisons of the breech face casts to representative test-fired cartridge cases from each of the guns used. Based on this assessment of the breech faces, the presence of minimal subclass influences does not appear to be a significant problem in the potential identification of cartridge cases marked by them, at least within the limited population of these three firearms.

Firing pins:

The firing pins were removed from each of the firearms and inspected microscopically; see Figures 4 through 6. No

Figure 1: Gun 1, breech face cast, 20X [Marks appear horizontal in photo, but marks run

vertically on original working surface of pistol]

Figure 2: Gun 2, breech face cast, 20X [same orientation as Fig. 1]

Figure 3: Gun 3, breech face cast, 20X [same orientation as Figs. 1 & 2]



Figure 4: Tip of firing pin from Gun 1, showing multiple apparent random defects, 60X

Figure 5: Tip of firing pin from Gun 2, showing defects, 60X

Figure 6: Tip of firing pin from Gun 3, showing defects, 60X

Figure 7: Apparent subclass carryover on ejector working surfaces; Gun 1 (left) vs. Gun 2 (right), 30X

Figure 8: Working surfaces of ejectors, showing apparent subclass carryover;

Gun 3 (left) vs. Gun 1 (right), 30X

Figure 9: Working surfaces of ejectors, showing apparent subclass carryover;

Gun 3 (left) vs. Gun 2 (right), 30X



circular lathe markings were observed on the firing pins to indicate that they were made using a lathe. Each of the firing pins had numerous surface defects in an apparent random pattern, which would indicate they were specific to these firing pins. These markings did not appear to be the same from firing pin to firing pin. However, because the firing pins have similar looking surface defects and are free to rotate during the firing process, an examiner could be confused by the random agreement found between impressions made by different firing pins.

Extractors:

The extractors for each firearm were examined microscopically. The surface of the extractor that would come into contact with the cartridge case could not be viewed directly due to its close proximity opposite the breech face, therefore the working surfaces of the extractors were not critically evaluated for subclass influence. In retrospect, casting would have provided an easy means of examining the working surfaces without having to remove the extractors for examination, or they could have been removed and examined. However, subclass characteristics have never been reported on the working surfaces (edges) of extractors.

Ejectors:

The faces of the ejectors were examined microscopically. Coarse striations across the ejector faces were observed. The ejectors from each firearm were compared to one another using a comparison microscope and some agreement of the coarse striae was found; see Figures 7 through 9 for these comparisons. This indicated that there were subclass features present on these ejectors that may transfer to the cartridge cases during firing. This potential could present a problem to examiners if these features do in fact transfer to the cartridge cases and an examiner were to rely upon these marks for identification without evaluating them for subclass.

Chamber marks:

During the initial evaluation of the firearms, the chamber marks were not examined and the chambers were not evaluated for subclass influence. After the results from the test kits were received it was found that a number of examiners used the chamber marks to aid in their identification. Since the working surfaces of the chambers had not been examined prior to distributing the test and by this time the original three firearms had already been returned to the CHP and were not available for reexamination, yet a fourth Smith & Wesson model 4006TSW firearm was obtained from a retired CHP officer in order to examine these surfaces. A silicone cast of

Figure 10: Gun 1, breech face cast (left) vs. breech face marks on fired cartridge case (right), 30X





the chamber area was made and the cast was examined using a stereomicroscope at various magnifications. The chamber marks appeared to be very small chatter-type marks distributed longitudinally around the entire circumference of the chamber surface. In this particular firearm, there was an area of the chamber that appeared to have a raised metal defect that would most likely transfer an impressed and/or striated mark to a cartridge case. The chatter-type marks appeared to be random and were oriented parallel to the direction that the cartridge case would enter and exit the chamber. Despite being parallel to the direction of the cartridge case insertion and extraction, their apparent random nature should make them unique characteristics for purposes of identification. Therefore, if the three original pistols had a similarly random array of chamber tool marks as the fourth firearm appeared to have, they too should have had no subclass influence. The author feels this is a reasonable assumption considering these firearms were batch produced using the same tools and machining processes.

Information on How These Working Surfaces Are Manufactured

A literature search was conducted for information on how the working surfaces of the Smith & Wesson firearms that would potentially be used for this study were tooled. In 2010, Lightstone provided a good summary of how some of these surfaces were made [37]. According to the information Lightstone received in regards to Smith & Wesson Sigma series firearms at the time of her Smith & Wesson factory tour in June 2009, the breech faces are cut in a single pass of a horizontal broach. After broaching, the firing pin aperture, extractor hole, and the firing pin assembly hole are drilled into the breech face using a gun drill. This drilling process leaves burrs on the surfaces which are later hand-sanded off. The slides are tumbled in a ceramic medium that has potential to contact the breech face if the media is small enough. The slides are then sand-blasted to smooth the outside surfaces, but the abrasive material often contacts the breech face surfaces as well. The final finishing step is glass bead blasting the slides. This bead blasting often contacts the breech faces of the slides even though it is not directly aimed at them.

Lightstone’s article also indicated that Smith & Wesson was going through a transition from the above described process to a newer, more automated process using fewer computer numeric control (CNC) machines. The main difference in this new process is that the breech faces are cut using two separate broaches. The first broach flattens out the rough areas and is followed by a step-broach which finishes the surface [38].

According to weapons officer Scott Fredrick of the CHP’s Northern Division office, they received their first lot of Smith

and Wesson model 4006TSW firearms in December 2006 and their order was completed in October 2007. The fact that Smith & Wesson did not switch over to its new CNC machining method until 2009 would indicate that the CHP firearms used for this study were produced using the older method of manufacturing.

In May 2011, the author was given the opportunity to tour the Smith & Wesson factory and ask questions of Product Engineer Jason Dubois regarding the machining of some of the parts used in model 4006TSW firearms. He indicated that the model 4006TSW, which is similar to the civilian 3rd Generation models, has been phased out of production but is still available to be purchased in a batch quantity if an agency were to request it. While some of the manufacturing techniques from the time the model 4006TSW’s were produced have changed, the basic processes for their current firearms are as follows:

For the newer models, the M&P series and the SD series, the slides start as a piece of bar stock of the desired metal. The bar stock is placed into a CNC machine which will perform five separate operations on the bar stock to bring it to its final shape and dimensions. The breech face of the slide is broached inside the CNC machine with a rectangular-shaped step broach. Once the slides are taken off of the CNC machine, they are wet blasted with a gritty material to de-burr any of the rough surfaces created during cutting. The slides are then sent for heat treating. If the slide is to be coated with Melonite®, they are sent to an outside vendor who performs this operation and then returns them to Smith & Wesson. The switch to a single CNC machine, versus the old production method which required more than one machine for all of the cuts, requires the use of fewer employees and also allows for less error in moving the parts from machine to machine. The Sigma series slides were produced in the older method described in Lightstone’s paper, which utilized a manual horizontal broach. The Sigma series pistols have since been discontinued.

The tools used on the CNC machines have a designated “tool life” that is programmed into the machines and is based on minutes of cutting time. Once a tool has reached its maximum amount of cutting time, the machine automatically switches out the tool and flags it as being in need of sharpening. If the tool is designed for precision cutting, the tool life is much shorter between sharpenings. For rough cutting tools, the tool life can be extended for a greater amount of time. Whether or not the tool will be re-sharpened or just replaced depends on the cost, design, and age of the tool. All of the tools have an acceptable limit to which they can be resurfaced, and once they have reached a point beyond this tolerance they are replaced.



During production of the slides, a certain percentage of them will be removed from the production line and their dimensions measured using a coordinate measuring machine (CMM). This is done to determine if the slides are being cut to the appropriate dimensions and shape by a particular CNC machine. This process is done on the manufacturing floor in order to allow any mistakes to be caught early in the production process.

For all of the pistols made by Smith & Wesson, the chamber surface is honed using a ball brush. A ball brush is a wire brush that has small balls of grit attached to the bristles. The grit material can either be metal or nylon depending on the surface that is being honed. The ball brush is attached to a machine that rotates the brush at high speeds, similar to a drill. The brush is then inserted into the chamber area and the grit smooths out the inside of the chambers.

The firing pins used in Smith & Wesson’s firearms are manufactured in a number of ways, depending on whether they are made in-house or are purchased. If they are made in-house at Smith & Wesson, they are turned on a screw machine and then placed into a ceramic medium where they are tumbled. If they are purchased from an outside vendor they are generally a metal injection molded (MIMed) part that is then tumbled or coated.

The extractors used by Smith & Wesson are all purchased from outside vendors, except for the 1911 series. The purchased extractors are MIMed and have no finishing performed by Smith & Wesson; any finishing processes on these extractors would have been performed by the outside supplier before Smith and Wesson received them.

As with the firing pins, the ejectors are manufactured in a number of ways: MIMed, stamped, or fine blanked. The MIMed ejectors are only used in the 1911 series and are purchased by Smith & Wesson. The stamped ejectors are also purchased from an outside vendor. The fine blanking is done by Smith & Wesson in their Maine factory. These fine blanked ejectors are the ones used in the model 4006TSW pistols. The ejectors used in Sigma series pistols were fine blanked by a vendor. Six of the fine blanked ejectors were provided as production samples by Smith & Wesson in order to evaluate their surfaces. On close examination of these samples, it was evident that there were subclass characteristics present on the ejector face. The presence of similar subclass on the ejectors of the Smith & Wesson model 5906 was noted by Evan Thompson in 2002 [39]. Thompson contacted Smith & Wesson and was informed that these ejectors were stamped from bar stock and then the faces were ground. Not only did the casts Thompson made of the ejectors display very similar

markings as the marks seen on the ejector samples provided to the author by Smith & Wesson, but they appeared very similar to the ejectors seen in the firearms from the real case example and the three firearms used in this study as well. If the ejector faces were ground, as Thompson was led to believe, subclass carry-over would be very unlikely based on the rapidly changing surface of most grinding wheels. However, firearm examiners are aware that depending on the manufacture and material of the grinding wheel, the hardness of the material being ground, the wear rate of the grinding wheel, and the fact that the face of the ejectors are very small surfaces, it is possible that subclass characteristics could carry over from one ejector to the next even if a grinding wheel was used [40]. The fact that these ejectors are reportedly fine blanked makes subclass more of a possibility due to the fact that the same die will be used to make each of the parts and the dies will change much less rapidly over time than a grinding wheel. Dubois did not know if any further finishing was performed on these parts but indicated that due to the precision machining techniques now used to make them, very little hand fitting is required on the final products.

Given the manufacturing techniques described in Lightstone’s paper and those observed by the author during her own Smith & Wesson factory tour in 2011, the potential for subclass characteristics is present on the breech faces and ejectors of model 4006TSW firearms and any other series of Smith & Wesson firearms that are produced using the same manufacturing techniques.

Materials and Methods

Independence brand, .40 S&W, 180 grain, FMJ ammunition was purchased in as few different lots as possible in order to limit the amount of variables in the research study (according to several internet sources, this brand of ammunition is made at the ATK-operated Lake City Army Ammunition Plant in Independence, Missouri, using Federal, CCI, and Speer components). A local gun store was able to provide enough ammunition for the study in two lots: A22R39 and A23R31. Prior to creating the test kits, the head of one cartridge from each box of each lot was examined and photographed using a Leica FSC comparison microscope at various magnifications to determine whether or not any manufacturing marks could be potentially mistaken for firearm-produced marks. While numerous small, irregular manufacturing marks (produced either during or incidental to the manufacturing process) were apparent on the case heads, none of these appeared to be in a quantity or configuration sufficient to interfere with comparison. Just to be absolutely certain, a Smith & Wesson model 4006TSW firearm was obtained and 10 of the Independence brand cartridges were fired through it and



the cases were collected. These cases were examined using a comparison microscope and the manufacturing marks on the cartridge cases were found to either be obliterated during the firing process due to obturation and impressing of the breech face and firing pin, or were easily distinguished from the breech face marks on the primer. Therefore, these pre-existing marks would not likely be confused by an examiner as having been produced by the firearm during firing.

The three Smith & Wesson model 4006TSW firearms obtained from the CHP for this study were designated “Gun 1,” “Gun 2,” and “Gun 3,” respectively, and the serial number for each firearm and its designation were documented. The serial numbers for the three firearms were within 300 numbers of one another and two were within 20 numbers of each other. While these numbers being close in sequence does not necessarily mean that the firearms were actually produced sequentially in this same order, it does indicate that they were made within a relatively short time frame of one another.

Three magazines were provided by the CHP with the firearms. In order to speed up the test firing process and to reduce the amount of time necessary to stop and reload magazines, all three magazines were used during the test firing of each firearm. Because the same magazines were used throughout the test firing, the magazine lip marks could not be used by an examiner during the examination process. The examiners who participated in the study were instructed to not use the magazine lip marks in their comparisons.

Known Cartridge Cases:

Each firearm was used to fire 124 cartridges. Ammunition lot number A23R31 was used for Gun 1, while lot number A22R39 was used for the test-fired components fired in both Gun 2 and Gun 3. Each firearm was used to fire 93 cartridge cases to be used as known specimens. These cartridge cases were collected and engraved with the gun number inside the mouth. The remaining 31 cartridge cases from each gun that were set aside as unknown, or questioned, were not engraved at this time but were placed into a box labeled with the firearm number in which they were fired. These unknowns were labeled with a random identifier by another examiner at a later time in order to keep the study blind in this respect.

Three known cartridge cases from each firearm were placed into each of 31 test kits that were created. Figures 13 through 18 show representative comparisons of the breech face marks and firing pin impressions from each of the three firearms on cartridge cases provided in the test kits.

Figure 13: Gun 1, reproducibility of impressed striae in breech face impressions on two

representative test-fired cartridge cases, 10X

Figure 14: Gun 1, reproducibility of impressed marks in firing pin impressions on two






Unknown Cartridge Cases:

The three boxes labeled “Gun 1,” “Gun 2,” and “Gun 3,” containing the unknown cartridge cases, were given to another examiner in the laboratory who then engraved each cartridge case with a randomly generated number. The numbers used were generated from a random number generator program available on the internet. The second examiner documented which randomly numbered cartridge case was associated with each firearm prior to mixing all of the unknown cartridge cases from all three firearms in a cardboard box.

A third examiner randomly selected three cartridge cases from the box for each test kit. The numbers selected for each kit were recorded, along with the kit number that they were assigned to. With each stage of the kit-making process performed by different examiners, no one examiner was aware of how a particular kit was produced.

The examiner who performed the original test firings then randomly selected the assembled kits for mailing out to the external examiners (one kit per examiner) who agreed to participate in the research project. In total, 30 kits were sent out to a total of 14 different crime laboratories. Multiple kits were sent to a few of the participating laboratories in order for more than one examiner in those laboratories to participate in the study. Agencies from eight different states in the U.S. and one from Canada participated in this study.

A standardized answer sheet was provided with each kit. The answer sheet asked for very basic demographic information from the participating examiners, including: number of years in the field, approximate case load per year, if the examiner’s laboratory was accredited, and if the examiner was certified. The participating examiners were not only from very different geographic locations but also had very different experience levels. The number of years in the field ranged from 2.5 to 30 years with the average number of years in the field being 13. The number of examinations worked per year for the participants ranged from 20 to 500 with the average number being 150 examinations. There were two outliers in this demographic information, including one participant who reported that they no longer perform firearm casework and another who reported they performed over 1000 examinations per year if each bullet or cartridge case examined was considered to be a single examination. All of the participants in the study worked for accredited laboratories. Eight of the participants were certified through either AFTE, the American Board of Criminalistics (ABC), or both. One of the examiners stated they were “certified” by “FirearmsID” (although it is unknown exactly what this examiner was referring to with this response, the website www.FirearmsID.com offers written









and practical self-tests in firearms identification to interested users, but this is not an independent certification program).

Throughout the entire preparation of the test kits, no individual preparer had access to the answer key for any of the test kits. For example, the person who made up the known cartridge cases did not know what the unknown cartridge case numbers would be for each firearm. The person who designated the unknown cartridge case numbers did not know which test kit these unknowns would be associated with or from which of the known firearms they came. The final kit preparer did not know which firearm any of the unknown samples came from nor did they know the serial number of the firearm for the knowns or the unknowns. Lastly, the person who sent out the test kits only knew from which serial number firearm the knowns came and was not privy to any of the answers for the unknowns.

Similar to the Bunch and Murphy and Giroux studies, an element of anonymity was provided to the participants in this study. A list of the kits and the agencies to which they were mailed was maintained, but only for tracking purposes and it was not retained after the kits were returned. This list was kept in the possession of a member of the clerical staff at the laboratory and was not revealed to the test creators. This was not a study about how an individual examiner or any particular agency would perform; it was intended to survey a cross-section of the firearms examiner community. Because the test participants were well aware they were participating in a test, and the aforementioned precautions were taken in order to blind the test administrators to any bias their knowledge of the test results might produce, this study can be considered to be a declared test with several blind elements.

The examiners were asked to examine the items as they would work a normal case, including second calls or technical reviews, if authorized by their laboratory procedures. The questions the examiners were asked to answer were: 1) “Were any of the unknown expended cartridge cases discharged by the same firearm as the known expended cartridge cases from Gun 1?”; 2) “Were any of the unknown expended cartridge cases discharged by the same firearm as the known expended cartridge cases from Gun 2?”; and 3) “Were any of the unknown expended cartridge cases discharged by the same firearm as the known expended cartridge cases from Gun 3?”. Although the test kits were designed so that each of the unknown cartridge cases had a corresponding known specimen in the same kit (a closed set), the questions were worded in a manner that would not imply to the participants that they were dealing with a closed set. The examiners were asked to answer these questions using one of three conclusions as defined by the Range of Conclusions in the AFTE Glossary: 1) Identification,

2) Inconclusive1, or 3) Elimination [41]. In addition to these questions, the participants were asked what areas of the cartridge cases they relied upon for their conclusions. Without directing the examiners as to what marks to use, the answer sheet provided the following choices: “breech face”, “firing pin impression”, “extractor”, “ejector”, “chambering marks”, and “other”. As previously mentioned, the participants were instructed not to use magazine lip marks since the same three magazines were used to fire all of the knowns and unknowns.

Results

Of the 30 test kits that were sent out to the participating agencies, results from 25 of the kits were returned. Since three unknowns were provided in each kit, this equals a total of 75 results. Of these, 74 of the responses correctly identified the firearm that had fired the unknown cartridge cases. One result was reported as inconclusive. This examiner did not indicate what led them to their inconclusive result (See addendum at the end of this paper).

Of the 74 correct responses, breech face marks (BFM) were indicated to have been used, at least in part, to reach the reported conclusions in all of them. Twenty of these responses stated BFMs were used exclusively, while an additional three responses implied that only BFMs were used through comments such as, “Used BFM for all identifications however, did not use gross contours of BFM (only used those to phase) – used smaller/individual w/in cross contours to ID,” and “Gun 2 had excellent fine detail in breech face impressions.” One of these examiners indicated that they were concerned that the parallel breech face marks may have been subclass so they looked for “endings, chatter, finer marks within, and nicks that appeared to be from other sources.” These comments indicate that although parallel breech face marks have the potential for subclass features to be present, at least some firearm examiners (the ones who commented) are aware of this and know how to evaluate the surfaces properly to limit the effect that subclass might have on their examinations. It should be noted that the examiner who reported the inconclusive result was the sole participant who did not indicate they used the breech face marks.

The second most relied-upon area for the participating examiners was the firing pin impression. The examiners indicated in 46 of the examinations that they used the firing pin impressions in part for their identifications. One examiner

1 For the sake of simplicity, the three subcategories of inconclusive results and the conclusion of “Unsuitable”, as described in the AFTE Range of Conclusions, were not offered as possible response choices; however, it is acknowledged that in actual casework the nature of any inconclusive result should be further described.



indicated that the firing pin impressions in the test fire sets “did not demonstrate sufficient repeatability to use that tool mark for identity or elimination.” No other comments were made in regards to the lack of reproducibility of the firing pin impressions, but at least two examiners indicated that they cast the case heads to better visualize the firing pin impressions. It is possible that the nickel-plated primers of these cartridge cases prevented good visualization of the firing pin impressions. The fact that the firing pins in the firearms used to prepare the tests had a “floating,” or non-fixed, design could have made the resulting impressions difficult to use for identification due to their lack of consistent orientation in reference to the firing pin drag marks or extractor and ejector locations.

In five responses, the participating examiners indicated they also used the ejector mark in their examinations, in combination with either the breech face marks only (three responses) or with the breech face marks and extractor marks (two responses). Some of the examiners indicated that they would not rely on the extractor or ejector marks for identification without having the gun to examine because they could not ascertain how these parts were made or evaluate them for the possibility of subclass features. However, a reading of the available literature on this subject shows there is a higher potential for subclass influence to be an issue with ejectors than it is with extractors.

A study by Nichols described the possibility of subclass characteristics on the non-working surfaces of extractors [42]. While the particular surfaces Nichols described as exhibiting subclass in his study did not make contact with the cartridge cases during firing or ejection in his study, the comments from some of the participants in this study show that firearm examiners are wary of the possibility of subclass characteristics on extractors, even though subclass has never been reported on their working surfaces (edges). Conversely, subclass is known to be a potential problem on the forward faces of ejectors, as indicated by Thompson’s above-mentioned article regarding the Smith & Wesson ejectors he encountered. However, as has been previously reported, the possibility of such characteristics getting transferred to the cartridge case depends greatly on the orientation of the ejector face to the head of the cartridge during cycling of the firearm [43]. The ejectors of the firearms used in this study did, in fact, display potentially significant subclass carryover on their working surfaces at 30X magnification (Figures 7 through 9), but it is not fully known to what extent, if any, this subclass influence carried over to the marks they left on fired cartridge cases. Based on the microscopic intercomparison of a few of the ejector marks at 120X magnification on fired cartridge cases produced by the firearms used in this study,

reproduction of the subclass influences present on the ejector faces does not appear to have been a significant factor, because the detail seen at 120X is smaller than the detail observed at 30X. Even if some subclass characteristics were imparted to the ejector marks, those examiners who stated that they used these marks as a basis for identification did not indicate to what extent they relied on them to reach their conclusions and could have merely used them as a class characteristic for orientation purposes due to their relative position on the cartridge case. Generally speaking, extractor and ejector marks on fired cartridge cases may not be as routinely compared as breech face marks and firing pin impressions are in firearms identification casework involving semiautomatic pistols, since the identification of extractor and ejector marks typically cannot support a conclusion that a cartridge case was fired in a gun like true cycle-of-fire marks (e.g. breech face marks and firing pin impressions) can. Still, reporting a finding that a cartridge or cartridge case was cycled through the action of a particular gun on the basis of extractor and ejector markings can be useful information in an investigation.

In eight of the reported examinations in this study, the chamber marks were used to aid in the identification. In seven of the examinations, it was reported that “other” features were also used. Some of these other features were indicated to include: “ejector cutout,” “unknown-possible chambering mark,” and “firing pin drag.”

Discussion

This project was designed to provide information on error rates in firearm related casework in a study that presented as little bias to the participating examiners as possible. In order to achieve this goal:

• The participants in the test were unaware of the answers.

• The kits were made up in a random fashion in an effort to prevent any two tests from being exactly the same. The test was designed so multiple examiners in one laboratory could participate in the study; therefore, each kit had to be different to prevent participants from possibly influencing each other’s answers.

• The test administrator was not aware of the answers for any of the test kits, as these were furnished by another examiner in the laboratory.

• The kits given to each examiner were documented only for tracking purposes and the list was not retained once the testing period was completed. At no time was this information viewed in relationship to answers received,



but only in an effort to locate missing test kits.

• The examiners tested were asked to treat these test kits as they would treat casework; second calls (“verifications”) and technical reviews were allowed and encouraged.

While the answers to each test kit were documented and retained in order to determine if the participants came to the correct conclusions, it was understood that errors in kit construction could occur. An effort was made to ensure that the unknowns in each kit were properly documented and that there were no transcription errors. Despite this precaution, the possibility still remained that an error could have occurred. The author was prepared to investigate any submitted result that was incorrect to ensure that this was an error made by the examiner being tested and not due to an improperly designed or documented test kit.

It is well understood among experienced firearms examiners that the same firearm can mark fired cartridge cases differently from shot to shot, even using the same ammunition, due to any number of variables that may occur during the firing process. For example, a firing pin may impress deeper into the primer in some tests than others, breech face marks may be visible on some cases while not present on others, an ejector may mark in the lettering of the headstamp and therefore not be discernible on some cases while marking others in a much clearer manner, and firing pin drag may occur only a fraction of the time. It is because of these variables that inconclusive results were not unexpected.

The firearms used in this study were known to produce similar markings to one another, which potentially made the examinations more challenging. This study was designed to present real world casework to examiners in a format that could be used to determine a realistic frequency of error in a real world situation. While the types of firearms used for this study may be more commonly encountered by firearm examiners in California than in other parts of the United States, this scenario mimics any officer-involved shooting scenario where the majority of the cartridge cases located at the scene will be from firearms of the same make and model. For many firearm examiners, their local law enforcement officers will have been issued Glock firearms [44]. A shooting case involving Glock-fired cartridge cases generally represents a best-case scenario for any firearm examiner because of the relative ease with which the firing pin aperture shear marks on these cartridge cases can be identified. However, Smith & Wesson firearms, including M&P or other third generation type pistols, are generally the second most issued firearms to law enforcement. From information provided by Smith & Wesson during the author’s factory tour, the slides for these firearms are produced

similarly, so it can be inferred that they will all likely produce tool marks on cartridge cases in a fashion similar to those of the Smith & Wesson model 4006TSW firearms used in this study. However, this assumption was not tested by the author and may be an interesting area for additional research.

Despite the difficulties that were specifically designed into this study to present the participants with realistic samples inspired by an actual case, all of the participating examiners were able to correctly distinguish which cartridge case was fired in which firearm, with only one examiner reporting an inconclusive result. Given these results, the author must reject her original hypothesis. This hypothesis stated that when firearm examiners are presented with a no-gun examination, where subclass features may be present, and marks commonly used for identification transfer minimally to test-fired cartridge cases, they will not be able to determine whether or not a particular cartridge case was fired in a particular firearm.

Areas for Additional Research

Looking ahead to future research studies that may deal with similar subjects, a more thorough pre-screening of any samples designed to test examiners’ assessment of subclass features is obviously one area of the current study that could be improved. This study could also be further improved upon if additional firearms were used and knowns from only a portion of those firearms used in the test kits, thus presenting an open set of unknowns to the participants. While this could increase the chances of inconclusive results, it would be a more accurate reflection of the types of evidence received in real casework. With regard to the reporting of inconclusive results, it would have been better to direct the participants to specify which category, 2. a), b), or c), as described in the AFTE Range of Conclusions [45], best described their observations, rather than merely reporting “inconclusive”. Had this been done, it may have provided some insight into what the participant who reported the inconclusive result observed that would not allow them to either identify or eliminate that particular unknown. Requesting that participants submit photographs of any comparisons that result in identification or inconclusive opinions could also go a long way towards answering any lingering questions (especially in instances where potential subclass characteristics may be an issue), but this would also require more time on the part of the participant.

Another way in which this study could be improved would be to use firearms having more prominent subclass features that would transfer to the fired components more readily. While some subclass characteristics appeared to be present on some of the working surfaces of the firearms in this study, most did not transfer significantly to the cartridge cases presented to the



participants. The use of sequentially-manufactured firearms having more defined subclass features would present a greater challenge to the examiners participating in the test.

Addendum: The Single Inconclusive Result

The author is of the opinion that the sole inconclusive result reported for this test was likely the result of cognitive bias on the part of the participating examiner. In order to explore the issue, the test kit that had been assigned to this examiner (who reportedly had 13 years of experience in firearms identification) was re-examined by the author in order to assess why the examiner may have come to this conclusion. Upon viewing the cartridge case in question, Unknown 2, on the comparison microscope, it was apparent that the case had good, clear markings on it. The cartridge case lacked a firing pin drag mark but it had clearly discernible breech face marks and a fairly deep, well-marked firing pin impression. Because the correct results were known to the author, the cartridge case was compared to one of the known cartridge cases from Gun 1, which had been documented on the answer key as having fired Unknown 2 for this kit. The observed agreement in the firing pin impression and the breech face marks was sufficient in quality and quantity to identify Unknown 2 as having been fired in Gun 1. The answer sheet for this examination was re-evaluated and it was noticed that the examiner had marked the result as “inconclusive” when comparing Unknown 2 to Gun 3. There was no indication on the answer sheet as to whether or not the examiner had eliminated, identified, or found inconclusive findings for this cartridge case in comparison to Gun 1 and/or Gun 2. It is interesting to note that in this particular test kit, Unknown 1 had been fired in Gun 2 and Unknown 3 had been fired in Gun 1, both of which the examiner correctly identified. Unknown 2 (the reported inconclusive) had also been fired in Gun 1. It is possible that the examiner assumed that corresponding unknowns were provided for each of the known firearms represented in the test kit, since each kit contained three unknowns and three knowns. If this assumption was made, the examiner could have thought Unknown 2 was likely fired from Gun 3 (since it had not been identified to either of the other unknowns) but could not find sufficient agreement between the compared markings to support this conclusion, hence the inconclusive result. However, this is speculation on the author’s part and this inconclusive result could be due to any number of other

factors, including fatigue or time constraints.

In order to assess what the participating examiner may have observed during their examination of this unknown that led them to report an inconclusive result, the author compared the unknown cartridge case to the known cartridge cases from Gun 3. Random agreement, of the type that can usually be found in the comparison of known non-matching tool marks, was found in both the breech face marks and in the firing pin impressions, but not in sufficient quantity to lead a trained examiner to misidentify it. A comparison of Unknown 2 with Unknown 3 (both of which, for this kit, had been fired in Gun 1) showed there was sufficient agreement of the firing pin impressions for identification; however, there was insufficient agreement of the breech face marks. While this information was useful, the author was aware of the fact that she was potentially subject to confirmation bias because she knew the true answer. Therefore, a second qualified firearm examiner in the author’s laboratory was presented with these items for examination without being given any contextual information. The second examiner compared a single known cartridge case from Gun 3 to Unknown 2 with inconclusive results. The examiner subsequently identified a known cartridge case from Gun 1 to Unknown 2. See Figures 19 through 24, which were taken during the re-evaluation of this test kit.

Acknowledgements

The author would like to acknowledge the following people for their contributions to this research: Michael Barnes of the California DOJ Redding Regional Crime Laboratory; the NFEA class of 2011 and staff, including S.A. Jim Yurgelitis, Jodi Marsanopoli, Sheila Hopkins, and all of the instructors; all of the examiners who participated in this study or offered to have their employees participate in the study; Jason Dubois of Smith & Wesson; and the California Highway Patrol.

The author also gratefully acknowledges and appreciates the significant editorial contributions made by Eric Collins and John Murdock to the blind trial and scientific method portions of this article. And finally, but of the utmost importance, although the research and methodology employed were planned and performed by the author, this paper, with the consent of the author, has been heavily edited and revised by Eric Collins. The author is in his debt for doing so.



Figure 19: Comparison of breech face marks on test-fired cartridge case from Gun 1 (left) vs.

Unknown Cartridge Case #2 (right), showing sufficient agreement for identification, 30X

Figure 20: Comparison of different area of breech face marks on same items as Figure 22, showing

sufficient agreement for identification, 30X

Figure 21: Comparison of breech face marks on test-fired cartridge case from Gun 3 (left) vs.

Unknown Cartridge Case #2 (right), showing insufficient agreement for identification, 30X

Figure 22: Comparison of different area of breech face marks on same items as Figure 22, showing

insufficient agreement for identification, 30X

Figure 23: Comparison of firing pin impressions on test-fired cartridge case from Gun 1 (left) vs.

Unknown Cartridge Case #2 (right), showing sufficient agreement for identification, 30X

Figure 24: Comparison of firing pin impressions on test-fired cartridge case from Gun 3 (left) vs.

Unknown Cartridge Case #2 (right), showing insufficient agreement for identification, 30X



References

[1] Berg, S. “The Drama of Forensic Ballistics,” AFTE Journal, 1979, 11(3), pp. 44-48.[2] Nichols, R., “The Scientific Foundations of Firearms and Tool Mark Identification-A Response to Recent Challenges,” The CACNews, Second Quarter 2006, pp 8-27.[3] Schwartz, A., “Challenging Firearms and Toolmark Identification—Part I,” The Champion, October 2008, pp 10-19. [4] “Understand the Causes: Unvalidated or Improper Forensic Science.” The Innocence Project [Internet]. Available from: <http://www.innocenceproject.org>. Accessed 2011 Aug 8.[5] Clarke, M. “Crime Labs in Crisis: Shoddy Forensics Used to Secure Convictions.” Prison Legal News, 2010, 21(10), pp 1-22. [6] Mnookin, J.L.; Cole, S.A.; Dror, I.E.; Fisher, B.A.; Houck, M.M.; Inman, K.; Kave, D.H.; Koehler, J.J.; Langenburg, G.; Risinger, D.M.; Rudin, N.; Siegel, J.; and Stoney, D.A., “The Need for a Research Culture in the Forensic Sciences,” UCLA Law Review, 2011, 58(3), pp 725-779. [7] National Academy of Sciences. “Strengthening Forensic Science in the United States: A Path Forward.” Washington, D.C.: The National Academies Press, 2009.[8] “FAQs About Clinical Studies”. 2010 Mar 1. National Institutes of Health [Internet]. Available from: <http://www.cc.nih.gov/participate/faqaboutcs.shtml>. Accessed 2011 Aug 8.[9] Saks, M.J and Koehler, J.J. “The Individualization Fallacy in Forensic Science Evidence,” Vanderbilt Law Review, 2008, 61(1), p. 201, footnote 9.[10] Peterson, J.L.; Lin, G.; Ho, M.; Chen, Yingyu, and Gaensslen, R.E. “The Feasibility of External Blind Proficiency Testing. I. Background and Findings,” Journal of Forensic Sciences, 2003, 48 (1), p. 6.[11] National Academy of Sciences, op. cit., p. 207.[12] “FAQs About Clinical Studies”, op. cit.[13] “CTS Statement on the use of Proficiency Testing Data for Error Rate Determinations.” 2010 March 30. Collaborative Testing Services, Inc. (CTS) Forensics [Internet]. Available from: <http://www.ctsforensics.com/>. Accessed 2011 August 10.[14] Ibid.[15] Grzybowski, R.; Miller, J.; Moran, B.; Murdock, J.; Nichols, R.; and Thompson, R. “Firearm/Toolmark Identification: Passing the Reliability Test Under Federal and State Evidentiary Standards,” AFTE Journal, 2003, 35(2), p. 216.[16] Kaye, D.H; Bernstein, D.E.; and Mnookin, J.L. The New Wigmore: A Treatise on Evidence: Expert Evidence. R.D. Friedman, General Editor. Aspen Publishers (2nd ed. 2011), § 13.3.3, pp. 605-606.

[17] Expert Working Group on Human Factors in Latent Print Analysis. Latent Print Examination and Human Factors: Improving the Practice through a Systems Approach. U.S. Department of Commerce, National Institute of Standards and Technology, 2012, p. 33.[18] DeFrance, C. and Van Arsdale, M.D. “Validation Study of Electrochemical Rifling,” AFTE Journal, 2003, 35(1), pp. 35-37.[19] Clow, C. “Cartilage Stabbing with Consecutively Manufactured Knives: A Response to Ramirez v. State of Florida,” AFTE Journal, 2005, 37(2), pp. 86-116.[20] Lightstone, L. “The Potential for and Persistence of Subclass Characteristics on the Breech Faces of SW40VE Smith & Wesson Sigma Pistols,” AFTE Journal, 2010, 42(4), pp. 308-322.[21] Smith, E.D. “Cartridge Case and Bullet Comparison Validation Study with Firearms Submitted in Case Work,” AFTE Journal, 2005, 37(2), pp. 130-135.[22] Peterson, et al., op. cit., pp. 7-10.[23] Bunch, S.G., and Murphy, D.P. “A Comprehensive Validity Study for the Forensic Examination of Cartridge Cases,” AFTE Journal, 2003, 35(2), pp. 201-203.[24] Giroux, B.N. “Empirical and Validation Study: Consecutively Manufactured Screwdrivers,” AFTE Journal, 2009, 41(2), pp. 153-158.[25] Bunch and Murphy, op. cit., p. 201.[26] Giroux, op. cit., p. 156.[27] Kaye, D.H. “Identification, Individualization, Uniqueness,” Law, Probability & Risk, 2009, 8(2), p, 87, footnote 18.[28] Schwartz, op. cit.[29] “Understand the Causes: Unvalidated or Improper Forensic Science,” op. cit.[30] Clarke, op. cit.[31] Mnookin, et al., op. cit.[32] Diamond, B.L. “The Scientific Method and the Law.” The Hastings Law Journal, Vol. 19, Nov. 1967, p.193.[33] Grzybowski, R. and Murdock, J. “Firearm and Toolmark Identification - Meeting the Daubert Challenge,” AFTE Journal, 1998, 30(1), pp. 6-8.[34] Mnookin, et al., op. cit., pp. 743-744.[35] Supica, J. and Nahas, R., Standard Catalog of Smith & Wesson, Second Edition, WI: Krause Publications, 2001, p. 237.[36] Murphy, Doug. “CTS Error Rates, 1992-2005 Firearms/Toolmarks” [PDF document]. Originally presented at the AFTE Training Seminar in Henderson, Nevada on May 5, 2010. Retrieved from Scientific Working Group for Firearms and Toolmarks website: http://www.swggun.org/resources/docs/CTSErrorRates.pdf. Accessed June 10, 2013.[37] Lightstone, op. cit., p. 310-313.[38] Ibid., p. 312.



[39] Thompson, E. “9mm Smith & Wesson Ejectors,” AFTE Journal, 2002, 34(4), pp. 406-407.[40] Kennington, R.H. “Grinding Marks: Paradigm Lost.” Unpublished paper presented at AFTE Annual Training Seminar in St. Louis, Missouri, June 2000. Kennington’s presentation abstract, containing his experimental results and photographs, were provided to the author courtesy of John E. Murdock.[41] “Range of Conclusions Possible When Comparing Toolmarks.” AFTE Glossary, 5th edition, ver. 5.070207, pp. 118-119.[42] Nichols, R.G. “Firearm and Tool Mark Identification: The Scientific Reliability and Validity of the AFTE Theory of Identification Discussed Within the Framework of a Study of Ten Consecutively Manufactured Extractors,” AFTE Journal,

2004, 36(1), pp. 67-88.[43] Biasotti, A., Murdock, J., and Moran, B. Chapter 35 “Firearms and Toolmark Identification”, pp. 641-730 in Vol. 4, Modern Scientific Evidence: The Law and Science of Expert Testimony, 2011-2012 Edition (Faigman, DL, Blumenthal, JA, Sanders, J, Chen, EK, Mnookin, JK, and Murphy, EE.), Eagan, MN: Thompson-Reuters/West, p. 690. [44] Ayoob, Massad, “The Glock 22: America’s Best-Selling Police Pistol,” Guns, July 2008, pp. 14. [45] AFTE Criteria for Identification Committee Report: “Theory of Identification, Range of Striae Comparison Reports and Modified Glossary Definitions - an AFTE Criteria for Identification Committee Report”, AFTE Journal, Vol. 24, No. 3, July 1992, pp. 336-340.

Association of Firearm and Tool Mark Examiners

2014-2015

PRESIDENT KATHERINE RICHERT Alabama Dept. of Forensic Sciences 525 Carter Hill Road Montgomery, AL 36106 Tel: (334) 242-2938 Fax: (334) 240-3284 E-mail: [email protected] lST VICE PRESIDENT BRANDON N. GIROUX Giroux Forensic, Inc. P. O. Box 231 Northville, MI 48167 Tel: (248) 762-5731 E-mail: [email protected] 2ND VICE PRESIDENT TRAVIS SPINDER Montana Dept. of Justice Forensic Science Division 2679 Palmer Street Missoula, MT 59808 Tel: (406) 329-1127 Fax: (406) 549-1067 E-mail: [email protected] SECRETARY WENDY M. GIBSON Virginia Department of Forensic Science Firearms/Toolmark Section 6600 Northside High School Road Roanoke, VA 24019-2837 Tel: (540) 283-5932 Fax: (540) 561-6608 E-mail: [email protected] MEMBERSHIP SECRETARY ALISON L. QUEREAU Palm Beach County Sheriff’s Office, Crime Lab, Firearms Unit 3228 Gun Club Road West Palm Beach, FL 33406 Tel: (561) 688-4288 Fax: (561) 688-4234 E-mail: [email protected] TREASURER MELISSA OBERG Indiana State Police Crime Laboratory 550 West 16th Street, Suite C Indianapolis, IN 46202 Tel: (317) 921-5387 Fax: (317) 927-3087 E-mail: [email protected] PAST PRESIDENT MARK A. KEISLER Indiana State Police Crime Lab 550 West 16th Street, Suite C Indianapolis, IN 46202 Tel: (317) 921-5384 Fax: (317) 927-3087 E-mail: [email protected] MEMBERS AT LARGE LANNIE G. EMANUEL Forensic Firearm & Toolmark Examiners (FFATME) 265 Valley View Trail Double Oak, TX 75077 Tel: (972) 200-5030 E-mail: [email protected] ANDY SMITH San Francisco Police Department Criminalistics Laboratory 850 Bryant Street San Francisco, CA 94103 Tel: (415) 671-3264 Fax: (415) 671-3290 E-mail: [email protected] AFTE JOURNAL EDITOR COLE GOATER Kansas Bureau of Investigation 1620 SW Tyler Street Topeka, KS 66612 Tel: (785) 296-8235 E-mail: [email protected]

December 7, 2014 Deborah Runkle Senior Program Associate American Association for the Advancement of Science 1200 New York Avenue, N.W. Washington, DC 20005 Ms. Runkle,

Last Wednesday, I was informed by John Murdock that during the National Institute of Standards and Technology (NIST) Forensic Symposium last week he was fortunate enough to have met you and learned that you are the Program Manager for the pending AAAS review of the compendium of reference articles submitted by the Association of Firearm and Tool Mark Examiners (AFTE) to the Subcommittee on Forensic Science (SoFS). He told me that as Program Manager you will be responsible for the group that will perform the long-awaited evaluation of the scientific references compiled by the Scientific Working Group for Firearms and Toolmarks (SWGGUN) and the AFTE Committee for the Advancement of the Science of Firearm and Toolmark Identification in response to the 25 foundational questions received from the SoFS, Research, Development, Testing, & Evaluation Interagency Working Group (RDT&E IWG) in 2011. John was encouraged to hear that a group with a reputation such as yours is evaluating these articles, and I concur. Since mid-2011, John has been in continuous contact with RDT&E IWG Chair John Paul Jones regarding an evaluation of this work product, and we are very pleased to hear that this is moving forward.

I want to extend to your group any assistance that AFTE can provide during your evaluation. We understand that the evaluation of our 94-page compilation of published research, along with relevant research published since then, will be a daunting task. If needed, AFTE would like to provide electronic copies of all of these publications in order to lessen the burden on your group to search for these articles. Furthermore, it is my understanding that, due to time limitations, not all 25 questions or references provided will be evaluated. AFTE can, and is willing, to also identify the articles which have more importance than others in the evaluation of the scientific underpinnings in the field of firearm and toolmark identification.

Additionally, I was told that you want to select someone that has a working knowledge of the field of firearm and toolmark identification as a member of your group. If approved by his agency to participate, Robert Thompson, currently with NIST, would be AFTE’s choice for this position. If Robert is selected, I have extended any support to him that AFTE can provide throughout this process.

AFTE firmly believes that firearm and toolmark identification rests on firm scientific underpinnings, and we sincerely hope that your evaluation group feels the same way after reviewing our references. Please feel free to contact me at anytime if AFTE may assist you.

Sincerely,

Katherine T. Richert 46th AFTE President


Email Cover Letter composed by Chairman John Murdock - Committee for the Advancement of the Science of Firearm and Toolmark Identification

Dated: June 14, 2011 John Paul: You are receiving this message in your capacity as the RDT&E IWG Executive Secretary. First, thank you for extending the deadline to today, June 14, 2011, for the AFTE Response to the 25 Questions related to firearms and toolmark examinations promulgated by the RDT&E IWG. The AFTE Board of Directors delegated the responsibility of editing and adding to the list of references that you received from SWGGUN on May 20, 2011 to its Committee for the Advancement of the Science of Firearm and Toolmark Identification, which I chair. I have attached what AFTE considers to be appropriate scientific references for each question. This 94 page document is dated June 14, 2011. Finally, it is important to me that you know the names of the AFTE Members on my committee. They are Brandon Giroux, Luke Haag, Jim Hamby, Andy Smith and Pete Striupaitis. Everyone of us contributed, in many ways, to AFTE's response to your 25 questions. But, we only assembled the references. Those that assemble must never be confused with those that create. The real credit goes to the many men and women who, over the years, have conducted the research that makes AFTE confident that you will find that firearms and toolmark identification, as a forensic specialty, rests on firm scientific underpinnings. We hope that these references will provide you with a better understanding of the research related to our field. If you experience difficulty acquiring any of these references, we will provide them to you. John Murdock, Chairman

_____________________________________________________________________________________

The 47 page SWGGUN response was submitted to the SoFS RDT&E IWG on May 20, 2011. This 94 page

combined response was submitted on June 14, 2011.

SWGGUN and AFTE Committee for the Advancement of the Science of Firearm

and Toolmark Identification’s response to 25 foundational firearm and toolmark

examination questions received from the Subcommittee on Forensic Science

(SoFS), Research, Development, Testing, & Evaluation Interagency Working

Group (RDT&E IWG) on April 18, 2011. This response is a compilation of

published research which addresses each question.

Foreword to Questions #1 through #9:

Toolmark Identification is an applied science. It is congruous with applied research, which uses some part of research communities‘ accumulated theories, knowledge, methods and techniques for a specific commercial or client driven purpose.

Applied science differs from fundamental science. Applied science focuses on practical applications with less emphasis on the most basic scientific principles.

The forensic science discipline of Firearm and Toolmark Identification (FA & TM ID) is derived from validated theories in the physical sciences. Specifically, the origination of toolmarks is based on previously established theories, principles and properties that were adapted in the material and engineering sciences. These essential principles, which can be found in innumerable textbooks, are delineated below, followed by a limited representative reference list:

I. Physical Properties

A. Pressure

B. Temperature- Friction & Heat

II. Metallurgical Properties

A. Plastic Deformation

B. Stress-Strain Relationships

C. Failure Mechanics

D. Forces

1. Compression

2. Torsion

3. Shear

4. Tensile

June 14, 2011June 14, 2011 1

5. Flexure

III. Mechanical Properties- Materials reaction to applied forces

A. Chip Formation Processes & Phenomena/Theory

B. Non-Chip or Chipless Formation Processes & Phenomena/Theory

1. Electro-chemical machining (ECM)

2. Electro-discharge machining (EDM)

3. Laser

IV. Surface Integrity

A. Fatigue/Fracture Mechanics

B. Hardness

C. Heat Transfer

D. Texture-

1. Roughness, Waviness & Lay

2. Metrology- Provides standard of three primary components to describe 3D Surface texture and supplies a quantitative basis for toolmarks. E. Tribology- Established body of knowledge that explains wear and the random affects of tool wear Fundamental References: Brandt D., Metallurgy Fundamentals, Goodheart-Wilcox Company Inc., 1985 Ostwald and Munoz, Manufacturing Processes and Systems, John Wiley & Sons, Ninth Edition, 1997 Wright R.T., Processes of Manufacturing, The Goodheart-Wilcox Co., Inc., 1987 DuVall J.B., Contemporary Manufacturing Processes, Goodheart-Wilcox Co., Inc., 1996 Hurd D., Silver M., Bacher A.B., & McLaughlin C.W., Physical Science, Prentice-Hall, New Edition, 1993 Salmon, S.C., Modern Grinding Process Technology, McGraw-Hill, Inc., 1992

2June 14, 2011 2

McCarthy WJ and Smith R.E., Machine Tool Technology, McKnight & McKnight Publishing, 1968 Ernst and Merchant, Chip Formation, Friction and Finish, The Cincinnati Milling Co., Cincinnati, Ohio De Garmo, E.P., Materials and Processes in Manufacturing, The MacMillian Co., 3rd Edition, 1969 De Garmo, E.P., Black, J.T., Kohser, R.A., Materials & Processing in Manufacturing, MacMillian Publishing Co., 7th Edition, 1988 Amstead, B.H., Ostwald, P.F., Begeman, M.L., Manufacturing Processes, Wiley & Sons, 8th Edition, 1987 Wright, T.R., Processes of Manufacturing, Goodheart-Wilcox, 1987 Pollack, H. W., Materials Science & Metallurgy, Reston Publications, 1973 Neely, J., Practical Metallurgy & Materials, Wiley & Sons, 1979 Crossover References Biasotti, A., ―The Principles of Evidence Evaluation as Applied to Firearms and Tool Mark Identification‖, AFTE Journal, Volume 9, Number 4, October 1964. Burrard, G, The Identification of Firearms and Forensic Ballistics, Butler & Tanner 1934, Reprinted Barnes & Company 1962 and Wolfe publishing 1990 Davis, JE, An Introduction to Toolmarks, Firearms and the Striagraph, Charles C. Thomas, 1958 Goddard, Waite, Fisher and Gravelle, Army Ordnance, November & December 1925 Gunther J.D., and Gunther C.O., The Identification of Firearms, Wiley & Sons, Inc. 1935 Hatcher, J.S., Textbook of Firearms Investigation, Identification and Evidence, Small Arms Technical Publishing Company, 1935 Hatcher Jury & Weller Hatcher, J.S., Jury, F.J. and Weller, J., Firearm Investigation Identification and Evidence, The Stackpole Company, 1957. Mathews, JH, Firearms Identification, Volumes I-III, University of Wisconsin Press, 1962

3June 14, 2011 3

Peterson, J.L., "Utilizing the Laser for Comparing Tool Striations"; Journal of the Forensic Science Society, 57 (14), 1974, pp. 57-62 Vandiver, J.V., "Identification and Use of Toolmark Identification", Law and Order, No. 7, 1976

4June 14, 2011 4

1. What literature documents the scientific domains used to inform the foundations of firearm/toolmark analysis? Have the relevant communities and/or standards setting organizations looked to engineering, material sciences, etc. for experimental design, lessons learned and research which can inform advancing the practice of firearms/toolmark analysis? If so, what references exist to document this crossover of information?

See Fundamental and Crossover References mentioned above.

The history of firearms identification and court acceptance of firearms and toolmark evidence in the United States goes back over 100 years and has been the subject of numerous publications. The following list of references represents a portion of these publications. The last two articles listed represent special contributions made by Kraft in 1931 when he authored a comprehensive two-part summary of the literature produced during the period 1919 through 1930 that dealt with (in part one) ―the identification of weapons by means of the projectile and cartridge case‖ and (in part two) ―other questions that may arise in forensic ballistics." Kraft‘s critical review of many articles written in German, etc. made them available to many English speaking examiners for the first time. The AFTE Theory of Identification, developed and adopted by the relevant scientific community (AFTE in 1992), has provided the toolmark identification community with a theory defining and describing the approach that examiners have traditionally taken when identifying/individualizing toolmarks.

Buxton, J., ―The Science of Ballistics: Judicial Applications‖, American Journal of Police Sciences, May – June, 1931, 2(3): 211-219

Serhant, J. ―The Admissibility of Ballistics in Evidence‖, American Journal of Police Sciences, May – June, 1931, 2(3): 202-210

Baker, N., ―The Campbell Case‖, American Journal of Police Science, Jan – Feb, 1931, 3(1): 21-31

Inbau, F., ―Scientific Evidence in Criminal Cases (Firearms Identification – ―Ballistics‖)‖, AFTE Journal, Vol. 13, No. 2, 1981: 281 (Originally appeared in Journal of Police Science, 1933)

Gunther, J.D., and Gunther C.O, ―The Identification of Firearms‖, Wiley & Sons, Inc. 1935.

Goddard, C. ―A History of Firearm Identification‖, AFTE Journal. Vol. 17, No. 1, 1985, pp. 55 – 68 (Originally printed in Chicago Police Journal, 1936).

FBI, ―Firearms Identification‖, U.S. Government Printing Office, 1941, pp 17 – 33 ―Development and Admissibility of Ballistics and Firearms Evidence‖.

5June 14, 2011 5

Goddard, C., ―A History of Firearms Identification to 1930.‖ AFTE Journal, Vol. 31, No. 3, 1999, pp. 225 – 241.

IAI Firearms Sub-Committee - Stanton O. Berg, Chairman, ―The History of Firearms Identification‖, Identification News, June 1965, pp. 5 – 15.

Saferstein, R. ―Firearms Identification – Historical Background‖, Forensic Science Handbook – Volume II, Prentice Hall, 1988, see p. 411 – 415.

Hamby, J., ―The History of Firearm and Toolmark Identification‖, AFTE Journal, Vol. 31, No. 3, 1999, pp. 266-284.

Kraft, B., ―Critical Review of Forensic Ballistics – Part I‖, American Journal of Police Science, Jan-Feb 1931, 2(1), pp. 52 – 66.

Kraft, B., ―Critical Review of Forensic Ballistics – Part II‖, American Journal of Police Science, Mar – Apr, 1931, 2(2), pp. 125 – 142.

Moran, B. and Murdock, J. ―Zen and the Art of Motorcycle Maintenance – Contribution to Forensic Science – An Explanation of the Scientific Method‖ Appendix No. 2 (pp. 234-240) from the article by Grzybowski, R., Miller, J., Moran, B., Murdock, J., Nichols, R., and R. Thompson titled ―Firearm/Toolmark Identification: Passing the Reliability Test Under Federal and State Evidentiary Standards‖ in AFTE Journal, Vol. 35, No. 2, Spring 2003, pp. 209-241.

This appendix describes how the classical scientific method is used in firearm and toolmark identification. Column four in the scientific method chart generally describes research of the type summarized by Nichols. It is this research that led directly to the adoption of the AFTE Theory of Identification in 1992.

Biasotti, A.A., (1981) Rifling Methods – A Review and Assessment of the Individual Characteristics Produced., Association of Firearm and Toolmark Examiners Journal, Vol. 13, No. 3, pp. 34-61.

The author reviews the various methods of rifling barrels and the types of marks that they produce on bullets.

Biasotti, A., (1981) Bullet Bearing Surface Composition and Rifling (Bore) Conditions as Variables in the Reproduction of Individual Characteristics on Fired Bullets Association of Firearm and Toolmark Examiners Journal , Vol. 13, No. 2, pp. 94-102.

The purpose of the experiment described herein is to demonstrate the effects of several of the more significant variables that may contribute towards the reproducibility of identifiable individual characteristics on fired bullets. The author discusses individual characteristics via an examination of various types of bullets (Lubaloy, Golden, and Nyclad) and various conditions of the bore.

6June 14, 2011 6

Nichols, R.G., ―Firearm and Toolmark Identification Criteria: A Review of the Literature‖,

Journal of Forensic Sciences, Vol. 42, No. 3, 1997, pp.446-474.

A review of 34 articles pertaining to the examination of consecutive manufactured tools, identification criteria for firearms and toolmark identification and mathematical and computer models developed for a standard identification.

Nichols, R.G. ―Firearm and Toolmark Identification Criteria: A Review of the Literature,

Part II‖, Journal of Forensic Sciences, Vol. 48, No. 2, March 2003, pp 318-327.

An update to a previously published review of articles pertaining to firearm and toolmark identification criteria is presented. In this update, 22 additional articles were reviewed, including works of a general nature, studies critically assessing the theory of consecutive matching striations, empirical studies involving various firearm components, toolmark studies, as well as articles discussing the utility of statistics in the firearms and toolmark identification discipline. These articles have been reviewed in a format to permit others to learn what has been published in the field in an effort to educate interested parties. Further, a discussion of the importance of articulation and communication within the discipline is presented.

Nichols, R.G., Defending the Science of the Firearms and Tool Mark Identification Discipline: Responding to Recent Challenges, Journal of Forensic Sciences, Vol. 52, No. 3, May 2007, pp. 586-594.

A compendium of fifty-six (56) references that includes approximately thirty-two (32) articles that describe the examination of consecutively, or nearly consecutively, manufactured firearms components.

Katterwe, H "Modern Approaches for the Examination of Toolmarks and Other Surfaces", Forensic Science Review, Volume. 8, Number. 1, Pp. 46-71, June 1996

The author explores the effects of the production of toolmarks on different materials‘ surfaces.

Wiercigroch, M., Cheng A. (1997) Chaotic and Stochastic Dynamics of Orthogonal Metal Cutting. Chaos, Solitons and Fractals, 8:4, April 1997, pp. 715-726.

The authors explore the effects of the machining processes as it relates to vibration of the machine tools and cutting resistance. It is demonstrated that the result is random material grain sizes.

7June 14, 2011 7

2. Have studies been conducted at the manufacturing level addressing material uniformity, reproducibility, and the QA/QC procedures of the manufacturer?

Bonfanti, M.S. and DeKinder, ―The Influence of Manufacturing Processes on the Identification of Bullets and Cartridge Cases- A Review of the Literature‖, Science and Justice, Volume 39, No. 1, 1999, pp. 3-10.

A compendium of fifty (50) references that describe the examination of consecutively, or nearly consecutively, manufactured firearms components.

Nichols, R.G., ―Firearm and Toolmark Identification Criteria: A Review of the Literature‖,

Journal of Forensic Sciences, Vol. 42, No. 3, 1997, pp.446-474.

A review of 34 articles pertaining to the examination of consecutive manufactured tools, identification criteria for firearms and toolmark identification and mathematical and computer models developed for a standard identification.

Nichols, R.G. ―Firearm and Toolmark Identification Criteria: A Review of the Literature,

Part II‖, Journal of Forensic Sciences, Vol. 48, No. 2, March 2003, pp 318-327.

An update to a previously published review of articles pertaining to firearm and toolmark identification criteria is presented. In this update, 22 additional articles were reviewed, including works of a general nature, studies critically assessing the theory of consecutive matching striations, empirical studies involving various firearm components, toolmark studies, as well as articles discussing the utility of statistics in the firearms and toolmark identification discipline. These articles have been reviewed in a format to permit others to learn what has been published in the field in an effort to educate interested parties. Further, a discussion of the importance of articulation and communication within the discipline is presented.

Nichols, R.G., Defending the Science of the Firearms and Tool Mark Identification Discipline: Responding to Recent Challenges, Journal of Forensic Sciences, Vol. 52, No. 3, May 2007, pp. 586-594.

A compendium of fifty-six (56) references that includes approximately thirty-two (32) articles that describe the examination of consecutively, or nearly consecutively, manufactured firearms components.

Springer, E., Toolmark Examinations – A Review of Its Development in the Literature., Journal of Forensic Sciences, Vol. 40, No. 6, November 1995, pp.964-8.

A review of forty-seven (47) articles pertaining to toolmark examinations. This includes a history of toolmark examinations, a review of its development from

8June 14, 2011 8

1900 to present, and addresses the use of automated technology in conducting toolmark examination/ validation .

Coffman, B.C., (2003). Computer Numerical Control (CNC) Production Tooling and Repeatable Characteristics on Ten Remington Model 870 Production Run Breech Bolts. Association of Firearm and Toolmark Examiners Journal, 35:1, pp. 49-54.

The authors examine ten shotgun bolt faces, consecutively produced by the same CNC manufacturing machine tool and compare for the presence subclass and individual characteristics. Results of these comparisons found that the manufacturing process used to fabricate these bolts produced subclass characteristics and sufficient individual characteristics to provide uniqueness.

Lopez, Laura and Sally Grew. ―Consecutively Machined Ruger Bolt Faces.‖ AFTE Journal, Vol. 32, No. 1, Spring 2000, pp.19 - 24.

This study warns that one should be careful with microscopic marks from a bolt face machined with an end mill. A misidentification is possible unless the identification is made using wear or machining ―chatter‖ marks.

9June 14, 2011 9

3. What toolmark reproducibility studies have been conducted?

Bachrach B., Jain A., Jung S., and Koons R.D.(2010) A Statistical Validation of the Individuality and Repeatability of Striate Tool Marks: Screwdrivers and Tongue and Groove Pliers. Journal of Forensic Sciences, Vol. 55, No. 2, pp 348-357.

Study that statistically validated the original premise of individuality in Toolmark Identification by analyzing statistical distributions of similar values resulting from the comparison of Known Matches (KM) and Known Non-Matched (KNM) pairs of striated toolmarks. This quantifiable analysis of KM and KNM toolmark similarity distributions showed nearly error-free identifications.

Doelling, B., ―Comparison of 4000 Consecutively Fired Steel Jacketed Bullets‖, Abstract B58, p. 53 from Proceedings of the AAFS Annual Meeting, Seattle, WA, February 19 -24, 2001

Author examined 4000 fired bullets using conventional pattern matching as well as quantitative consecutive matching striation (CMS) techniques

Fadul, T.G., An Empirical Study to Evaluate the Repeatability and Uniqueness of Striations/Impressions Imparted on Consecutively Manufactured Glock EBIS Barrels, AFTE Journal, Vol. 43,No. 1, Winter, 201, pp.37-44.

An empirical study of ten consecutively manufactured Glock barrels containing the Enhanced Bullet Identification System (EBIS). Study consisted of test sets sent to 238 examiners from 150 laboratories in 44 states and 9 countries that were designed to test the examiner‘s ability to correctly identify fired bullets to the barrel that fired them. The results from 183 of these examiners produced an error rate of 0.4%. This study validated the repeatability and uniqueness of striated markings in gun barrels, as well as the ability of a competent examiner to reliably identify fired bullets to the barrels that marked them.

Gouwe, J., Hamby, J.E., Norris, S. (2008). Comparison of 10,000 Consecutively Fired Cartridge Cases from a Model 22 Glock .40 S&W Caliber Semiautomatic Pistol. AFTE Journal, Vol. 40, No. 1, pp. 57-63.

Ten thousand (10,000) .40 S&W caliber cartridge cases fired from a Glock, model 22, pistol were compared. All 10,000 fired cases could be identified to each other. This study validates previous durability studies that showed identifiable markings from a tool could persist for a long period of time.

Hamby, J. Identification of Projectiles. AFTE Journal, Vol. 6, No. 5/6, Fall 1974, pp 22-24

Durability study of 501 fired bullets and cartridge cases fired through a worn M16 rifle. The rifle was fired as quickly as the 20 round magazines could be changed

10June 14, 2011 10

into the rifle. It was possible to identify fired bullet 1 to bullet 501 as well as identifying fired cartridge case 1 to case 501.

Hamby, J.E., Brundage, D., Thorpe, J., The Identification of Bullets Fired from 10 Consecutively Rifled 9mm Ruger Pistol Barrels: A Research Project Involving 507 Participants from 20 Countries. AFTE Journal, Vol. 41, Number 2, Fall 2009, pp. 99 – 110.

Ten consecutively rifled RUGER P-85 pistol barrels were obtained from the manufacturer and then test fired to produce known test bullets and 'unknown' bullets for comparison by firearms examiners from around the world. This study is a continuation of one originally designed and reported on by David Brundage. The original study was primarily limited to examiners from nationally accredited laboratories in the United States. For this study, the sets were provided to firearms examiners around the world. The Ruger P85 pistol and the 10 consecutively rifled barrels used for the original study were borrowed from the Illinois State Police. Ammunition was obtained from the Winchester Ammunition Company (A Division of Olin) and 240 tests sets were produced and distributed to forensic scientists and researchers worldwide. A thesis which involved a total of 201 participants including the original 67 reported on by Brundage was published by Hamby and Thorpe in 2001. This paper reports the final conclusions of the research conducted by Brundage, Hamby and Thorpe over a 10 year period.

The total number of participants to date (6-2011) is 561 participants from 21 countries, eight of whom used instrumental analysis to correctly identify the consecutively rifled barrels. Also reported within the research was the examination and identification of the 16,800 fired cartridge cases produced during this experiment.

Kirby, S. ―Comparison of 900 Consecutively Fired Bullets and Cartridge Cases from a. 455 Caliber S&W Revolver‖, AFTE Journal, Vol. 33. No. 3, Summer 2001, pp. 113-125.

Durability study of major working edges of a revolver.

Ogihara, Y., et al, ―Comparison of 5000 Consecutively Fired Bullets and Cartridge Cases From a 45 Caliber M1911 Pistol‖, AFTE Journal, Vol. 15, No. 3, July 1983, pp. 127-140.

Durability study of test-fired bullets and cartridge cases from a single pistol. It was possible to identify bullet number 1 to number 5000 and cartridge case 1 to number 5000.

Schecter, B., et al. ―Extended Firing of a Galil Assault Rifle‖, AFTE Journal, Vol. 24, No. 1, January 1992, pp. 37 – 45.

11June 14, 2011 11

Authors performed extended firings of a new Galil Assault Rifle firing some 7,100 cartridges. Identifiable ejector marks were successfully compared between case number 9 and number 7,060.

Shem, R. and Striupaitis, P., ―Comparison of 501 Consecutively Fired Bullets and Cartridge Cases from a Raven 25 Caliber Raven Pistol‖, AFTE Journal, Vol. 15, No. 3, July 1983, pp. 109 – 112.

Durability study of major working surfaces of a pistol for the test-fired bullets and cartridge cases. It was possible to identify bullet number 1 to number 501 and cartridge case 1 to number 501.

12June 14, 2011 12

4. As manufacturing techniques and materials change over time, what studies

have been performed to validate or invalidate older foundational studies?

Austin, Peter F., ―The Identification of Bullets Fired From High Velocity Rifles with Consecutively Rifled Micro-Groove Barrels‖, Presented at the Firth International Meeting

of Forensic Sciences – University of Toronto, Toronto, Ontario, Canada, June 5 to 11, 1969.

Author discusses his ability to identify and individualize fired bullets obtained by test firing consecutively rifled micro-groove barrels.

Lutz, M., ―Consecutive Revolver Barrels‖, AFTE Newsletter, No.9, Page 24 (1970).

Author obtained one long revolver barrel, had it cut into two barrels, and then test fired both barrels to determine that it was possible to individualize the fired bullets to their specific barrel.

Matty, W.‖ Raven .25 Automatic Pistol Breech Face Tool Marks‖. AFTE Journal, Vol. 16, No. 3, Fall 1984, pp. 57-60.

For this study, three consecutively made breechfaces from Raven pistols were compared. The concentric toolmarks on the breechfaces were found to be individual and not subclass.

Lyons, D. J. ―The Identification of Consecutively Manufactured Extractors‖, AFTE Journal, Vol. 41, No. 3, Fall 2009, pp.246-256.

Study conducted on ten consecutively manufactured firearm extractors. Firearm and toolmark examiners from different laboratories were given ten sets of cartridge cases marked by these extractors to attempt to make the correct associations between the known and unknown cases. Each examiner also received twelve unknown marked cases in addition to the standards for the ten consecutively manufactured cartridge cases, with each known specimen having at least one unknown specimen associated with it.

Hamby, J.E., Brundage, D., Thorpe, J., ―The Identification of Bullets Fired from 10 Consecutively Rifled 9mm Ruger Pistol Barrels: A Research Project Involving 507 Participants from 20 Countries‖, AFTE Journal, Vol. 41, Number 2, Fall 2009, pp. 99 – 110.

Ten consecutively rifled RUGER P-85 pistol barrels were obtained from the manufacturer and then test fired to produce known test bullets and 'unknown' bullets for comparison by firearms examiners from around the world. This study is a continuation of one originally designed and reported on by David Brundage. The original study was primarily limited to examiners from nationally accredited laboratories in the United States. For this study, the sets were provided to

13June 14, 2011 13

firearms examiners around the world. The Ruger P85 pistol and the 10 consecutively rifled barrels used for the original study were borrowed from the Illinois State Police. Ammunition was obtained from the Winchester Ammunition Company (A Division of Olin) and 240 tests sets were produced and distributed to forensic scientists and researchers worldwide. A thesis which involved a total of 201 participants including the original 67 reported on by Brundage was published by Hamby and Thorpe in 2001. This paper reports the final conclusions of the research conducted by Brundage, Hamby and Thorpe over a 10 year period.

The total number of participants to date (6-2011) is 561 participants from 21 countries; eight of whom used instrumental analysis to correctly identify the consecutively rifled bullets. Also reported within this research was the examination and identification of the 16,800 fired cartridge cases produced during this experiment. It was possible to individualize fired case number 1 to case 16,800.

Fadul, T.G., "An Empirical Study to Evaluate the Repeatability and Uniqueness of Striations/Impressions Imparted on Consecutively Manufactured Glock EBIS Barrels", AFTE Journal, Vol. 43,No. 1, Winter, 201, pp.37-44.

The author details an empirical study of ten consecutively manufactured Glock barrels containing the Enhanced Bullet Identification System (EBIS). Study consisted of test sets sent to 238 examiners from 150 laboratories in 44 states and 9 countries that were designed to test the examiner‘s ability to correctly identify fired bullets to the barrel that fired them. The results from 183 of these examiners produced an error rate of 0.4%. This study validated the repeatability and uniqueness of striated markings in gun barrels, as well as the ability of a competent examiner to reliably identify fired bullets to the barrels that marked them.

Fadul, T.G., et al ―An Empirical Study to Evaluate the Repeatability and Uniqueness of Striations / Impressions in Fired Cartridge Casings Fired in 10 Consecutively Manufactured Slides‖ (This project was supported by Award NO. 2009-DN-BX-K230 awarded by the National Institute of Justice, Office of Justice Programs, US Department of Justice) Presented at the 42nd Annual Training Seminar of the Association of Firearm and Toolmark Examiners, held in Chicago, IL on May 29th through June 3rd 2011.

The authors conducted this study to improve understanding of the accuracy, reliability, and measurement validity in the firearm and tool mark discipline of forensic science. Participants were firearms examiners throughout the United States. Some 160 test sets were distributed to laboratories in forty-six states and the District of Columbia.

The test sets were designed to determine an examiner‘s ability to correctly identify cartridge casings fired from 10 consecutively manufactured Ruger slides

14June 14, 2011 14

to test fired cartridge casings fired from the same slides. This empirical study established an error rate of less than 0.1 percent. Durability testing established that the Ruger slides maintained their individual characteristics after multiple firings.

Hamby, J., and Norris, S., ―The Examination, Evaluation and Identification of 9mm Caliber Cartridge Cases From 1275 GLOCK Semiautomatic Pistols Manufactured Over an 18 Year Period‖ Presented at the 42nd Annual Training Seminar of the Association of Firearm and Toolmark Examiners, held in Chicago, IL on May 29th through June 3rd 2011.

Six hundred and seventeen (617) GLOCK pistols, manufactured in 1993 were obtained, test fired and the fired cartridge cases examined, evaluated and identified to each other. This research was the subject of one chapter of a Ph.D. thesis by Hamby at the University of Strathclyde, Glasgow, Scotland.

Over the past 4 ½ years the authors have been examining fired GLOCK cases as part of the GLOCK Manufacturing Facilities Quality Assurance Program. Five hundred and seventy five (575) cases from the original study were combined with 700 fired cases from the current study and evaluated and identified against each other. These examinations, using conventional optical microscopy, and conventional pattern matching techniques resulted in being able to individualize the fired cases. Combining the fired cases – manufactured over an 18 year period – resulted in a total number of examinations totaling over 1,600,000.

15June 14, 2011 15

5. Have studies been conducted to review the level of similarity between marks produced by consecutively manufactured tools/firearms vs. randomly manufactured tools/firearms?

Stone, R. (2003) How Unique are Impressed Marks, Association of Firearm and Toolmark Examiners Journal, 35:4, pp. 376-383.

This study outlines several theoretical types of impressed toolmark characteristics (point, line, curve, enclosure and three-dimensional) and applies mathematical probability estimates in an attempt to quantify them. It was found that with marks of ―reasonable complexity‖ the odds of the same marks being repeated on another tool were astronomical.

Collins, E.R., (2005) How ―Unique‖ Are Impressed Toolmarks? – An Empirical Study of 20 Worn Hammer Faces, Association of Firearm and Toolmark Examiners Journal, 37:4, pp. 252-295.

This study utilizes 20 worn hammer faces to determine if Stone‘s (2003) theoretical types of toolmark characteristics model ―accurately and consistently represents the occurrence of individualizing effects.‖ This study includes an addendum by Stone which outlines refinements to his original model. The refinements to the original model continue to provide probabilities that are astronomical.

Hamby, J., and Thorpe, J., ―The Examination, Evaluation and Identification of 9mm Caliber Cartridge Cases From 617 Model 17 & 19 Semiautomatic Pistols‖, Association of Firearm and Toolmark Examiners Journal, Vol. 41, No. 4, Fall 2005, pp.310-324.

Six hundred and seventeen (617) GLOCK pistols, manufactured in 1993 were obtained, test fired and the fired cartridge cases examined, evaluated and identified to each other. This research was the subject of one chapter of a Ph.D. thesis by Hamby at the University of Strathclyde, Glasgow, Scotland.

Of the 617 pistols that were test fired, 550 had consecutive serial numbers which would indicate they were manufactured within a short time frame of each other. The others were in different serial number ranges. It was possible to individualize the fired cases to themselves and to the exclusion of the other fired cases.

Howitt D., Tulleners F., ―A Calculation of the Theoretical Significance of Matched Bullets‖, Journal of Forensic Sciences, Volume 53, Number 4, July 2008, Pp.868-875.

Study that calculated random occurrence probability for the correspondence of impression marks on a subject bullet to a random distribution of similar marks on a suspect bullet of the same type. These calculations produced values that supported previous reported empirical probabilities on consecutive matching

16June 14, 2011 16

bullet striae and also indicate that larger consecutive matching sequences are extremely unlikely to occur.

Neel M., and Wells M., ―A Comprehensive Statistical Analysis of Striated Tool Mark Examinations Part I: Comparing Known Matches and Known Non-Matches‖, AFTE Journal, Volume 39, (4), Summer 2007, pp. 176-198.

Study of 4000 striated toolmark comparisons concluded that known matches (KM) and known non-matches (KNM‘s) can be statistically distinguished from one another with 3D toolmarks containing a 1 in 802,919 and 2D toolmarks containing a 1 in 12,090,164 likelihood ratio.

May L., ―Identification of Knives, Tools and Instruments‖ Journal of Police Science Vol. 1, No. 3, 1930, pp. 247-248.

The author conducted a pioneering study on striated type toolmarks on numerous cutting tools, especially knives, with working edges containing some type of ground finish. Also, conducted first attempt at a statistical validation in Toolmark Identification; in which, it was calculated that the possibility of the same identifying mark(s) appearing on another tool is approximately 100,000 X 650 (quadrillion).

Brackett, J.W. ―A Study of Idealized Striated Marks and their Comparisons using Models.‖ Journal of the Forensic Science Society, Vol. 10, No. 1, January, 1970, pp. 27-56.

Comparison of various proposed probability models for striated marks, with an eye toward the development of an automated system. CMS model tended to support empirical work of Biasotti.

Deinet, Werner. ―Studies of Models of Striated Marks Generated by Random Processes.‖ Journal of Forensic Sciences, Vol. 26 (1), Jan., 1981, pp. 35-50.

Computer-aided studies of the degree of similarity of striated marks are described. Digitized image data on 40 grinding marks were fed into a minicomputer, and the position values of the lines were determined semiautomatically. Idealized models were defined for an objective comparison of striated marks and then applied to the grinding mark data. Necessary conditions of the models were tested by comparing them with actual, measured properties of the marks. Results of the model calculations are presented and the properties of the models discussed.

Stone, R., ―How Unique are Impressed Marks,‖ AFTE Journal, Vol. 35(4), Fall 2003, pp. 376-383.

17June 14, 2011 17

This study outlines several theoretical types of impressed toolmark characteristics (point, line, curve, enclosure and three-dimensional) and applies mathematical probability estimates in an attempt to quantify them. It was found that with marks of ―reasonable complexity‖ the odds of the same marks being repeated on another tool were astronomical.

18June 14, 2011 18

6. What studies review the degree of variability that exists in (a) bullet striations observable from the same firearm; (b) bullet striations from different firearms

Biasotti, A., (1981) Bullet Bearing Surface Composition and Rifling (Bore) Conditions as Variables in the Reproduction of Individual Characteristics on Fired Bullets, AFTE Journal , Vol.13, No. 2, pp. 94 – 102.

The purpose of the experiment described herein is to demonstrate the effects of several of the more significant variables that may contribute towards the reproducibility of identifiable individual characteristics on fired bullets. The author discusses individual characteristics via an examination of various types of bullets (Lubaloy, Golden, and Nyclad) and various conditions of the bore.

Biasotti, A.A., (1959) A Statistical Study of the Individual Characteristics of Fired Bullets, Journal of Forensic Science, 4:1, pp. 34-50

Hamby, J. PhD, et al. (2009) ―The Identification of Bullets Fired From 10 Consecutively

Rifled 9mm Ruger Pistol Barrels: A Research Project Involving 507 Participants from 20 Countries‖, AFTE Journal 41:2 pp. 99 - 110.

This lengthy undertaking demonstrated that a large number of examiners from numerous countries ranging from trainees to highly experienced examiners could correctly associate fired 9mm bullets with their parent barrels even though the barrels were consecutively rifled with the same broach. If inconclusive answers were included, an error rate of 0.059% was found. The comparisons considered the normal variability in striae patterns that examiners encounter in bullets fired from the same barrel as well as the degree of agreement in individual characteristics necessary to affect an identity of source. Numerous additional published studies for this same purpose and with the same result, i.e.- bullets from consecutively manufactured barrels can be correctly associated with the source barrel by trained examiners. These are listed below in order of appearance in the peer-reviewed literature. Additionally, many of these studies also addressed persistence of matching individual characteristics over repeated firings ranging from hundreds to thousands of rounds representing far more shots than are likely to occur between the use of a firearm in a crime and its subsequent recovery and submission to the crime laboratory.

Hall, E., (1983) Bullet Markings From Consecutively Rifled Shilen DGA Barrels, AFTE Journal, 15:1 pp. 33-47 Kirby, S., (1983) Comparison of 900 Consecutively Fired Bullets and Cartridge Cases From a .455 Caliber S&W Revolver, AFTE Journal, 15:3, pp. 113-126

Lutz, M., (1970) Consecutive Revolver Barrels, AFTE Newsletter #9, Page 24

19June 14, 2011 19

Matty, W., (1985) A Comparison of Three Individual Barrels Produced From One Button Rifled Barrel Blank, AFTE Journal,17:3, pp. 64-69 Miller, J., (2000) An Examination of Two Consecutively Rifled Barrels and A Review of the Literature, AFTE Journal, 32:3, pp. 259-270. Miller, J., (2001) An Examination of the Application of the Conservative Criteria for Identification of Striated Tool Marks Using Bullets Fired From Ten Consecutively Rifled Barrels, AFTE Journal, 32:2, pp. 125-132. Murdock, J., (1981) A General Discussion of Gun Barrel Individuality and an Empirical Assessment of the Individuality of Consecutively Button Rifled .22 Caliber Rifle Barrels, AFTE Journal, 13:3, pp. 84-111 Ogihara, Y., Kubota, M., Sanada, M., Fukuda, K., Uchiyama, T., and Hamby, J., (1983) Comparison of 5000 Consecutively Fired Bullets and Cartridge Cases From a .45 Caliber M1911A1 Pistol, AFTE Journal, 15:3 pp. 127-140

Roberge, D., Beauchamp, A., (2006) The Use of BulletTrax-3D in a Study of Consecutively Manufactured Barrels, AFTE Journal 38:2 pp. 166 – 172.

Forensic Technology challenged its newest 3D technology, BulletTRAX-3DTM, with a test provided by firearms examiner Evan Thompson of the Washington State Police Crime Laboratory. This test involves 21 pairs of bullets, among which 20 are fired from ten consecutively manufactured 9mm caliber Luger Hi-Point barrels. Each of the ten first pairs of bullets is connected to a distinct known barrel and is labeled from 1 to 10, the remaining 11 pairs being labeled from A to K. The purpose of this test is to correctly match each pair from the first set to a pair in the second set. The relation between both sets is given by a confidential key, which is a set of ten couples, the first element being a digit (1 to 10), the second a letter (in the A-K range). All pairs of bullets in the Thomson test were imaged with BulletTRAX-3DTM. From the correlation scores, the key was found by a process that can easily be automated by software.

Schecter, B., Silverwater, H., and Etzion, M., (1992) Extended Firing of a Galil Assault Rifle, AFTE Journal, 24:1, pp. 37-45 Shem, R. and Striupaitis, P., (1983) Comparison of 501 Consecutively Fired Bullets and Cartridge Cases From a .25 Caliber Raven Pistol, AFTE Journal, 15:3, pp. 109-112 Skolrood, R. (1975) Comparison of Bullets Fired From Consecutively Rifled Cooey .22 Calibre Barrels, Canadian Society of Forensic Science Journal, 8:2, pp. 49-52 Tulleners, F., Giusto, M., Hamiel, J., (1998) Striae Reproducibility of Sectional Cuts of One Thompson Contender Barrel, AFTE Journal, 30:1 pp. 62-81.

20June 14, 2011 20

Tulleners, F. and Hamiel, J., (1999) Sub-Class Characteristics of Sequentially Rifled .38 Special S&W Revolver Barrels, AFTE Journal, 31:2, pp. 117-122

Uchiyama, T., ―Toolmark Reproducibility on Fired Bullets and Expended Cartridge Cases‖, AFTE Journal, Vol. 40, No.1, 2008) pp. 3 – 46.

The reproducibility of landmarks, breechface marks and firing pin marks on one hundred successively fired bullets and cartridge cases was examined. Three types of Speer brand, one of Remington brand and one of DFA brand frangible cartridges were fired in a semi-automatic pistol. Remarkable differences were observed in the general appearance of the landmarks, breechface marks and firing pin marks which were impressed on the different brands of cartridges, even when consecutively fired. Identification of the landmarks between bullets from different brands of cartridges was difficult because their general appearance differed greatly. Difference in bullet diameters was found to be a major cause of changes in landmarks among different manufacturer's bullets. Although the depth and number of striations decreased gradually, reproducibility of breechface marks on the primers of cartridges was rather good. The diameter of firing pin indentations also differed among different brands of cartridges. Although the reproducibility of the diameter of circular lines on firing pin indentations was good, the detail in these circular lines fluctuated a great deal. Quantitative CMS was used as a means of critically evaluating and communicating the extent of striated pattern agreement among the rifling impressions on the fired bullets in this study.

21June 14, 2011 21

7. Do studies exist which examine the wear rates of materials used to manufacture tools/firearms/bullets and cartridge casings and the factors that affect wear?

General comments: The various studies related to the rifling of gun barrels, the visible changes along the interior surface of a rifled gun barrel (largely, but not exclusively due to galling and/or chip formation during the rifling process) are well known to forensic firearms examiners. It is these very reasons that bullets from new, consecutively rifled barrels can usually be discriminated and associated with the parent barrel. This is especially true when the toolmarks from the original drilling process used to form the bore of a gun barrel survive the subsequent steps involved in the rifling process. This situation is apparent to the forensic firearms examiner upon inspection of the bore of a submitted firearm. The random changes that occur over time with use and abuse only stand to enhance the differences between consecutively manufactured barrels. This is also true for other consecutively manufactured tools. These same reasons and phenomena typically apply to the manufacture of the breech faces and breech blocks in firearms with these areas being largely responsible for the subsequent markings imprinted in the heads and primers of fired cartridge cases.

The previously cited studies directed exclusively to bullets fired from consecutively manufactured gun barrels and listed in response to Question 6 apply here and will not be repeated. Some studies that are relevant to tools other than gun barrels are listed below. The classic textbooks on firearms identification address the sources of class and individual characteristics to varying degrees. These texts are listed separately at the end of this response.

Burd, D. and Kirk, P., (1942) Tool Marks: Factors Involved in Their Comparison and Use as Evidence, Journal of Police Science, 32:6 pp. 679-686 Burd, D. and Gilmore, A., (1968) Individual and Class Characteristics of Tools, Journal of Forensic Science, 13:3 pp. 390-396 Coffman, B., (2003) Computer Numerical Control (CNC) Production Tooling and Repeatable Characteristics on Ten Remington Model 870 Production Run Breech Bolts, AFTE Journal, 35:1 pp. 49-54 Haag, L., (2007) The Matching of Cast Bullets to the Moulds that Made Them, AFTE

Journal, 39:4 pp. 313-322

This article dealt with consecutively machined bullet moulds, subclass (carryover marks) and individual characteristics produced by galling and chip formation during the machining process. All of these machining marks are reproduced over the entire surface of lead and lead alloy bullets cast in bullet moulds. Those areas and sites that were the consequence of galling and chip formation were obvious upon low power inspection of the surfaces of cast bullets and

22June 14, 2011 22

represented the individual characteristics that allowed consecutively manufactured moulds to be discriminated.

Hu, J., Chou, K., (2007) Characterizations of cutting tool flank wear-land contact, Wear, 263 pp. 1454-1458.

Koshy, P., Dewes, R.C., Aspinwall, D.K., (2002) High speed end milling of hardened ASI D2 tool steel, Journal of Materials Processing Technology, 127 pp. 266-273.

This article explores the factors of the machining process that effect tool wear and the resulting surface roughness. Various cutting speeds and tool coatings were used to evaluate the amount of tool wear and resulting surface roughness. The result of this research was wear of a cutting tool regardless of the combination of tool speed and tool coating. The tool wear resulted in a varying degree of surface roughness (individual characteristics) on all samples.

Lopez, L. and Grew, S., (2000) Consecutively Machined Ruger Bolt Faces, AFTE Journal, 32:1 pp. 19-24 Matty, W., (1984) Raven .25 Automatic Pistol Breech Face Tool Marks, AFTE Journal, 16:3 pp. 57-60 Miller, J., (2001) An Introduction to the Forensic Examination of Tool Marks, AFTE Journal, 33:3 pp. 233-248 Rosati C., (2000) Examination of Four Consecutively Manufactured Bunter Tools, AFTE Journal, 32:1 pp. 49-50 Classic articles and textbooks dealing with the formation and deposition of class and individual characteristics by firearms and various other tools on contacting surfaces (listed in order of publication).

Mezger O, Heess W, Hasslacher F, (1931) Die Bestimmung des Pistolensystems aus Verfeuerten Hülsen und Geschossen, Archive für Krimologie 89 pp.1-32 und 93-116, Verlag von F.O.W. Vogel, Berlin

Mezger, 0., Heess, W., Hasslacher, F., (1931) Determination of the Type of Pistol Employed from an Examination of Fired Bullets and Shells," Am. J. Police Science 2:6 pp. 473-500 (1932) 3:2 pp. 124-146

Burrard, G., (1934) The Identification of Firearms and Forensic Ballistics, Herbert Jenkins, Ltd., London, England

Gunther, J., (1935) The Identification of Firearms, John Wiley, New York

23June 14, 2011 23

Hatcher, J., (1935) Textbook of Firearms Investigation, Identification and Evidence,

Small-Arms Technical Publishing Co., Plantersville, SC

Hatcher, J., Jury, F. and Weller, J., (1957) Firearms Investigation, Identification and

Evidence, Stackpole Publishing, Harrisburg, PA

O‘Hara and Osterberg, J., (1949) An Introduction to Criminalistics, The Macmillian Co., New York

Kirk, P., (1953) Crime Investigation, John Wiley & Sons, New York, London, Sydney, Toronto

Davis, J., (1958) An Introduction to Tool Marks, Firearms and the Striagraph, Charles C. Thomas Publishing Co., Springfield, IL

Mathews, J., (1962) Firearms Identification-3 Volumes, Charles C. Thomas Publishing Co., Springfield, IL

Saferstein, R., (1977) Criminalistics: An Introduction to Forensic Science, Prentice-Hall, Inc., Englewood Cliffs, NJ

De Forest, P. Gaensslen, R and Lee, H., (1983) Forensic Science-An Introduction to

Criminalistics, McGraw-Hill Publishing Co.

Warlow, T., (1996) Firearms, the Law and Forensic Ballistics, Taylor & Francis, London, England & Bristol, PA

24June 14, 2011 24

8. What research exists that examines the durability of firearms/manufacture tools (screwdrivers, pry bars, hammers, etc.) as a result of wear/tear, care and abuse in relation to conservation of markings and their effects on identification?

The numerous, previously-cited persistence studies involving hundreds to thousands of rounds fired through a variety of firearms also address this question and will not be re-published here. However, the events associated with the discharge of firearms involve heat and pressures seldom, if ever, will prevent individual characteristics from being deposited on fired cartridge cases and engraved on fired bullets.

Bacharach, B. (2009) Statistical Validation on the Individuality of Tool Marks Due to the Effect of Wear, Environment Exposure and Partial Evidence‖, NIJ/NCJRS Document #227929.

An objective, quantifiable toolmark study on marks imparted onto wires by diagonal cutters. This study examined the effects of wear, environmental conditions and partial toolmark impressions by an automated 3-D system that mathematically correlated results of toolmarks to the tools that produced them. This study validated, and thus strengthened, the foundations of Toolmark Identification.

Gouwe J., Hamby J.E., Norris, S. (2008). Comparison of 10,000 Consecutively Fired Cartridge Cases from a Model 22 Glock .40 S&W Caliber Semiautomatic Pistol, AFTE Journal, 40:1, pp. 57-63.


Kirby, S. ―Comparison of 900 Consecutively Fired Bullets and Cartridge Cases from a .455 Caliber S&W Revolver‖, AFTE Journal, Vol. 33. No. 3, Summer 2001, pp. 113-125.

Durability study of major working edges/surfaces of a revolver.

Miller, J. (1998) Cut Nail Manufacturing and Toolmark Identification, AFTE Journal, 30:3 pp. 492-498 Miller, J. (1998) Reproducibility of Impressed and Striated Tool Marks: 4d Cut Flooring Nails, AFTE Journal, 30:4 pp. 631-638

These two articles are especially useful in responding to question 8 in that the equipment (tools) used to manufacture cut nails produced both impressed and

25June 14, 2011 25

striated marks on steel nails that were found to be reproducible and identifiable over lengthy production runs involving tens of thousands of cut nails. The ―tools‖ consisted of shearing machines, cutters and header punches. Although some changes in these toolmarks were observed by the author, nails manufactured after 32,000+ specimens could still be matched to initial samples. Toolmarks on other sites on these nails changed more rapidly but front and end samples ranging from 10,000 to 15,000 units could still be matched on the basis of impressed and striated marks produced during the manufacturing processes. To quote the author, “Any change in toolmarks left by the cutter or header is so gradual as to allow an identification of toolmarks over a long period of production.”

Miller, J. (2001) An Introduction to the Forensic Examination of Tool Marks, AFTE Journal, 33:3 pp. 233-248

This well-illustrated article with 75 references provides an excellent primer on the subject of toolmarks not produced by firearms. It also addresses the sources of individuality in a wide variety of tool types and how they can be imparted to various receiving surfaces. Finally, and most germane to this question, the author specifically comments on the effects of wear and usage and the persistence of a tool‘s individuality [page 244 ―Tool Wear‖]

Ogihara, Y., et al, ―Comparison of 5000 Consecutively Fired Bullets and Cartridge Cases From a 45 Caliber M1911 Pistol‖, AFTE Journal, Vol. 15, No. 3, July 1983, pp. 127-140.

Durability study of major working edges/surfaces of a pistol.

26June 14, 2011 26

9. What literature exists that describes the current state and scope of databases related to firearms/bullets/cartridge casings? Have analyses been conducted which define the gaps related to databases in firearms and toolmark research?

Barrett, M., Tajbakhsh, A., Warren, G. (2011) Portable Forensic Ballistics Examination Instrument: Advanced Ballistics Analysis System (ALIAS), Association of Firearm and Toolmark Examiners Journal 43:1 pp. 74-78.

A portable measurement instrument and analysis tool for use by forensic ballistics and firearms examiners that creates, compares and analyses three-dimensional, volumetric models of fired cartridge cases and spent bullets. The technology can measure and examine toolmarks as small as two microns. ALIAS includes computer hardware, an open database infrastructure, a high-precision, Swiss-built, application-specific interferometer with a ―six-pac‖ cartridge case or expended bullet holder (patents pending) and an open software architecture.

De Kinder, J., Tulleners, F., Thiebaut, H. (2004), Reference Ballistic Imaging Database Performance, Forensic Science International, 140, pp. 207-215

These publications point out the limitations of image storage and search systems for fired cartridge cases at the time of these writings. The CALDOJ publication lists 9 issues and areas of concern in attempting to apply this technology to large databases of newly manufactured semi-automatic handguns. The combined American-European study by De Kinder, et al. involved 4200 fired cartridge cases from 6 brands of ammunition discharged in 600 model P226 SIG-Sauer 9mm pistols. The authors realized a 72% success rate when the same brand of cartridge case was scanned and entered into the IBIS system with success defined as the correct match appearing in the top 10 candidates. This fell to 21% when different brands of ammunition were scanned and entered in the system.

George, W., (2004) The Validation of the Brasscatcher Portion of the NIBIN/IBIS System Part Two: Fingerprinting Firearms Reality or Fantasy, Association of Firearm and Toolmark Examiners Journal 36:4 pp. 289 – 296.

A study of the Brasscatcher portion of the NIBIN/IBIS system was conducted using a database of 850 cartridge cases fired in Smith & Wesson, .40 S&W caliber pistols. Correlations were generated for entries from Federal, Winchester and Remington brand ammunition and a study to locate the placement of matching cartridge cases initiated. Forensic Technology was able to open the entire database for viewing instead of the normal user field of 20 %. This study provided a real test of the ability of Brasscatcher to identify cartridge cases fired from similar firearms, and addresses the concept of fingerprinting firearms for use in criminal investigations. During this study an additional advantage regarding the second breech face impression image was revealed. The second image is not used for correlation purposes.

27June 14, 2011 27

Nennstiel R., Rahm J. (2006) A Parameter Study Regarding the IBIS Correlator; J

Forensic Sci.,51:1 pp. 18-23

Nennstiel R., Rahm J., (2006) An Experience Report Regarding the Performance of the IBIS Correlator, J Forensic Sci., 51:1, pp. 24-30

The articles by Nennstiel and Rahm provide success rate values for the IBIS technology for bullet and cartridge cases searches. These authors also found that as the database increases the system‘s success rate decreases.

Tulleners, F., (October 5, 2001) Technical Evaluation: Feasibility of a Ballistics Imaging Database for all New Handgun Sales. CALDOJ Publication (a peer reviewed report).

28June 14, 2011 28

10. What studies, if any, have been designed to attempt to falsify the idea that a specific tool produced a specific mark to the practical exclusion of all others?

The numerous studies of consecutively manufactured tools (gun barrels, semi-automatic pistol slides and breech faces, firing pins, knifes, chisels, screwdrivers, etc.) all address this very issue. A number of these have been previously listed in response to Questions 6, 7 and 8 and will not be republished here. It should not be forgotten or overlooked that the final identification determination in any firearm or toolmark comparison does not include the entire universe of all firearms or tools in existence as the conclusion of Question 10 would suggest. Rather the objective and easily measureable Class Characteristics (caliber and cartridge designation, land and groove count, land and groove widths, direction of twist for firearms; tip shape and width for prying tools, the shape and diameter of the face of a hammer, etc. for tools) that are determined first greatly narrow the potential sources of a fired bullet, a fired cartridge case, a prymark from a crowbar or an impact mark from a hammer, etc.

FIREARM BIBLIOGRAPHY

Bachrach, Ben. ―Development of a 3D-Based Automated Firearms Evidence Comparison System.‖ Journal of Forensic Sciences, vol. 47 (6), November, 2002, pp. 1253-1264.

This study reports on a computerized system that calculates correlation coefficients for comparisons of bullet striation patterns using generated 3-D maps of bullet surfaces. Was validated using known matches (KMs) and known non-matches (KNMs), so therefore the system arrives at a conclusion of identification (or not), with an associated probability of error. Highly relevant to our work, because it shows conclusively that an objective observer (a machine) detects significant visual differences between KNMs and KMs.

Biasotti, Alfred A. ―A Statistical Study of the Individual Characteristics of Fired Bullets.‖ Journal of Forensic Sciences, vol. 4(1), January, 1959, pp. 34-50.

Validity study in which no more than three consecutively matching striations (CMS) were found on lead bullets fired from different guns and no more than four CMS were found on jacketed bullets fired from different guns.

Brown, C. and W. Bryant. ―Consecutively Rifled Gun Barrels Present in Most Crime Labs.‖ AFTE Journal, vol. 27 (3), July, 1995, pp. 254-258.

Study of multi-barreled derringers in which it was assumed that barrels were rifled consecutively. One set of derringer test fires showed some good

29June 14, 2011 29

correspondence in the groove impressions (gross marks), but showed little correspondence in the land impressions.

Brundage, David J. ―The Identification of Consecutively Rifled Gun Barrels.‖ AFTE Journal, vol. 30(3), Summer, 1998, pp. 438-444.

Validation study in which ten consecutively broach rifled pistol barrels produced by Ruger were used to used to test the fundamental claim that qualified examiners will rarely, if ever, commit false identifications or false eliminations. Thirty examiners were given the test nationwide and no misidentifications were made.

Bunch, Stephen G. ―Consecutive Matching Striation Criteria: A General Critique.‖ Journal of Forensic Sciences, vol. 45 (5), Sept. 2000, pp. 955-962.

This paper critiques the Quantitative Consecutive Matching Striation (CMS) approach to toolmark identification. The author discusses the practical and theoretical weaknesses of the approach, argues that it demands a statistical/probabilistic treatment of results - such as the use of Bayesian likelihood ratios - and also suggests much additional research is needed.

DeFrance, Charles S. and Michael VanArsdale. ―Validation Study of Electrochemical Rifling.‖ AFTE Journal, vol. 35 (1), Winter, 2003, pp. 35-37.

Validation study in which nine examiners participated in the comparison of bullets from electrochemically rifled barrels produced by Smith & Wesson. No misidentifications were made.

Fadul, T.G., ―An Empirical Study to Evaluate the Repeatability and Uniqueness of Striations/Impressions Imparted on Consecutively Manufactured Glock EBIS Gun Barrels‖, AFTE Journal, Volume 43, Number 1, Winter 2011, Pp. 37-44.

An empirical study of ten consecutively manufactured Glock barrels containing the Enhanced Bullet Identification System (EBIS). Study consisted of test sets sent to 238 examiners from 150 laboratories in 44 states and 9 countries that were designed to test the examiner‘s ability to correctly identify fired bullets to the barrel that fired them. The results from 183 of these examiners produced an error rate of 0.4%. This study validated the repeatability and uniqueness of striated markings in gun barrels, as well as the ability of a competent examiner to reliably identify fired bullets to the barrels that marked them.

Freeman, Ray A., ―Consecutively Rifled Polygon Barrels,‖ AFTE Journal, vol.10 (2), June 978, pp.40-42.

This study documents the comparison of bullets fired through three consecutively manufactured polygon barrels produced by H&K for the Model P9S pistol. It was

30June 14, 2011 30

found that the bullets fired from these barrels could easily be identified to the correct barrel. Additionally, these barrels possessed a fluted chamber. Marks from the fluted chambers were visible on the bullets and could also be used for identification.

Hall, E. ―Bullet Markings from Consecutively Rifled Shilen DGA Barrels.‖ AFTE Journal, vol. 15(1), Jan., 1983, pp. 33-53.

Study of consecutively button rifled polygonal style barrels. Conclusion implies that there should be no risk of misidentification.

Hamby J. E., Brundage D. J. , Thorpe J. W., ―The Identification of Bullets Fired from 10 Consecutively Rifled 9mm Ruger Pistol Barrels: A Research Project Involving 507 Participants from 20 Countries‖, AFTE Journal, Volume 41, Number 2, Spring 2009, pp. 99-110.

Bullets fired from ten (10) consecutively manufactured barrels were correctly identified to the respective barrel that fired them by five hundred-seven (507) firearm examiners from twenty (20) countries. This study validates the underlying theory that: 1) there are identifiable features imparted by a gun on the surfaces of fired bullets that 2) enable a competent firearms examiner to accurately and reliably link them to the barrel that fired them.

Intelligent Automation, Incorporated, ―A Statistical Validation of the Individuality of Guns Using High Resolution Topographical Images of Bullets‖, National Institute of Justice Grant #2006-DN-BX-K030, October, 2010

Study of marks on fired bullets by a topography based (3D) automated system. This study continued the analysis of a previous 2005 NIJ bullet study and validated the original premise of Firearm/Toolmark ID. This study also concluded that 1) the ability to determine that a given bullet was fired from a specific barrel depends on the individual barrel itself and not only on the brand of its manufacture, and 2) the performance of the automated analysis system used in this study is not representative of that of a trained firearms examiner as humans have a remarkable ability to perform pattern matching that is difficult to be replicated in any automated system.

Lomoro, Vincent J. ―Class Characteristics of 32 SWL, FIE Titanic Revolvers.‖ AFTE Journal, vol. 6 (2), 1974, pp. 18-21.

This paper points out the pitfalls of basing an identification on groove impressions on bullets fired from F.I.E. Titanic Revolvers. Bullets from three different guns were shown to have agreement in the groove impressions, but were found to differ significantly in the land impressions.

Lutz, M. ―Consecutive Revolver Barrels.‖ AFTE Newsletter #9, Aug., 1970, pp.24-28.

31June 14, 2011 31

http://www.swggun.org/resources/docs/232136.pdf


Reported results of the comparison of jacketed and lead bullets fired from two consecutively rifled barrels and that the markings on the bullets were identifiable and unique to the barrel that fired them.

Matty, William. ―A Comparison of Three Individual Barrels Produced from One Button-Rifled Barrel Blank.‖ AFTE Journal, vol. 17(3), July, 1985, pp. 64-69.

Study of the uniqueness of marks produced on bullets fired from three barrels that were produced from the same rifled barrel blank. Subclass characteristics noted in the groove impressions, but not in the land impressions. Study also notes that over the first few firings the striations on the bullets change significantly.

Miller, Jerry. ―An Examination of Two Consecutively Rifled Barrels and a Review of the Literature.‖ AFTE Journal, vol. 32 (3), Summer, 2000, pp.259-270.

Study in which bullets were pushed through two consecutively broached.44 caliber barrels and were examined using Biasotti/Murdock conservative quantitative CMS (QCMS) criteria for identifications. No misidentifications.

Miller, Jerry. ―Criteria for Identification of Toolmarks, Part II: Single Land Impression Comparisons.‖ AFTE Journal, vol. 32 (2), Spring, 2000, pp. 116-131.

This study compares bullets fired by Raven 25 Auto, Lorcin 380 Auto, and Stallard Arms 9mm pistols to specimens in the NIBIN database. This study supports the Biasotti/Murdock QCMS criteria.

Miller, Jerry. ―An Examination of the Application of the Conservative Criteria for Identification of Striated Toolmarks Using Bullets Fired from Ten Consecutively Rifled Barrels.‖ AFTE Journal, vol. 33 (2), Spring, 2001, pp. 125-132.

Using the bullets from the Brundage Ruger ten barrel test, the author: 1) identified some very minor subclass characteristics but not sufficient to cause a misidentification, and; 2) applied the conservative quantitative CMS criteria which resulted in no misidentifications.

Miller, Jerry and Michael McLean. ―Criteria for Identification of Toolmarks.‖ AFTE Journal, vol. 30 (1), 1998, pp.15-61.

Using IBIS, the authors compared land impressions of .38 Special jacketed bullets fired from S&W revolvers. Found no CMS counts greater than six (6) for KNMs, using the computer monitor. Using a separate set of test fires and the comparison microscope, no CMS counts greater than four (4) for KNMs were found.

32June 14, 2011 32

Murdock, John E. ―A General Discussion of Gun Barrel Individuality and an Empirical Assessment of the Individuality of Consecutively Button Rifled .22 Caliber Rifle Barrels.‖ AFTE Journal, vol. 13 (3), 1981, pp. 84-95.

This study discusses rifling methods, including the ―new‖ method of button rifling. Examination of nine barrels (three consecutively rifled barrels from three manufacturers) and test fired bullets from each indicated no subclass characteristics. First two bullets fired from each barrel could not be identified to each other which is indicative of rapid change in a new barrel‘s interior, which in turn confirms individuality of barrels.

Skolrood, R. W. ―Comparison of Bullets fired from Consecutively Rifled Cooey .22 calibre Barrels.‖Canadian Society of Forensic Science, vol. 8(2), 1975, pp. 49-52.

This paper discusses the potential for broaches to produce reproducible gross marks and that examiners should be wary of these gross marks.

Smith, Erich. ―Cartridge Case and Bullet Comparison Validation Study with Firearms Submitted in Casework.‖ AFTE Journal, vol. 37 (2), Spring 2005, pp. 130-135.

This validation study was designed to test the accuracy of examinations by trained firearms examiners who use pattern recognition only (no CMS tabulation) as a method for identification. Eight FBI examiners took the test which consisted of both bullets and cartridge cases. No false positives or false negatives were reported.

Tulleners, Fred and Mike Giusto. ―Striae Reproducibility on Sectional Cuts of One Thompson Contender Barrel.‖ AFTE Journal, vol. 30(1), 1998, pp. 62-81.

For this study, a Thompson Center Contender button rifled barrel was sectioned one inch at a time after each test firing. A total of six sections were removed from the barrel. Each sections bullets were compared to each other to see how much the CMS count had changed. Striae on the bullets were found to be significantly altered from one barrel section to the next. The results obtained from adjacent barrel sections were apparently comparable to the results Biasotti obtained from different, uncut barrels. It was also significant that while total line counts differed, the critical observation of CMS generally not exceeding three for known non-matching regimes was very consistent between examiners.

Tulleners, Fred and James Hamiel. ―Sub Class Characteristics of Sequentially Rifled .38 Special S&W Revolver Barrels.‖ AFTE Journal, vol. 31 (2), 1999, pp. 117-222.

This article discusses the potential for sub-class characteristics in S&W revolver barrels. The article points out that examiners should be careful when examining the groove impressions on fired bullets from broach rifled barrels.

33June 14, 2011 33

Bunch, Stephen G. and Douglas P. Murphy. ―A Comprehensive Validity Study for the Forensic Examination of Cartridge Cases.‖ AFTE Journal, vol. 35 (2), Spring 2003, pp. 201-203.

This validity study used 10 consecutively manufactured Glock slides to test the proposition that qualified examiners rarely or never commit false positive or false negative errors in cartridge case exams. FBI examiners participated in this blind study. False positive and false negative rates were 0%.

Coffman, B.C., ― Computer Numerical Control (CNC) Production Tooling and Repeatable Characteristics on Ten Remington Model 870 Production Run Breech Bolts‖, AFTE Journal, Volume 35, Number 1, Winter 2003, pp. 49-54.

Ten shotgun bolt faces, consecutively produced by the same CNC manufacturing machine tool, were examined and compared for the presence of subclass and individual characteristics. Results of these comparisons found that the manufacturing process used to fabricate these bolts produced both subclass characteristics and sufficient individual characteristics to provide uniqueness.

Coody, A.C., ―Consecutively Manufactured Ruger P-89 Slides‖, AFTE Journal, Volume 35, Number 2, Spring 2003, pp. 157-160.

Ten consecutively produced pistol slide breechfaces were examined and compared for the presence of subclass and individual characteristics. Results of these comparisons found that the manufacturing processes used to fabricate these breechfaces produced both subclass characteristics and sufficient individual characteristics to provide uniqueness.

Gouwe J., Hamby J.E., Norris, S., ―Comparison of 10,000 Consecutively Fired Cartridge Cases from a Model 22 Glock .40 S&W Caliber Semiautomatic Pistol‖, AFTE Journal, Volume 40, Number 1, Winter 2008, pp. 57-63.


Grooss, Klaus Dieter. ―The ‗Hammer-Murderer.‘‖ AFTE Journal, vol. 27 (1), 1995, pp. 27-30.

An actual murder case in Germany that in effect amounted to a blind test of both examiner skill and theoretical validity for cartridge case comparisons. A police officer was suspected of murder, but the lack of clues led to all Walther P5 pistols issued to police in Germany being test fired and compared to the evidence cartridge cases at the BKA lab. An identification occurred with a test-fired cartridge

34June 14, 2011 34

case from the 3704th pistol. Almost simultaneous events elsewhere proved this conclusion to be accurate. No false identifications occurred.

Hamby J., and Thorpe J., ―The Examination, Evaluation and Identification of 9mm Cartridge Cases Fired from 617 Different GLOCK Model 17 & 10 Semiautomatic Pistols‖, AFTE Journal, Volume 41(4), Fall 2009, Pp. 310-324.

A study of cartridge cases fired from 617 different Glock pistols was conducted utilizing conventional comparative optical microscopy and electronic imaging technology to test the premise of individualization in FA/TM ID. Results of this study validated not only the premise of individualization but also the hypothetical proposition that a competent firearm and toolmark examiner can correctly identify the firearm that fired an ammunition component without committing a misidentification.

Kennington, Robert. ―Identification of Cartridge Cases Fired in Different Firearms: ‗Pre-Identified Cartridges.‘‖ AFTE Journal, vol. 31(1), 1999, pp. 15-19.

This research discusses the pitfall that toolmarks produced during the manufacturing process of ammunition components pose and that one should be mindful that these marks exist.

Lardizabal, P. ―Cartridge Case Study of the HK USP.‖ AFTE Journal, vol. 27 (1), Jan., 1995, pp. 49-51.

This study examined two consecutively manufactured H&K 40 S&W caliber USP breechfaces. Subclass characteristics were identified on the breechface impressions. Test fired bullets from three H&K barrels were also examined and little correspondence was found between signatures from bullets fired from different barrels.

Lopez, Laura and Sally Grew. ―Consecutively Machined Ruger Bolt Faces.‖ AFTE Journal, vol. 32 (1), 2000, pp. 19-24.

This study warns that one should be careful with microscopic marks from a boltface machined with an end mill. Misidentification is possible unless the identification is based on wear or individual machining ―chatter‖ marks.

Lyons, D. J., ―The Identification of Consecutively Manufactured Extractors‖, AFTE Journal, Volume 41, Number 3, Summer, 2009, pp.246-256.

Study conducted on ten consecutively manufactured firearm extractors. Firearm and toolmark examiners from different laboratories were given ten sets of cartridge cases marked by these extractors to attempt to make the correct associations between the known and unknown cases. Each examiner also received twelve unknown marked cases in addition to the standards for the ten

35June 14, 2011 35

consecutively manufactured extractors, with each known specimen having at least one unknown specimen associated with it. Study showed that extractors could be distinguished from each other despite the fact they were consecutively manufactured and had subclass toolmarks on some surfaces.

Matty, William. ―Raven .25 Automatic Pistol Breech Face Tool Marks.‖ AFTE Journal, vol. 16 (3), 1984, pp. 57-60.

For this study, three consecutively made breechfaces from Raven pistols were compared. The concentric toolmarks on the breechfaces were found to be individual and not subclass.

Matty, William and Torrey Johnson. ―A Comparison of Manufacturing Marks on Smith & Wesson Firing Pins.‖ Journal of AFTE, vol. 16 (3), 1984, pp. 51-56.

This study examined the concentric marks produced by Smith & Wesson firing pins. Subclass characteristics were found. These subclass marks are a result of the lathe mounted cutter being much harder than the firing pins and thus marks can be reproduced; however, using the areas of the firing pins that show wear can be used for identification.

Rosati, Carlo. ―Examination of Four Consecutively Manufactured Bunter Tools.‖ AFTE Journal, vol. 32 (1), 2000, pp. 49-50.

For this study, four bunters produced by Electrical Discharge Machining (EDM) used by Remington for .45 Auto cartridge case manufacture were used to determine if this process was random in nature. Confirms random nature of marks from EDM process on headstamp characters.

Saribey, A. Y., Hannam A. G., Tarimci C., ―An Investigation into Whether or Not the Class and Individual Characteristics of Five Turkish Manufactured Pistols Change During Extensive Firing‖, Journal of Forensic Sciences, Volume 54, Number (5), September 2009, pp.1068-1072.

Conducted statistical durability study of fired cartridge cases from five different pistols. Each pistol had at least 1000 cartridge cases fired in them with every 250th case compared to the first fired case. Although there were noted changes in individual and some class characteristics, these wear changes were not statistically significant based on standard deviation measurements. This study statistically validated previous durability studies.

Thompson, Evan. ―Phoenix Arms (Raven) Breechface Toolmarks.‖ AFTE Journal, vol. 26 (2), 1994, pp. 134-135.

This is a follow-up study of the Matty article on Raven breechfaces. Four breechfaces from Phoenix pistols (formerly Raven) were compared to determine

36June 14, 2011 36

the nature of their marks. As in the Matty article, the breechfaces were found to possess unique identifying marks.

Thompson, Evan. ―False Breechface ID‘s.‖ AFTE Journal, vol. 28 (2), April, 1996, pp. 95-96.

This study examines the manufacturing process of Lorcin pistol breechfaces. Of noteworthiness is the fact that Lorcin breechfaces are stamped and then painted over, not machined. False identifications could be possible if the only marks considered are from the breechface. Also noted was the fact that paint on breechfaces has a tendency to chip off and that one should not solely rely on the breechface impression as a means for identification.

Uchiyama, T. ―Similarity among Breech Face Marks Fired from Guns with Close Serial Numbers.‖ AFTE Journal, Vol. 18, No. 3, 1986, pp. 15-52.

This study examined the breechface marks produced by Browning Baby, Raven P-25 and Titan pistols. Subclass characteristic were found to be significant on the breechface of each of these pistol models and examiners should use caution when encountering them.

TOOLMARK BIBLIOGRAPHY

Chumbly, L. Scott, et al, ―Validation of Tool Mark Comparisons Obtained Using a Quantitative, Comparative, Statistical Algorithm‖ Journal of Forensic Sciences, Volume 55, Number 4, July 2010, Pp. 953-961.


Bachrach B., Jain A., Jung S., and Koons R.D., ―A Statistical Validation of the Individuality and Repeatability of Striate Tool Marks: Screwdrivers and Tongue and Groove Pliers‖, Journal of Forensic Sciences, Volume 55, Number 2, March 2010, pp 348-357.

Study that statistically validated the original premise of individuality in Toolmark Identification by analyzing statistical distributions of similar values resulting from the comparison of Known Matches (KM) and Known Non-Matched (KNM) pairs of striated toolmarks. This quantifiable analysis of KM and KNM toolmark similarity distributions showed nearly error-free identifications.

37June 14, 2011 37

Bacharach, B., ―Statistical Validation on the Individuality of Tool Marks Due to the Effect of Wear, Environment Exposure and Partial Evidence‖, NIJ/NCJRS Document #227929, August, 2009.

An objective, quantifiable toolmark study on marks imparted onto wires by diagonal cutters. This study examined the effects of wear, environmental conditions and partial toolmark impressions by an automated 3-D system that mathematically correlated results of toolmarks to the tools that produced them. This study validated, and thus strengthened, the foundations of Toolmark Identification.

Burd, David Q. and Allen E. Gilmore. ―Individual and Class Characteristics of Tools.‖ Journal of Forensic Sciences, Vol. 13 (3), July, 1968, pp. 390-396.

This article discusses tools made from molds, such as die stamps and die forgings, and the possibility of confusing class marks as individual marks.

Butcher, S. and D. Pugh. ―A Study of Marks made by Bolt Cutters.‖ Journal of the Forensic Science Society, Vol. 15 (2), April 1975, pp. 115-126.

This study examines test marks made by ten consecutively made bolt cutters and ten randomly selected bolt cutters with ground working surfaces. The study determined that no more than 29% matching stria for known non-matches and between 87% and 93% matching stria for known matches. Implication: no risk of misidentification.

Cassidy, F. ―Examination of Toolmarks from Sequentially Manufactured Tongue and Groove Pliers.‖ Journal of Forensic Sciences, vol. 25 (4), Oct., 1980, pp. 796-809.

This study examines the individuality of striated marks produced by consecutively broach cut tongue and groove pliers. Examination of the jaw teeth and their test marks revealed no subclass marks and that the striated marks produced are individual to the tool that made them.

Clow, Charles M. ―Cartilage Stabbing with Consecutively Manufactured Knives: A Response to Ramirez v. State of Florida.‖ AFTE Journal, vol. 37 (2), Spring, 2005, pp. 86-116.

This study utilized ten consecutively manufactured knives used in a stabbing motion to determine if the marks produced were unique and if marks were reproducible and identifiable in pig cartilage. Marks were found to be unique. Marks reproduced and were found to be potentially identifiable in cartilage.

Eckerman, Stephanie J. ―A Study of Consecutively Manufactured Chisels.‖ AFTE Journal, vol. 34 (4), Fall 2002, pp. 379-390.

38June 14, 2011 38

In this study, consecutively belt sanded chisels were examined for the possibility of subclass marks. The marks were found to be individual to each chisel.

Flynn, Emmett M. ―Toolmark Identification.‖ Journal of Forensic Sciences, vol. 2 (1), Jan., 1957, pp. 95-106.

In this study, Chicago Police Crime Lab examined 100 consecutively made chisels finished with a grinding process. 5050 total comparisons made. No misidentifications.

Giroux B. N., ―Empirical and Validity Study: Consecutively Manufactured Screwdrivers‖, AFTE Journal, Volume 41, Number 2, Spring 2009, pp. 153-158.

The fundamental propositions of Toolmark Identification were tested with an empirical and validation study of five consecutively manufactured screwdrivers. The empirical study compared the machining marks imparted on the working surfaces of these screwdrivers to toolmark specimens produced by these screwdrivers. Eight qualified examiners at the FBI Laboratory participated in a blind validation study where eighty comparisons were conducted on the toolmarks produced by these screwdrivers. The results of this blind validity study resulted in no mis-identifications and one mis-elimination.

Hall, J. ―Consecutive cuts made by bolt cutters and their effect on identification.‖ AFTE Journal, vol. 24 (3), July, 1992, pp. 260-272.

This study showed that consecutive cuts in lead with bolt cutters are identifiable, showing that lead is a suitable material for test marks. Cuts in shackles may or may not change the tool, depending upon the hardness of the shackle.

Hornsby, B. ―MCC Bolt Cutters.‖ AFTE Journal, vol. 21 (3), July, 1989, p. 508.

This study randomly selected bolt cutters from the same production run. The working surfaces of the bolt cutters were produced through milling and tumbling. The study concluded that, test marks produced by these bolt cutters were unique to the tool that made them.

Jordan, Tom. ―Individual Characteristics on Copper Insulated Wire.‖ AFTE Journal, Vol. 14 (1), 1982, pp. 53-56.

Using 3 to 6 inch sections of #12 insulated copper wire, this study revealed that the drawing marks are unique to the tool that produced them during manufacture.

Lee, Susan E. ―Examination of Consecutively Manufactured Slotted Screwdrivers.‖ AFTE Journal, vol. 35 (1), Winter, 2003, pp. 66-70.

39June 14, 2011 39

This study used five consecutively made screwdrivers to test the reproducibility of marks produced at various angles with both pushing and pulling motions. Each screwdriver‘s marks were found to be individual to tool that produced them.

Miller, Jerry and G. Beach, ―Toolmarks: Examining the Possibility of Subclass Characteristics,‖ AFTE Journal, Vol. 37 (4), Fall 2005, pp. 296-345.

This study utilizes consecutively manufactured diagonal cutting pliers, slip joint pliers, center punches, cold chisels and beveled wood chisels to determine if these tools possess subclass characteristics and individual characteristics. In all cases, except the center punches, subclass characteristics and individual characteristics were observed. The grinding process used to finish the punches produce individual characteristics only. The remaining tools produce marks that are easily identified based on individual characteristics or a combination of subclass and individual characteristics which are easily discernible.

Miller, Jerry. ―Cut Nail Manufacturing and Toolmark Identification.‖ AFTE Journal, Vol. 30 (3), Summer 1998, pp. 492-498.

This study discusses the cut nail manufacturing process and a study was conducted in which 32,000 + nails were identified to the tools that made them.

Murdock, John E. ―The Individuality of Tool Marks Produced by Desk Staplers.‖ AFTE Journal, Vol. 6 (5), 1974. pp. 23-39.

This study found that Pilot brand staplers produced individual marks on staples, while Swingline brand staplers produced only subclass marks. The manufacturers used different manufacturing methods which was the reason for the differing types of marks.

Reitz, J. ―An Unusual Toolmark Identification case.‖ AFTE Journal, vol. 7 (3), Dec., 1975, pp. 40-43.

Consecutively ground and randomly selected twist drill bits were studied. Results show no risk of misidentification. The grinding process caused no subclass influence.

Thompson, Evan and R. Wyant, ―Knife Identification Project (KIP),‖ AFTE Journal, Vol. 35 (4), Fall 2003, pp. 366 – 370.

This study utilizes ten consecutively manufactured knives produced by the Benchmade Knife Corporation to produce a test to evaluate the uniqueness of striated toolmarks. One hundred and forty tests were distributed at the 2002 AFTE Training Seminar. One hundred and three examiners submitted results for inclusion in the study. Of the possible 1,030 possible answers, 1,022 were correct

40June 14, 2011 40

(8 incorrect answers). The error rate for this study was calculated to be 0.776 percent.

Tuira, Y.J. ―Tire Stabbing with Consecutively Manufactured Knives.‖ AFTE Journal, Vol. 14 (1), 1982, pp. 50-52.

Two consecutively made Buck knives were thrust into inflated tires and the toolmarks compared. The toolmarks were found to be significantly different.

Van Dijk, T.M. ―Steel Marking Stamps: Their Individuality at the Time of Manufacture.‖ Journal of the Forensic Science Society, Vol. 25 (4), July/Aug, 1985, pp. 243-253.

Fifty steel marking stamps made from the same hob (die) were examined for subclass marks. Unique defects from the hobbing process could be used to correctly identify each stamp.

Watson, D. ―The Identification of Toolmarks Produced From Consecutively Manufactured Knife Blades in Soft Plastics.‖ AFTE Journal, vol. 10 (3), September, 1978, pp. 43-45.

This article discusses the uniqueness of two consecutively manufactured knives. No carryover was found to exist between the two knives.

Watson, Donald J., ―The Identification of Consecutively Manufactured Crimping Dies,‖ AFTE Journal, vol. 10, September 1978, pp. 19-21.

This study documents the manufacturing process of crimping dies and the results of the comparison of two consecutively manufactured crimping dies. It was found that the crimping dies bore no ―carry-over‖ effects and that lead seals crimped with these dies could be identified back to their source.

41June 14, 2011 41

11. What research exists that has examined the minimum set of skills a practitioner should possess in order to perform a specific task (e.g. pattern recognition and aptitude versus education)?

Minimum Qualifications for Firearm and Toolmark Examiner Trainees (4/20/2006).

Scientific Working Group for Firearms and Toolmarks. http://www.swggun.org/swg/index.php?option=com_content&view=article&id=30:minimum-qualifications-for-firearm-and-toolmark-examiner-trainees&catid=10:guidelines-adopted&Itemid=6

This document is a guideline that addresses the minimum education requirements for individuals seeking employment as a firearm and toolmark examiner.

The Association of Firearm and Tool Mark (AFTE) Training manual.

The ATF National Firearms Training Academy (NFEA) -minimum educational

qualifications requirements

National Forensic Science Technology Center (NFSTC)- Firearm Examiner Training Program

42June 14, 2011 42

http://www.swggun.org/swg/index.php?option=com_content&view=article&id=30:minimum-qualifications-for-firearm-and-toolmark-examiner-trainees&catid=10:guidelines-adopted&Itemid=6



12. Are there any studies that use digital imaging to either validate or invalidate the basic tenets of firearm and toolmark comparisons? If so, what automated methods exist and how can they be refined?

Chew, Wei et al (2010). Striation Density for Predicting The Identifiability of Fired Bullets With Automated Inspection Systems. Journal of Forensic Sciences, Vol. 55, No. 5, 1222-1226. Deinet, Werner. (1981). ―Studies of Models of Striated Marks Generated by Random Processes.‖ Journal of Forensic Sciences, Vol. 26 (1), pp. 35-50.

The author uses computer-aided studies of the degree of similarity of striated marks. He produced digitized images of 40 grinding marks which were entered into a computer wherein position values were defined and idealized models were defined for an objective comparison of striated marks.

Evans, Paul & Smith, Clifton. (2004). ―Validation of the Linescan Imaging Technique for Imaging Cylindrical Forensic Ballistics Specimens.‖ AFTE Journal, Vol 36, No 4. pp 275-280.

Abstract: The imaging of cylindrical surfaces of forensic ballistics specimens with an area camera system requires the geometric registration of a number of different images in order to effect a 360° view of its surface. This view is very difficult to achieve in terms of consistent spatial resolution. A line-scan imaging technique utilising an area array camera producing images of rotating ballistics specimens is described. A discussion of the validity of the technique to use the images for the identification of firearms is presented. The 2-D planar images of complete 3-D cylindrical surfaces produced by the linescan technique have a geometric format that is particularly suited to morphological image processing and the production of image databases.

Geradts, Zeno, et al. (2001). ―Pilot Investigation of Automatic Comparison of Striation Marks with Structured Light.‖ SPIE Proceedings Paper, Vol 4232. pp 49-56

Abstract: We have developed and tested an algorithm that can compare striation marks that are acquired with a standard camera and sidelight as well as 3D- information acquired with structured light. With six different screwdrivers test marks have been made with an angle of 45 degrees to the surface. These striation marks are moulded with gray silicon casting material. Then these marks are digitized with the structured light approach and with side light. For the structured light approach, it appeared that there are artifacts and variations in the image due to the number of stripes in the LCD projections and the camera resolution. We have compensated for these variations by averaging the lines over an area that is selected by the user. In the method that has been used for averaging, the slopes of the striae are followed. This method is also used for side light images to compensate for variations in the striation mark. In this research,

43June 14, 2011 43

signatures of the tool marks are calculated then compared with a database of signatures by calculating the standard deviation of the difference. For the limited test set of six striation marks made with six different screwdrivers, the algorithm was able to distinguish the global shape of the screwdriver and the depth information itself. Since the images acquired with structured light contain more information on the toolmark itself, the correlation results were better than with side light images.

Intelligent Automation, Incorporated (October, 2010). A Statistical Validation of the Individuality of Guns Using High Resolution Topographical Images of Bullets. National

Institute of Justice Grant #2006-DN-BX-K030.

Study of marks on fired bullets by a topography based (3D) automated system. This study continued the analysis of a previous 2005 NIJ bullet study and validated the original premise of Firearm/Toolmark ID. This study also concluded that 1) the ability to determine that a given bullet was fired from a specific barrel depends on the individual barrel itself and not only on the brand of its manufacture, and 2) the performance of the automated analysis system used in this study is not representative of that of a trained firearms examiner as humans have a remarkable ability to perform pattern matching that is difficult to be replicated in any automated system.

Leon, Fernando. (2006). ―Automated Comparison of Firearm Bullets.‖ Forensic Science International, Vol 156. pp 40-50.

Abstract: Fired bullets bear striation marks that can be thought of as a ‗‗fingerprint‘‘ left by the firearm. This new comparison approach is based on an automated extraction of a ‗‗signature‘‘ encompassing the relevant marks from an image. To this end, multiple pictures of the bullet are recorded first by using different illumination patterns, and a high quality image is generated by means of fusion techniques. After a preprocessing, the image intensities are filtered along the striations direction, yielding a compact representation of the marks. A non-linear filter selects the striae of interest. The actual comparison takes place by cross correlating the signatures obtained this way. Finally, an assessment strategy is proposed to objectively evaluate the performance of the system. As demonstrated with an image database of real bullets, the proposed approach outperforms a state-of-the-art commercial system.

Miller, J., McLean, M. (1998). Criteria for Identification of Toolmarks. AFTE Journal, Vol. 30, No.1, pp.15-61.

Using IBIS, the authors compared land impressions of .38 Special jacketed bullets fired from S&W revolvers. Found no CMS counts greater than six (6) for KNMs, using the computer monitor. Using a separate set of testfires and the comparison microscope, no CMS counts greater than four (4) for KNMs were found.

44June 14, 2011 44

Miller, J. (2000). Criteria for Identification of Toolmarks, Part II: Single Land Impression Comparisons. AFTE Journal, vol. 32 (2), Pp.116-131.

This study compares bullets fired by Raven 25 Auto, Lorcin 380 Auto, and Stallard Arms 9mm pistols to specimens in the NIBIN database. This study supports the Biasotti/Murdock Quantitative Conservative Matching Striae (QCMS) criteria.

Smith, C. L. (2002). Linescan Imaging of Ballistics Projectile Markings for Identification. Security Technology Proceedings, 36th Annual International Carnahan Conference, pp. 216 – 222.

The identification of firearms from forensic ballistics specimens is an exacting and intensive activity performed by specialists with extensive experience. The introduction of imaging technology to assist the identification process of firearms has enhanced the ability of forensic ballisticians to conduct analyses of these specimens for identification. The characteristic markings on the cartridge and projectile of a bullet fired from a gun can be recognised as a fingerprint for identification of the firearm. Forensic ballistics imaging has the capacity to produce high-resolution digital images of cartridge cases and projectiles for matching of crime scene specimens to test specimens. Projectile bullets fired through the barrel of a gun will exhibit extremely fine striation markings, some of which are derived from minute irregularities in the barrel produced during the manufacturing process. The examination of these striations on the land marks and groove marks of the projectile is difficult using conventional optical microscopy. However, digital imaging techniques have the potential to detect the presence of striations on ballistics specimens for identification matching. This paper describes a linescan imaging technique to examine the striation markings on the land marks and groove marks of projectiles for positive identification. The paper discusses the application of the technique to cylindrical forensic ballistics specimens, and the potential of the technique for image matching. Digital images of land marks and groove marks of projectiles produced by the line scan technique are presented, and analyses of the images are conducted.

Smith, C.L. and Cross, J.M. (1995). ―Optical Imaging Techniques for Ballistics Specimens to Identify Firearms.‖ IEEE Proceedings: 29th Annual International Carnahan Conference on Security Technology. pp 275-289.

Abstract: Characteristic markings on the cartridge and projectile of a bullet are produced when a gun is fired. Over thirty different features within these markings can be distinguished, which in combination produce a ―fingerprint‖ for identification of a firearm. This paper describes an investigation into the development of an imaging system which can store, analyze, retrieve, and match high resolution digital images of cartridge cases. A computerized imaging system for ballistics identification will produce efficiencies in time and personnel, and

45June 14, 2011 45

permit a more precise audit of firearms within a country. The project has produced good quality high resolution digitized images of cartridge cases. The development of the optical system to optimize image quality has been crucial for the image identification phase. By classifying cartridge image attributes, it is possible to store the unique ―signatures‖ of cartridge cases for identification. Appropriate image processing provides the signatures for the image library. The FIREBALL forensic ballistics interactive database incorporates a graphics user interface (GUI) to obtain precise ballistics metrics of cartridge case class characteristics. This project will significantly improve the effectiveness and efficiency of ballistics records and assist the tracing of firearms used in criminal activities by law enforcement agencies.

Smith, C.L. (2006). ―Profiling Toolmarks on Forensic Ballistics Specimens: An Experimental Approach.‖ IEEE Proceedings: 40th Annual International Carnahan Conference on Security and Technology. pp 281-286.

Abstract: This paper will discuss an experimental approach for matching toolmark images from the linescan technique and measurements from the profilometry technique of surface mapping. By relating the topography of a region on the surface of a cartridge case and projectile to that region on the unwrapped image from the linescan technique, a potential match may be developed. The application of depth profiling of imperfections on the projectiles and cartridges from the firing of a weapon has the potential to map unique markings on the ballistics specimens from crime scene firearms. The paper will show depth profiles of land marks and groove marks of projectiles, and firing pin marks and breach face marks on cartridges as examples supporting the potential for the profiling technique.

Zographos, A., et al. (1997). ―Ballistics Identification Using Line-Scan Imaging Techniques.‖ IEEE Proceedings: 31st Annual International Carnahan Conference on Security Technology. pp 82-87.

Abstract: A new line-scan imaging technique, well-suited to the inspection of ballistics specimens, is presented. The proposed system addresses a number of problems associated with the imaging of cartridge cases when conventional inspection techniques are used. This particular paper deals exclusively with imaging of the cylindrical surface of cartridge cases. Imaging of the firing pin marks on the end of the case is not considered. Results obtained so far from a prototype system are presented. It should be noted that this technique is still very much in its experimental phase and that as yet it has not been fully investigated for this forensic ballistics application.

Zhihu Huang; Jinsong Leng. (2010). ―An Online Ballistics Imaging System for Firearm Identification.‖ IEEE - 2nd International Conference on Signal Processing Systems, Vol 2. pp 68-72.

46June 14, 2011 46

Abstract: Since the traditional ballistics imaging system is dependent upon the expertise and experience of end-user, an intelligent ballistics imaging system is highly demanded to overcome the drawbacks of traditional techniques. This paper aims to develop a novel ballistics imaging system so as to combine the traditional functions with new features such as the line-scan image module, the characteristics extraction module, and the intelligent image processing module. With the help of these features, the new system can identify firearm more efficiently and effectively than the traditional techniques.

Zhihu Huang; Jinsong Leng. (2010). ―A Novel Binarization Algorithm for Ballistics Imaging Systems.‖ IEEE – 3rd International Congress on Image and Signal Processing, Vol. 3. pp 1287 – 1291.

Abstract: The identification of ballistics specimens from imaging systems is of paramount importance in criminal investigation. Binarization plays a key role in preprocess of recognizing cartridges in the ballistic imaging systems. Unfortunately, it is very difficult to get the satisfactory binary image using existing binary algorithms. In this paper, we utilize the global and local thresholds to enhance the image binarization. Importantly, we present a novel criterion for effectively detecting edges in the images. Comprehensive experiments have been conducted over sample ballistic images. The empirical results demonstrate the proposed method can provide a better solution than existing binary algorithms.

47June 14, 2011 47

13. What literature documents the automated methods of comparison that exist for firearms/toolmarks examination, how they are being applied to the examination process, and any potential shortcomings.

Bachrach, B. (2002). Development of a 3D-Based Automated Firearms Evidence Comparison System. Journal of Forensic Sciences, vol. 47 (6), 1253-1264.

Abstract: This study reports on a computerized system that calculates correlation coefficients for comparisons of bullet striation patterns using generated 3-D maps of bullet surfaces. Was validated using known matches (KMs) and known non-matches (KNMs), so therefore the system arrives at a conclusion of identification (or not), with an associated probability of error. Highly relevant to our work, because it shows conclusively that an objective observer (a machine) detects significant visual differences between KNMs and KMs.

Bachrach, B. (et al). (2010). ―A Statistical Validation of the Individuality and Repeatablity of Striated Tool Marks: Screwdrivers and Tongue and Groove Pliers.‖ Journal of Forensic Sciences, Vol 55, No. 2. pp 348-357.

Abstract: Tool mark identification relies on the premise that microscopic imperfections on a tool‘s working surface are sufficiently unique and faithfully transferred to enable a one-to-one association between a tool and the tool marks it creates. This paper presents a study undertaken to assess the validity of this premise. As part of this study, sets of striated tool marks were created under different conditions and on different media. The topography of these tool marks was acquired and the degree of similarity between them was quantified using well-defined metrics. An analysis of the resulting matching and nonmatching similarity distributions shows nearly error-free identification under most conditions. These results provide substantial support for the validity of the premise of tool mark identification. Because the approach taken in this study relies on a quantifiable similarity metric, the results have greater repeatability and objectivity than those obtained using less precise measures of similarity.

Blackwell, R.J., and Framan E.P. (1980). ―Automated Firearms Identification System (AFIDS): Phase I. AFTE Journal, Vol. 12, No. 4. pp 11-37. (Reprinted in AFTE Journal – original article was prepared by the authors of the Jet Propulsion Laboratory for the Applications Technology Office, National Aeronautics and Space Administration)

Abstract: Items critical to the future development of an automated firearms identification system (AFIDS) have been examined, with the following results: 1. Types of objective data, that can be utilized to help establish a more factual basis for determining identity and nonidentity between pairs of fired bullets, have been identified. 2. A simulation study has indicated that randomly produced line, similar in nature to the individual striations on a fired bullet, can be modeled and that random sequences, when compared to each other, have predictable relationships., 3. A schematic diagram of the general concept for AFIDS has

48June 14, 2011 48

been developed and individual elements of this system have been briefly tested for feasibility., Future implementation of such a proposed system will depend on such factors as speed, utility, projected total cost and user requirements for growth. The success of the proposed system, when operational, would depend heavily on existing firearms examiners.

Chu, Wei, et al. (2010). Pilot Study of Automated Bullet Signature Identification Based on Topography Measurements and Correlations. J Forensic Sci, Vol 55, No 2, 1-7.

Abstract: The authors outline a procedure for automated bullet signature identification based on topography measurements using confocal microscopy and correlation calculation. The correlation results show a 9.3% higher accuracy rate compared with a currently used commercial system based on optical reflection.

De Kinder, Jan and Bonfanti, Monica. (1999). ―Automated Comparisons of Bullet Striations Based on 3D Topography.‖ Forensic Science International, Vol 101. pp 85-93.

Abstract: A system capable of comparing the signatures on bullets in the field of firearms identification is presented. It is based on the recording of the topography of a bullet using laser profilometry. A procedure to derive a one-dimensional array of characteristics out of the recorded data is presented. These so-called feature vectors are compared with similar quantities from other bullets using a correlation technique. Good results were obtained for weapons leaving well-defined characteristics.

Dongguang, L. (2006). Ballistics Projectile Image Analysis for Firearm Identification. IEEE Transactions on Image Processing, Vol 15, No. 10, 2857-2865.

Abstract: The author proposes a new analytic system based on the fast Fourier transform for identifying projectile specimens by the line-scan imaging technique. His paper develops optical, photonic, and mechanical techniques to map the topography of the surfaces of projectiles for the purpose of identification.

Dongguang, L.. (2009). ―Ballistics Image Processing and Analysis for Firearms Identification.‖ Image Processing. Chapter 9, pp 141-174. ISBN: 978-953-307-026-1.

A chapter in the book ―Image Processing‖ in which the author discusses line-scan imaging techniques of firearm related evidence (bullet and cartridge cases) coupled with a new analytic technique of Fast Fourier Transform. The system gives an approach for projectile capturing, storing, and automatic analysis and makes a significant contribution towards the efficient and precise identification of projectiles.

49June 14, 2011 49

Jacque-Mann, M. and Espinoza, E. (1992). ―Firearms Examination by Scanning Electron Microscopy: Observations and an Update on Current and Future Approaches.‖ AFTE Journal, Vol. 24, No. 3. pp 294-303.

Abstract: The use of scanning electron microscopy (SEM) in the examination of firearms evidence was introduced in the literature over twenty years ago. Due to cost and convenience factors, however, scanning electron microscopy has remained largely inaccessible and therefore under-exploited in the firearms community. We will review the significant advantages of SEM in the examination of firearms evidence and present information on recent and future improvements in methodology which make this approach more attractive and available to forensic laboratories.

Jones, B.C., Press, M., and Guerci, J.R. (1998). ―Decision Fusion Based Automated Drill Bit Correlator.‖ SPIE Conference on Investigation and Forensic Science Technologies: SPIE, Vol. 3576. pp 253-263.

Abstract: This paper describes a recent study conducted to investigate the reproducibility of toolmarks left by drill bits. This paper focuses on the automated analysis aspect of the study, and particularly the advantages of using decision fusion methods in the comparisons. To enable the study to encompass a large number of samples, existing technology was adapted to the task of automatically comparing the test impressions. Advanced forensic pattern recognition algorithms that had been developed for the comparison of ballistic evidence in the DRUGFIRETM system were modified for use in this test. The results of the decision fusion architecture closely matched those obtained by expert visual examination. The study, aided by the improved pattern recognition algorithm, showed that drill bit impressions do contain reproducible marks. In a blind test, the DRUGFIRE pattern recognition algorithm, enhanced with the decision fusion architecture, consistently identified the correct bit as the source of the test impressions.

Miller, J. (2000). Criteria for Identification of Toolmarks, Part II: Single Land Impression Comparisons. AFTE Journal, vol. 32 (2), 116-131.

Abstract: This study compares bullets fired by Raven 25 Auto, Lorcin 380 Auto, and Stallard Arms 9mm pistols to specimens in the NIBIN database. This study supports the Biasotti/Murdock Quantitative Conservative Matching Stiae (QCMS) criteria.

Senin, Nicola, et al. (2006). ―Three-Dimensional Surface Topography Acquisition and Analysis for Firearm Identification.‖ Journal of Forensic Sciences, Vol. 51, No. 2. pp 282-295.

Abstract: In the last decade, computer-based systems for the comparison of microscopic firearms evidence have been the subject of considerable research

50June 14, 2011 50

work because of their expected capability of supporting the firearms examiner through the automated analysis of large amounts of evidence. The Integrated Ballistics Identification System, which is based on a two-dimensional representation of the specimen surface, has been widely adopted in forensic laboratories worldwide. More recently, some attempts to develop systems based on three-dimensional (3D) representations of the specimen surface have been made, both in the literature and as industrial products, such as BulletTRAX-3D, but fundamental limitations in achieving fully automated identification remain. This work analyzes the advantages and disadvantages of a 3D-based approach by proposing an approach and a prototype system for firearms evidence comparison that is based on the acquisition and analysis of the 3D surface topography of specimens, with particular reference to cartridge cases. The concept of 3D virtual comparison microscope is introduced, whose purpose is not to provide fully automated identification, but to show how the availability of 3D shape information can provide a whole new set of verification means, some of them being described and discussed in this work, specifically, visual enhancement tools and quantitative measurement of shape properties, for supporting, not replacing, the firearm examiner in reaching the final decision.

Smith, C. and Li, Dongguang. (2008) ―Intelligent Imaging of Forensic Ballistics Specimens for ID.‖ IEEE Proceedings: Congress on Image and Signal Processing, Vol 3. pp 37-41.

Abstract: The paper describes some of important technologies in firearm identification using forensic ballistics specimens. The mapping of micro-surfaces on regions on the specimens for comparison to establish identification according to the precision of measurement of the features has been proposed. The physical techniques of linescan, laser depth proofing, and photonic 3D topography can be developed into future tools for forensic ballisticians for identification of cartridge cases and projectiles.

Uchiyama, Tsueno. (1988). ―Automatic Comparison Model of Land Marks.‖ AFTE Journal, Vol. 20, No. 3. pp 252-258.

Abstract: In the comparison of fired bullets, examiners compare the contour of the bullet surfaces using a comparison microscope. There can be much information on the surfaces of fired bullets that is related to the gun bore surface through which bullets are fired. However, one cannot expect that all of the bore surface characteristics will be reproducibly transferred; even on consecutively fired bullets it may not be possible. Also, there are always many extraneous markings on fired bullet surfaces which are not representative of the gun bore surface and which the examiner must learn to distinguish and ignore as useless information. Usually, a comparison microscope examination of a pair of fired bullets requires more than 20 minutes before the examiner can form an opinion as to identity or non-identity. In some cases, such an examination requires up to one week. In this paper, the bullet comparison process conducted by the

51June 14, 2011 51

examiner was analyzed in order to see if any of this manual process could be replaced or assisted by the use of a computer.

Uchiyama, Tsueno. (1993). ―Automated Landmark Identification System.‖ AFTE Journal, Vol 25, No. 3. pp 172-196.

Abstract: We have developed the Automated Landmark Identification System (ALIS). This system handles the images of the striae in the landmarks on fired bullets through a CCD camera. First, image processor calculate the histogram of intensity of an image from the CCD camera. Then a personal computer converts the histogram data into "bar code" like stripes. These "bar code" and histogram data are compared to any other similarly recorded and stored data. In comparison process, the data of the landmark with the narrower width is moved from left to right, and the percent match of striae, sum of square between histograms of image data and sum of depth of matching striae are calculated for each shift in position. The maximum number of consecutively matched striae is also counted for each shift in position. From the fluctuating pattern of these calculated parameters, the reliability of the positive conclusion can be calculated.

52June 14, 2011 52

14. What studies exist regarding the use of databases to facilitate an automated approach to analysis?

Bachrach, B. (2000). ―Ballistic Matching Using 3D Images of Bullets and Cartridge Cases: Project Summary.‖ National Institute of Justice Grant Award Number 97-LB-VX-0008.

Abstract/Results of Study: To determine the feasibility of using 3D information from a bullet‘s surface to improve the matching rate of existing automated search and retrieval systems, it was required to develop and implement all the elements of an acquisition component. Furthermore, this particular acquisition component would operate based on 3D captured data, as opposed to 2D captured data. Together with the acquisition component, a preliminary version of a correlation component was developed in order to verify the usefulness of the 3D captured data. The complete automated search and retrieval system was tested through a number of independent evaluations. Among these evaluations, we have performed a number of so-called ―blind tests.‖ For these blind tests, we were provided with control bullets from different guns (i.e., we were told which gun fired each of the ―control bullets‖), and with questioned bullets. The task was to identify which gun fired each of the questioned bullets based on the data obtained from the control bullets. In all cases the system was able to perform in a very satisfactory manner, making very few mistakes in the identification of which gun fired each of the questioned bullets. As a direct result of the research done under this project we have developed a fully functional prototype of the 3D ballistic analysis system (named SCICLOPs).

Baldwin, D., Morris, M., Bajic, S., Zhou, Z., Kreise, M. J. (April 2004). Statistical Tools for Forensic Analysis of Toolmarks. Ames Laboratory, USDOE Office of Science, IS-5160.

Recovery and comparison of toolmarks, footprint impressions, and fractured surfaces connected to a crime scene are of great importance in forensic science. The purpose of this project is to provide statistical tools for the validation of the proposition that particular manufacturing processes produce marks on the work-product (or tool) that are substantially different from tool to tool. The approach to validation involves the collection of digital images of toolmarks produced by various tool manufacturing methods on produced work-products and the development of statistical methods for data reduction and analysis of the images. The developed statistical methods provide a means to objectively calculate a "degree of association" between matches of similarly produced toolmarks. The basis for statistical method development relies on "discriminating criteria" that examiners use to identify features and spatial relationships in their analysis of forensic samples. The developed data reduction algorithms utilize the same rules used by examiners for classification and association of toolmarks.

53June 14, 2011 53

Banno, Atsuhiko, et al. (2004). Three Dimensional Visualization and Comparison of Impressions on Fired Bullets. Forensic Science International, 140, pp 233-240.

In this study, the authors focused on 3D geometric data of landmark impressions on fired bullets for identification. They presented an algorithm for a shape comparison of impressions on bullets. They were concerned only with visualization and comparison and not identification. However, the authors feel the most important future work regarding this method is the identification phase which would require the comparison of numerous pairs of bullets to determine the rigid threshold.

Banno, Atsuhiko. (2004). ―Estimation of Bullet Striation Similarity Using Neural Networks.‖ Journal of Forensic Sciences, Vol 49, No 3. pp 1-5.

Abstract: A new method that searches for similar striation patterns using neural networks is described. Neural networks have been developed based on the human brain, which is good at pattern recognition. Therefore, neural networks would be expected to be effective in identifying striated toolmarks on bullets. The neural networks used in this study deal with binary signals derived from striation images. This signal plays a significant role in identification, because this signal is the key to the individuality of the striations. The neural network searches a database for similar striations by means of these binary signals. The neural network used here is a multilayer network consisting of 96 neurons in the input layer, 15 neurons in the middle, and one neuron in the output layer. Two signals are inputted into the network and a score is estimated based on the similarity of these signals. For this purpose, the network is assigned to a previous earning. To initially test the validity of the procedure, the network identifies artificial patterns that are randomly produced on a personal computer. The results were acceptable and showed robustness for the deformation of patterns. Moreover, with ten unidentified bullets and ten database bullets, the network consistently was able to select the correct pair.

Bolton-King, Rachel S. et. al. (2010). What are the Prospects of 3D Profiling Systems Applied to Firearms and Toolmark Identification? AFTE Journal, Vol 42, No 1, 23 – 33.

This article concluded that focus-variation microscopy has potentially the most promising approach for a forensic laboratory instrument, in terms of functionality and 3D imaging performance, and is worthy of further investigation.

De Kinder, Jan, et al. (1998). ―Surface Topology of Bullet Striations: An Innovating Technique.‖ AFTE Journal, Vol. 30, No. 2. pp 294-299.

Abstract: Laser topography is presented as a way to obtain characteristic information of the striation marks on bullets. A profilometer, equipped with a translational and rotational stage, is used tor this purpose. Using the translational stage, the parameters of the system were optimized and one groove on a 9mm

54June 14, 2011 54

Para bullet was studied. The correspondence between our results and light microscopy is shown. The first results of measurements of the whole circumference of a bullet using the rotational stage promise a potential application in the field of firearms and tool marks identification.

De Kinder, Jan. (2002). ―Ballistic Fingerprinting Databases.‖ Science & Justice, Vol 42, No. 4. pp 197-203

Abstract: This article discusses a number of questions regarding the setting up of ballistic fingerprinting databases, consisting of ammunition components fired by all the firearms held in legal possession. These questions can be classified into three categories--the efficiency of the database, forensic issues, and practical issues to be dealt with. The current New York State legislation is used as an illustration of the choices to be made when setting up a ballistic fingerprinting database. Three important arguments are formulated against the installation of a ballistic fingerprinting database.

Demoli, N. et al. (2004). Toolmarks Identification using SEM Images in an Optoelectronic Correlator Device. Optik, Vol 115, No. 11, pp. 487-492.

The authors propose a method for identifying toolmarks by utilizing an optoelectronic correlator device as a possible solution. The effectiveness of the proposed approach is demonstrated by the results of the identification of marks on wires by lap joint pliers. Since this method combines fast optical processing and digital image information, the proposed method can be automated.

George, W. (2004). ―A Validation of the Brasscatcher Portion of the NIBIN/IBIS System.‖ AFTE Journal, Vol. 36, No. 4. pp 286-288.

Abstract: A study was conducted using over 500 cartridge casings fired in Smith & Wesson½ .40 S&W caliber pistols and entered into the BRASSCATCHER portion of IBIS. Entries were made using Remington½ and Federal½ ammunition and the study examined the ability of IBIS to match these cartridge casings. The results of this study raise the issue of limiting the viewable correlation results to the top 20% of the entries in the database

George, W. (2004). ―The Validation of the Brasscatcher Portion of the NIBIN/IBIS System Part Two: Fingerprinting Firearms Reality or Fantasy.‖ AFTE Journal, Vol. 36, No. 4. pp 289-296.

Abstract: A study of the Brasscatcher portion of the NIBIN/IBIS system was conducted using a database of 850 cartridge cases fired in Smith & Wesson .40 S&W caliber pistols. Correlations were generated for entries from Federal, Winchester and Remington brand ammunition and a study to locate the placement of matching cartridge cases initiated. Forensic Technology was able to open the entire database for viewing instead of the normal user field of 20 %.

55June 14, 2011 55

This study provided a real test of the ability of Brasscatcher to identify cartridge cases fired from similar firearms, and addresses the concept of fingerprinting firearms for use in criminal investigations. During this study an additional advantage regarding the second breech face impression image was revealed. The second image is not used for correlation purposes.

Geradts, Zeno, et al. (1994). A New Approach to Automatic Comparison of Striation Marks. Journal of Forensic Sciences, Vol. 39, No 4, pp. 974-980.

The authors created a database for toolmarks named TRAX using a PC. The database is filled with video images and administrative data about the toolmarks. The authors also developed an algorithm for the automatic comparison of digitized striation patterns. The system works well for deep and complete striation marks which will be implemented in TRAX.

Geradts, Z. et al. (1999). ―Pattern Recognition in a Database of Cartridge Cases.‖ SPIE Proceedings: Investigation and Forensic Science Technologies, Vol. 3576. pp 104-115.

Abstract: On the market several systems exist for collecting spent ammunition for forensic investigation. These databases store images of cartridge cases and the marks on them. The research in this paper is focused on the different methods of feature selection and pattern recognition that can be used for comparison. For automatic comparison of these images it is necessary to extract firstly the useful parts of the images. On databases of 2000 images several preprocessing steps have been tested and compared. The results and methods, which have been implemented, are presented.

Giverts P., Springer E., and Argaman U., Using the IBIS for the Examination of Bullets Fired from Polygonally Barreled Guns Such as the Glock Pistol, AFTE Journal, Volume 36, Number 3, Summer 2004, pp 226-229.

Polygonally rifled barreled handguns have enjoyed much popularity and have become widespread in recent years. However, as of now, the IBISÖ is not too efficient in searching polygonal bullets. Thus, there is now all the more need for one to be able to successfully handle, in the IBISÖ, bullets fired from such. This paper describes and suggests a possible solution for enabling the IBISÖ to successfully handle such bullets.

Kong, Jun, et al. (2003). A Firearm Identification System Based on Neural Network. AI 2003: Advances in Artificial Intelligence; Lecture Notes in Computer Science, Vol. 2903, pp.315-326.

The authors present a firearm identification system based on Self-Organizing Feature map (SOFM) neural network. The experiments performed showed the model proposed has high performance and robustness by integrating the SOFM neural network and the decision-making strategy. The model also will make a

56June 14, 2011 56

significant contribution towards the further processing, such as the more efficient and precise identification of cartridge cases by combination with more characteristics on cartridge case images.

Kong, Jun, et al. (2004). An Automatic Analysis System for Firearm Identification Based on Ballistics Projectiles. Rough Sets and Current Trends in Computing; Lecture Notes in Computer Science, Vol 3066, Pp. 653-658.

Over 30 different features within the marks left on bullets and projectiles can be distinguished which in combination produce a ―fingerprint.‖ The authors present a means of automatically analyzing features within a firearm ―fingerprint‖ where it is possible to identify not only the type and model of a firearm, but also each individual weapon. A new analytic system based on fast Fourier transform (FFT) for identifying the projectile specimens digitized using the line-scan imaging technique. Experimental results show that the proposed system has the ability of efficient and precise analysis and identification for projectiles specimens.

Kou, Chenyuan and Tung, Cheng-Tan. (1994). ―FISOFM: Firearms Identification Based on SOFM Model of Neural Network.‖ Proceedings – Institute of Electrical and Electronics Engineers – 28th Annual International Carnahan Conference on Security Technology. pp 120-125.

Abstract: Firearms Identification (F1) has been getting serious and tremendous in crime investigation for the last two decades. We propose a solution to Fl in Neural Network (NN) technology. There are lots of methods have been using in FI such as extractor mark, breach mark, ejector mark, and chambering mark identification, etc. We choose the chambering mark identification as our method in this research. It is the simple and useful method for crime investigation. Because of the principle of tool mark, we may identify the firearms. The chambering mark needs to be scanned, preprocessed, segmented, described, reduced and enhanced the noise pattern, and will be recognized its individual characteristic via the Self Organizing Feature Map(S0FM) model of NN. lt really eases the burden of Forensic Laboratory's technicians, because they do not need to Identify the tool mark via microscope, instead of using Neural Network technology of Artificial Intelligence to identify firearms.

Peterson, J.L., (1974). Utilizing the Laser for Comparing Tool Striations. Journal of Forensic Sciences., Vol. 14, No 1, pp. 57-62.

The author describes a method for examining the contour of striated tool marks by focusing laser light on the tool striations moving at a constant rate. The graphical representations of the reflected light may be used to compare tool marks without utilizing a comparison microscope, however the author determined that the system would require refinement prior to its regular utilization in a forensic science laboratory.

57June 14, 2011 57

Robinson, M. et al. (1998). ―Ballistics Imaging – Latest Developments.‖ IEEE Proceedings: 32nd Annual International Carnahan Conference on Security Technology. Pp 181-183.

Abstract: This paper outlines the latest developments concerning a forensic imaging system that uses a CCTV camera in a line-scan mode for the inspection of bullet specimens placed on a rotating platform. Specifically, line-scan images obtained from the cylindrical sides of fired rounds including both projectiles and cartridge cases plus firing pin marks are discussed. Ultimately, it is anticipated that this research will lead to the development of a forensic imaging application from which a data base can be built to automatically match different bullet specimens in much the same way as a fingerprint data base is operated.

Smith, C. (1997). ―Fireball: A Forensic Ballistics Imaging System.‖ Institute of Electrical and Electronic Engineers Proceedings: 31st Annual International Carnahan Conference on Security Technology. pp 64-70.

Abstract: Characteristic markings on the cartridge and projectile of a bullet are produced when a gun is fired. Over thirty different features within these markings can be distinguished, which in combination produce a ―fingerprint‖ for identification of a firearm. This paper will describe an investigation into the development of an imaging system for Police Services which can store, analyse, retrieve, and match high resolution digital images of cartridge cases. A computerised imaging system for ballistics identification will produce efficiencies in time and personnel, and permit a more precise audit of firearms within a country. The project has produced good quality high resolution digitised images of cartridge cases for the identification function. The development of the optical system to optimise image quality has been crucial for the image identification phase. By classifying cartridge image attributes, it is possible to store the unique ―signatures‖ of cartridge cases for identification. Appropriate image processing provides the signatures for the image library. The Fireball forensic ballistics interactive database incorporates a Graphics User Interface (GUI) to obtain precise ballistics metrics of cartridge case class characteristics. Features of the Fireball forensic ballistics database will be presented, together with a description of the major applications of this ballistics imaging system.

Thompson, R., et al. (1996). ―Computerized Image Analysis for Firearms Identification; The Integragted Ballistic Identification System (IBIS) BRASSCATCHER Performance Study.‖ AFTE Journal, Vol. 28, No. 3. pp 194-203

Abstract: The Bureau of Alcohol, Tobacco and Firearms (ATF) San Francisco Laboratory Center has conducted a Performance Study of the IBIS BRASSCATCHER hardware and software used in acquiring and correlating breech face and firing pin impressions on expended cartridge casings. Pairs of casings from over 200 pistols representing .25, .380, 9mm, .45 calibers were

58June 14, 2011 58

correlated. There was no microscopic pre-screening of the cartridge casings prior to selection for testing. The correct "twin" casing was found in the first position of ranked scores between 58 to 78 percent of the time, and 74 to 93 percent of the time in the top five positions. The correct "twin" casing was found in the first position 94 percent of the time for a Glock 9mm caliber pistol database. Visual subjective estimations of cartridge casing "matchability" show that images judged "good" and "fair" in quality constituted the bulk of "first position " matches selected by the computer. However, a sizeable number of "poor" quality images of matching casings were still found in the top scoring position.

Uchiyama T., Toolmark Reproducibility on Fired Bullets and Expended Cartridge Cases, AFTE JournaL, Volume 40, No. 1, Winter, 2008, pp. 3-46

The reproducibility of landmarks, breechface marks and firing pin marks on one hundred successively fired bullets and cartridge cases were examined. Three types of Speer brand, one of Remington brand and one of DFA brand frangible cartridges were fired in a semi-automatic pistol. Remarkable differences were observed in the general appearance of the landmarks, breechface marks and firing pin marks which were impressed on the different brands of cartridges, even when consecutively fired. Identification of the landmarks between bullets from different brands of cartridges was difficult because their general appearance differed greatly. Difference in bullet diameters was found to be a major cause of changes in landmarks among different manufacturer's bullets. Although the depth and number of striations decreased gradually, reproducibility of breechface marks on the primers of cartridges was rather good. The diameter of firing pin indentations also differed among different brands of cartridges. Although the reproducibility of the diameter of circular lines on firing pin indentations was good, the detail in these circular lines fluctuated a great deal. Quantitative CMS was used as a means of critically evaluating and communicating the extent of striated pattern agreement among the rifling impressions on the fired bullets in this study.

59June 14, 2011 59

15. Does research exist that supports the “comparative” nature of firearms/toolmarks examinations versus “blind” analysis of known and unknown (e.g. documentation of features and then comparing the resuls)? Does research exist which identifies how and which cognitive factors impact the analysis process?

Balthazard, V. (2004). ―Identification of Projectiles from Firearms.‖ AFTE Journal, Vol. 36, No. 3. Pp 219-225. (REPRINT from Identification de Projectiles d‘Armes a Feu. Archives d‘Anthropologie Criminelle. Vol. 28 (1913), pp 421-433.

Foundational article in the field of firearm and tool mark identification in which the author establishes the general criteria and procedure/processes for determining an identification between two items. The author uses photography as his method of determining an identification as this article predates the advent of the comparison microscope.

Bunch, S. (et al). (2009). ―Is a Match Really a Match? A Primer on the Procedures and Validity of Firearm and Toolmark Identification.‖ Forensic Science Communications, Vol. 11, No. 3. pp 1-10.

Abstract: The science of firearm and toolmark identification has been a core element in forensic science since the early 20th century. Although the core principles remain the same, the current methodology uses validated standard operating procedures (SOPs) framed around a sound quality assurance system. In addition to reviewing the standard procedure the FBI Laboratory uses to examine and identify firearms and toolmarks, we discuss the scientific foundation for firearm and toolmark identification, the identification criterion for a "match," and future research needs in the science.

Burd, D. and Kirk, P. (1942). ―Tool Marks. Factors Involved in Their Comparison and Use as Evidence.‖ Journal of Criminal Law and Criminology, Vol. 32, No. 6. pp 679-686.

Text from Article: Comparison of tool marks as an aid in the solution of crime is a well known and widely used procedure which is generally considered as yielding valid court evidence. It is true; nevertheless, that much misapprehension exists as to the individuality of such marks and the probability of repetition of a mark by more than one tool. Such misapprehension as exists has undoubtedly arisen in part from a lack of careful study of the factors which influence the character of the marks left by a tool, and which must be considered in the identification of such marks. In view of the great difficulty of obtaining at any time an absolutely perfect match of the striations making up a tool mark of the friction type, it is of crucial importance to determine what degree of identity must be established before it can be stated that two marks were made by the same tool.

Gunther, C. O. (1932). ―Markings on Bullets and Shells Fired from Small Arms. Mechanical Engineering, Vol. 54. pp 341-345.

60June 14, 2011 60

AND Gunther, C. O. (1930). ―Markings on Bullets and Shells Fired from Small Arms. Mechanical Engineering, Vol. 52. pp 107-118 and 1065-1069.

AND Gunther, C.O. (1932). ―Principles of Firearms Identification: Further Analysis of Accidentical Characteristics.‖ Army Ordnance, Vol. 13. pp 40-43.

AND Gunther, C.O. (1932). ―Principles of Firearms Identification: Fingerprinting Ordnance in the War on Crime.‖ Army Ordnance, Vol. 12. pp 339-340.

In this series of articles the author explores the application of science to firearm and toolmark examination. He establishes the processes of comparison of evidence from both a class and individual basis. He further describes the techniques of photography and microscopy and how to utilize these techniques during a comparison examination. These articles are foundational for the field of firearm and toolmark identification.

Itiel E. Dror, Christophe Champod, Glenn Langenburg, David Charlton, Heloise Hunt, Robert Rosenthal, Cognitive issues in fingerprint analysis: Inter- and intra-expert consistency and the effect of a ‗target‘ comparison

Deciding whether two fingerprint marks originate from the same source requires examination and comparison of their features. Many cognitive factors play a major role in such information processing. In this paper we examined the consistency (both between- and within-experts) in the analysis of latent marks, and whether the presence of a ‗target‘ comparison print affects this analysis. Our findings showed that the context of a comparison print affected analysis of the latent mark, possibly influencing allocation of attention, visual search, and threshold for determining a ‗signal‘. We also found that even without the context of the comparison print there was still a lack of consistency in analysing latent marks. Not only was this reflected by inconsistency between different experts, but the same experts at different times were inconsistent with their own analysis. However, the characterization of these inconsistencies depends on the standard and definition of what constitutes inconsistent. Furthermore, these effects were not uniform; the lack of consistency varied across fingerprints and experts. We propose solutions to mediate variability in the analysis of friction ridge skin.

Kellett, PM, Individualization: Principles and Procedures in Criminalistics Laboratory Director, San Bernardino County Sheriff's Department, CA

The author's stated purpose in writing this text is to identify and discuss first principles common to all comparisons and individualizations. The material presented grew out of ―Forensic Identification‖ taught at Ontario Police College. The author writes in a style suitable for students, trial attorneys and criminalists. The book is well-referenced (128 footnotes) and contains an index.

61June 14, 2011 61

Osterburg, James W. (1969). ―The Evaluation of Physical Evidence in Criminalistics: Subjective or Objective Process?‖ Journal of Criminal Law and Criminology, Vol. 60, No. 1. pp 97-101.

The author discusses the role statistics might play in establishing identify/identification as well as the current state of firearms identification in determining when individualization has occurred.

Thornton, J. (1979). ―The Validity of Firearms Evidence.‖ AFTE Journal, Vol. 11, No. 2. pp 16-19. (Reprint from Forum July/August 1978).

The author discusses the nature of the comparison process in firearms identification. He further delves into the concept of objective criteria for the practice of firearms identification. He establishes what processes are objective and what processes are subjective in the comparison of two items of evidence.

Tuthill, H., and George, G. (1994). Principles and Procedures in Criminalistics, Lightening Powder Company

This book not only defines the steps of Analysis, Comparison and Evaluation of fingerprint evidence, but defines these steps for all types of physical evidence. The principles of comparison are discussed in the concepts of uniqueness and individualization of physical evidence, with a section devoted to fingerprints. Other topics are ethical and moral considerations, class and individual characteristics, degrees of opinions, and expert witness testimony.

62June 14, 2011 62

16. What studies exist which compare one toolmark analytical method to another? Biasotti, A. (1959). A Statistical Study of the Individual Characteristics of Fired Bullets. Journal of Forensic Sciences, vol. 4 (1), 34-50.


Bunch, Stephen G. ―Consecutive Matching Striation Criteria: A General Critique.‖ Journal of Forensic Sciences, Vol. 45 (5), Sept. 2000, pp. 955-962.

This paper critiques the Consecutive Matching Striation (CMS) approach to toolmark identification. The author discusses the practical and theoretical weaknesses of the approach, argues that it demands a statistical/probabilistic treatment of results - such as the use of Bayesian likelihood ratios - and also suggests much additional research is needed.

Burrard, G, The Identification of Firearms and Forensic Ballistics, Herbert Jenkins, Ltd., London, 1934, Reprinted Barnes & Company 1962 and Wolfe publishing 1990.

This textbook discusses and highlights the reliability of the microscopic comparative method.

Chumbly, L. Scott, et al, ―Validation of Tool Mark Comparisons Obtained Using a Quantitative, Comparative, Statistical Algorithm‖ Journal of Forensic Sciences, Volume 55, Number 4, July 2010, pp. 953-961.


Gunther, J.D., and Gunther, C.O., The Identification of Firearms, Wiley & Sons, Inc. 1935.

This textbook discusses and highlights the reliability of the microscopic comparative method.

Matthews, J. Howard. Firearms Identification: Volume I, II, and III, University of Wisconsin Press 1962.

63June 14, 2011 63

These three textbooks discuss and highlight the reliability of the microscopic comparative method.

Goddard, C.H., ―Scientific Identification of Firearms and Bullets‖, Journal of Criminal Law and Criminology, Vol. 16, No. 2, August 1926, pp 254-263. This article discusses the effectiveness of the microscopic comparative method. Moran, B. (2003). ―Toolmark Criteria for Identification: Pattern Match, CMS, or Bayesian?‖ AFTE Journal, Vol. 35, No. 4. pp 359-360. (Reprint from INTERfaces, Vol. 28, Nov-Dec 2001. pp 9-10.)

Abstract: The purpose of this discussion is to 1.) open a dialogue with both our Bayesian and non-Bayesian colleagues in the British Forensic Science Service on the topic of approaches to criteria for identification and conclusion giving in toolmark (and firearm) identification, 2.) to solicit some specific examples to the practical use of Bayes Rule in toolmark identification cases and 3.) to solicit feedback on more objective approaches to the interpretation of striated toolmarks such as consecutive matching striae (CMS).

Biasotti, A. and Murdock, J. (1984). ―Criteria for Identification or State of the Art of Firearms and Toolmark Identification.‖ AFTE Journal, Vol. 16, No. 4. pp 16-24

Authors lay out the current state of firearms identification and what is required to determine an identification. They also discuss the practical certainty of an identification vs. an absolute identification. The authors lay a foundational framework for more objective criteria in the formulation of an identification conclusion.

Miller, J. (2001). ―An Examination of the Application of the Conservative Criteria for Identification of Striated Toolmarks Using Bullets Fired from Ten Consecutively Rifled Barrels.‖ AFTE Journal, Vol. 33, No. 2. pp 125-132.

Abstract: The possibility of the reproduction of subclass characteristics between fired bullets is most likely to occur from barrels that are consecutively rifled. An evaluation of bullets fired from ten consecutively rifled barrels is made using the previously proposed conservative criteria for identification of striated toolmarks. The influence of any subclass characteristics on an identification and the validity of using the conservative criteria for identification as a criteria to determine an identification is examined.

Moran, B. (2001). ―The Application of Numerical Criteria for Identification in Casework Involving Magazine Marks and Land Impressions.‖ AFTE Journal, Vol. 33, No. 3. pp 41 – 46.

64June 14, 2011 64

Abstract: This paper addresses the evaluation of magazine marks and the magazine surfaces that produce them with regard to potential for subclass and individualizing characteristics. It also describes the practical use of numerical criteria in the evaluation of striated toolmarks in routine casework (magazine marks and rifling impressions) and it‘s significance in providing objective criteria for examining striated toolmarks having limited information such as magazine marks on fired cartridge cases.

Nichols, R. (2003). ―Consecutive Matching Striations (CMS): Its Definition, Study and Application in the Discipline of Firearms and Tool Mark Identification.‖ AFTE Journal, Vol. 35, No. 3. pp 298-306.

Abstract: The concept of consecutive matching striations (CMS) has been met with aggressive opposition and suspicion within the discipline of firearms and tool mark identification. It is believed that this is due to a lack of a fuller understanding as to its definition and application within the field. The purpose of this paper is to help resolve some of the persistent issues that critics of CMS have consistently presented throughout the years. To fulfill that purpose, this paper will articulate a definition of CMS that helps to demonstrate that it is not in conflict with what has been referred to as the traditional pattern matching approach, but is simply a means of describing the observed pattern. In addition, the paper will critically evaluate and summarize those articles that have had as their basis the intent to invalidate the conservative minimum criteria for identification. Finally, this article will address frequently expressed concerns in an effort to put them to a final rest.

65June 14, 2011 65

17. What research has been completed, if any, to determine if a threshold exists to assess when there is sufficient data to complete an examination?

AFTE Criteria For Identification Committee Report: ―Theory of Identification, Range of

Striae Comparison Reports and Modified Glossary Definitions-an AFTE Criteria For Identification Committee Report‖, AFTE Journal, Vol. 24, No. 3, July 1992, pp. 336-340.

Nichols (see Question 2 for Nichols Part I and Part II references) has summarized the scientific studies that allow us, assuming no subclass influence, to predict that: 1) the working surfaces of different tools produce discernibly different toolmarks even though some quality/quantity of microscopic agreement may be present (these toolmarks are referred to as known non-matches) and; 2) toolmarks produced by the same tool working surface (referred to as known matches) can be identified with one another and exhibit a greater quality/quantity of microscopic agreement than known non-matching toolmarks.

The references summarized by Nichols are examples of the extensive testing done in this area. As a result of these studies, AFTE formulated and adopted a Theory of Identification to explain the basic theory that allows opinions of common origin to be made in toolmark comparisons. The AFTE Theory of Identification, adopted in 1992, states:

1. The theory of identification as it pertains to the comparison of toolmarks enables opinions of common origin to be made when the unique surface contours of two toolmarks are in ―sufficient

agreement‖.

2. This ―sufficient agreement‖ is related to the significant duplication of random toolmarks as evidence by a pattern or combination of patterns of surface contours. Significance is determined by the comparative examination of two or more sets of surface contour patterns comprised of individual peaks, ridges and furrows. Specifically, the relative height or depth, width, curvature and special relationship of the individual peaks, ridges and furrows within one set of surface contours are defined and compared to the corresponding features in the second set of surface contours. Agreement is significant when it exceeds the best agreement demonstrated between toolmarks known to have been produced by different tools and is consistent with agreement demonstrated by toolmarks known to have been produced by the same tool. The statement that ―sufficient agreement‖ exists between two toolmarks

means that the agreement is of a quantity and quality that the

66June 14, 2011 66

likelihood another tool could have made the marks is so remote as to be considered a practical impossibility.

3. Currently the interpretation of individualization/identification is subjective in nature, founded on scientific principles and based on the examiner‘s training and experience.

In accordance with the AFTE Theory of Identification, and a commitment to standardization, AFTE has developed a specific range of conclusions possible when comparing toolmarks. As adopted in 1992, the range of conclusions was preceded by: ―The examiner is encouraged to report the objective observations

that support the findings of toolmark examinations. The examiner should be conservative when reporting the significance of these observations.‖ These two

statements were designed to give the examiner license to explain his or her reasoning for reaching his or her conclusions. These conclusions are based on a specific comparison of individual characteristics, having eliminated any possibility of subclass influence. They are:

1. Identification: Agreement of a combination of individual characteristics and all discernible class characteristics where the extent of agreement exceeds that which can occur in the comparison of toolmarks made by different tools and is consistent with the agreement demonstrated by toolmarks known to have been produced by the same tool.

2. Inconclusive:

a. Some agreement of individual characteristics and all discernible class characteristics, but insufficient for an identification.

b. Agreement of all discernible class characteristics without agreement or disagreement of individual characteristics due to an absence, insufficiency, or lack of reproducibility.

c. Agreement of all discernable class characteristics and disagreement of individual characteristics, but insufficient for an elimination.

3. Elimination: Significant disagreement of discernible class characteristics and/or individual characteristics.

4. Unsuitable: Unsuitable for examination.

67June 14, 2011 67

It is important to note that the word ―inconclusive‖ does not have to be included in

a laboratory report. Each of the subcategories under Inconclusive above were designed to stand alone, as conclusive findings. The word ―inconclusive‖ was

chosen simply because the three associative evidence statements, a), b), and c) are findings less conclusive than Identification.

Biasotti, A. (1959). A Statistical Study of the Individual Characteristics of Fired Bullets. Journal of Forensic Sciences, vol. 4 (1), 34-50.


Burrard, G. (1934). The Identification of Firearms and Forensic Ballistics. Herbert Jenkins, London.

The Gutteridge case is described where 1375 revolvers of the same make/model were compared with the suspect revolver. Six of the 1375 had similar irregular indentations on the breechfaces. Test firings from these six were distinctly different from each other and from the toolmarks on the ―crime‖ cartridge case, which matched test firings from the suspect revolver.

Miller J., and McLean M., Criteria for Identification of Toolmarks, AFTE Journal, Vol. 30, No. 1, Winter 1998, pp. 15-61.

Miller J., Criteria for Identification of Toolmarks, Part II, AFTE Journal, Vol. 32, No. 2, Spring 2000, pp. 116-131.

Miller J. and Neel M., Criteria for Identification of Toolmarks, Part III, AFTE Journal, Vol. 36, No. 1, Winter 2004, pp. 7-38.

Extensive three-part study on striated toolmarks contained on various caliber fired bullets was conducted by using a computer to correlate the KM and KNM striae groups of these test-fired specimens. These studies validated Biasotti‘s previous work that concluded consecutiveness of matching striae is more reliable than percent of matching striae. Additionally, these studies support the conclusions made by examiners using the conservative quantitative consecutive matching striae criteria authored by Biasotti and Murdock in 1997.

Smith, Erich. ―Cartridge Case and Bullet Comparison Validation Study with Firearms Submitted in Casework.‖ AFTE Journal, vol. 37 (2), Spring 2005, pp. 130-135.

This validation study was designed to test the accuracy of examinations by trained firearms examiners who use pattern recognition as a method for identification. Eight FBI examiners took the test that consisted of both bullets and cartridge cases. No false positives or false negatives were reported.

68June 14, 2011 68

Uchiyama T. (1988). A Criterion for Land Mark Identification. AFTE Journal, Vol. 20, No. 3, 236-251.

This article describes the examination process of firearms identification beginning with class characteristic agreement and followed by individual characteristic agreement. The use of a digital image processor is discussed as a viable counting method of lines on a bullet.

Uchiyama T. (1988). A Criterion for Land Mark Identification Using Rare Marks. AFTE Journal, Vol. 20, No. 3, 260-268.

In this paper, an example is presented for making a judgment of identity based on rare marks appearing on metal jacketed bullets. The significance level of the calculated probability estimates using this model is only moderately low.

69June 14, 2011 69

18. What research has been completed, if any, to determine the threshold for identification (individualization)?

STRIATED TOOLMARKS:

Quantitative Consecutive Matching Striae (CMS) Theory

There are a number of scientists in the firearm/toolmark identification community who have specifically researched the individuality of striated toolmarks based on the quantity of consecutively matching striae. These scientists approached this through the use of theoretical, mathematical, and empirical studies. By doing this, the examiner is able, from data gathered using the scientific method, to assign measurable weight to the qualitative component described in the AFTE Theory of Identification. This allows the examiner to supplement his/her training and experience with data that is scientifically defensible. The following references validate this approach:

GENERAL

1959 – Biasotti, A., A Statistical Study of the Individual Characteristics of Fired Bullets. Journal of Forensic Sciences. 1959 Jan; 4(1): 34-50.

This paper summarizes the study of the probability of occurrence of consecutive matching striae in land impressions from fired bullets in both match and nonmatch positions reported by the author in 1955. The findings are based on the author‘s research conducted between 1951 and 1955. Biasotti concluded

that, ―The most significant point of the data collected is the fact that 3

consecutive matching lines for lead bullets and 4 consecutive matching lines for metal-cased bullets appears to be the dividing line between data for same and different guns; and therefore, these critical series form the base line upon which the data for bullets from the same gun can be differentiated from the data for different guns.‖ Therefore, approximately 3 - 4 consecutive matching striae appeared to be the threshold number between a known match and known non-match.

MATHEMATICAL MODELS

1970 – Brackett, J., A Study of Idealized Striated Marks and Their Comparison Using Models. Journal of the Forensic Science Society. 1970; 10(1): 27-56.

Brackett published his study of various mathematical models that could be applied to the study of ―idealized‖ striated marks including geometric models,

number-based models, random number outcome models, and random number replica models. His purpose was to develop a theoretical basis for striated mark analysis that could be developed into mechanical (empirical) models that could

70June 14, 2011 70

be compared to toolmarks in order to obtain sufficient information to establish objective criteria for identity between two sets of toolmarks. He discovered from these different models that the development of a random number table was quite reliable in representing a striated toolmark model. Although his ideal striations were devoid of width, his work is significant in that he demonstrated the concept of consecutiveness alone to be a very powerful tool in deciphering coincidence from common associations. His ideal mathematical striated marks model was described as ―tedious‖, but it closely supported the empirical study of consecutive

matching striae conducted by and reported in 1959 by Biasotti. The author suggested that such models could be more practically used with the assistance of computers in the future.

1980 – Blackwell, R., Framan, E. Automated Firearms Identification System (AFIDS): Phase I. AFTE Journal 1980 Oct; 12(4): 11-37.

Blackwell and Framan published their research into the development of automated firearms identification systems. In their concern for establishing applicable criteria for developing computerized systems that could be objectively and reliably ―utilized to help establish a more factual basis for determining identity

and nonidentity between two pairs of fired bullets‖, the authors researched the literature. The results of this effort revealed that there was little in the literature that provided such objective criteria that could provide a ―universal factual basis

for establishing identity of a firearm‖ with the exception of the work of Biasotti reported in 1959. They observed ―Biasotti has conducted research which could

prove very useful to future developments in firearm identification‖. To investigate

this possibility, the authors conducted a simulation study of striated marks by applying Brackett‘s formulas and models, which they found reliable, and in

agreement with each other. Results of their work were found to be in general agreement with the results of Biasotti‘s empirical studies described in 1959. They

observed in the simulation study that ―the results substantiated Biasotti‘s

hypothesis and regardless of the phase relationship of one sequence with the other, the chance occurrence of consecutive matching lines exceeding those proposed by Biasotti did not occur.‖

1981 – Deinet, W., Studies of Models of Striated Marks Generated by Random Processes. Journal of Forensic Sciences 1981 Jan; 26(1): 35-50.

Dienet described his use of computer-aided studies of the degree of similarity of striated markings. Dienet addressed the problem that ―a high degree of similarity between two sets of marks is not sufficient to identify a tool if it is highly probable that the similarity may occur by chance‖. With this in mind, Dienet defined this

problem in the form of the following question: ―Given two patterns that are similar

71June 14, 2011 71

to a certain degree, what is the probability that such a similarity, or an even greater one, occurs at random?‖ He attempted to answer this question by

constructing models based on assumptions that: ―1) random processes generate

the patterns on the tools; 2) different patterns on tools produced by the same machine are independent; and 3) the probability of the occurrence of a line is independent of its position‖. His paper described the calculation of probability of

random occurrence of matches using actual striated toolmarks using blades of 20 shears that were ground to produce random imperfections that were then used to produce toolmarks in lead. Two sets of marks were produced with each tool and a 1.2 mm portion of each pattern was photographed and scanned into the computer. The images were prepared and evaluated with respect to line position. The ―Digitized image data on 40 grinding marks were fed into a minicomputer,

and the position values of the lines were determined semi-automatically.‖

Idealized models were defined for an objective comparison of striated marks and then applied to the grinding mark data. Necessary conditions of the models were tested by comparing them with actual measured properties of the marks. Three different probability theory models were examined including combinatorial model, a renewal theory model, and binomial function fit model. The results of the model calculations were presented and the properties of the models were discussed. Each model presented certain strengths and weaknesses in fulfilling the above three requirements but were not entirely ideal. However, he also concluded that ―numerical values computed with the aid of models permit an evaluation of the

degree of similarity‖ and ―for automation of pattern comparisons a preselection is possible, but any probability-related statements require additional studies and examinations‖

1988 – Uchiyama, T., A Criterion for Land Mark Identification. AFTE Journal 1988 Jul; 20(3): 236-251.

Uchiyama sought to develop criteria for identification of land impressions using probability theory and also developed a significance level associated with this approach. This was developed because of his observations that neither the total number of matching striae nor the percentage of matching striae was sufficient to establish identity. His significance level approach provided an evaluation of goodness of fit and primarily provided the probability of an accidental or random match of striae. He developed the probability equation based on actual fired bullets. Using his significance level of evaluation he observed that consecutiveness of matching striae played a principle part in indicating the identity of bullets fired from the same gun.

1992 – Uchiyama, T., The Probability of Corresponding Striae in Toolmarks. AFTE Journal 1992 Jul; 24(3): 273-290.

72June 14, 2011 72

Uchiyama‘s study provides an estimate of the maximum number of consecutively

corresponding lines that might be expected given the considerations of: 1) striae density; 2) critical coincidence ratio (CCR – a method of quantitatively representing how well two lines match relative to width, a range of zero to 1 with 1 being a perfect match); and 3) striae width. He developed a computer model to generate and compare striated toolmarks using these considerations. His model demonstrated that when the widths of lines were varied (as might be expected in actual striated toolmarks), the number of coincidental consecutive matching striae decreased with increasing deviations in striae width. From this experimentation he demonstrated that the additional influence of line width, in the critical evaluation of consecutive agreement, had significant additional influence on the maximum number of matching consecutive striae. When the conditions of his variables were such that the coefficient of variation for striae width was 0.9 and the CCR value was 0.8, the maximum number of coincidental consecutive matching striae was approximately 3 – 4. These results of a maximum of 3 to 4 consecutive lines that would represent a known nonmatch very closely agreed with Biasotti's original empirical work.

1997 – Biasotti, A., Murdock, J., in Faigman DL, Kaye DK, Saks MJ, Sanders J, editors. Modern Scientific Evidence: The Law and Science of Expert Testimony. St. Paul: West, 1997 – Chapter 23, Firearms and Toolmark Identification

This work includes, for the first time, a proposed conservative numerical criterion based on counting runs of consecutive matching striae. The purpose of this proposal was to offer a standard that could be used to distinguish between a striated toolmark identification and non-identification. The developed conservative quantitative criteria is as follows:

In three dimensional toolmarks when at least two different groups of at least three consecutive matching striae appear in the same relative position, or one group of six consecutive matching striae are in agreement in an evidence toolmark compared to a test toolmark.

In two dimensional toolmarks when at least two groups of at least five consecutive matching striae appear in the same relative position, or one group of eight consecutive matching striae are in agreement in an evidence toolmark.

For these criteria to apply, however, the possibility of subclass characteristics must be ruled out.

Since this chapter was first published in 1997, there have been a number of studies which have included an evaluation, using quantitative consecutive matching striae, of the numerical criteria for the identification of striated toolmarks proposed above by Biasotti and Murdock. No known non-matching (two or three

73June 14, 2011 73

dimensional) toolmarks were found in these studies which exhibited agreement in excess of the proposed Biasotti-Murdock criteria. These studies, as well as a description of a survey and training in the use of quantitative CMS as a criterion for the identification of striated toolmarks are described on pages 702 through 707 of the following reference:

Biasotti, A., Murdock, J., Moran, B., Firearms and Toolmark Identification. Chapter 35, Vol.4, pp 645-723 in Modern Scientific Evidence: The Law and Science of Expert Testimony (Faigman DL, Kay DK, Saks MJ, Sanders, J, Chen EK., eds, 2009-2010), St. Paul: Thompson-West.

A footnote on pages 681 through 685 in the above reference discusses the consideration of subclass toolmark influence. Toolmarks of any kind cannot be identified if subclass influence is present.

IMPRESSED TOOLMARKS:

Biasotti, A., Murdock, J., Moran, B., Firearms and Toolmark Identification. Chapter 35, Vol.4, pp 645-723 in Modern Scientific Evidence: The Law and Science of Expert Testimony (Faigman DL, Kay DK, Saks MJ, Sanders, J, Chen EK., eds, 2009-2010), St. Paul: Thompson-West.

Research conducted thus far by Biasotti, Murdock, and Moran, indicates that the practical probabilities limits in known non-matches for impression toolmarks similar to those found for striated toolmarks. Some progress has been made in developing quantitative criteria for the identification of compression toolmarks. Following Stone‘s publication of a theoretical model for the mathematical evaluation of well defined types of impressed toolmarks, Collins used and evaluated Stone‘s model while performing an empirical study of twenty worn

hammer faces. His preliminary results show that combinations of even low numbers of simple impressed defects are, on a practical level, quite discriminating. However, more research is needed involving very fine, high density, randomly distributed individual impression characteristics, viewed two dimensionally, before definitive practical probability limits can be stated confidently.

Stone, R. ―How Unique are Impressed Toolmarks?‖ AFTE Journal. Vol. 35, No. 4, Fall 2003. pp. 376-383.

The comparison model developed by Stone offers a springboard upon which those interested in studying the occurrence of impressed contours and establishing the basis for a quantifiable identification criterion of impressed toolmarks may now do so.

74June 14, 2011 74

Collins, E. ―How Unique are Impressed Toolmarks: An Empirical Study of 20 Worn

Hammer Faces.‖ AFTE Journal. Vol. 37, No. 4, Fall 2005. pp. 252-295.

Collins tested the validity of Stone‘s theories on the statistical uniqueness of

impressed toolmarks through the empirical examination of the defects observed on the faces of twenty hammers that had been subjected to various degrees of wear and abuse through normal use. These examinations were carried out under controlled conditions that would simulate those used in practical casework. The results of this study led to a re-evaluation of Stone‘s work and a modification of

related formulae. The revised formulae were used to calculate practical but conservative probabilities associated with impressed toolmarks using the data collected from the hammers in the study.

75June 14, 2011 75

19. Does research exist which demonstrates that criteria for identifications (individualization) vary with method of manufacture or type of tool?

The AFTE Theory of Identification, adopted by AFTE in 1992, is intended to be universal, accounting for all known methods of manufacture. Biasotti, A.A., ―Statistical Analysis of Bullet Comparison (Preliminary, Unpublished, Report for Masters Thesis)‖, prepared for Criminology 299 course, University of California Berkeley, June 1, 1951.

In this report, Biasotti described how the purpose of his research was to arrive at a concept of probability for bullet comparison based on the relative frequency of occurrence of common characteristics (e.g. consecutive series of matching lines) between bullets fired from the same gun and bullets fired from similar guns of the same caliber. He went on to describe how 1) the purpose of his study was to give the Criminalist some basis for establishing what constitutes a match and what degree of probability can be attached and 2) the relative frequency of occurrence of consecutive series as will be compiled in this study could be used directly as empirical findings without deriving any concept of probability; but if this were done, the data would apply only to the particular type of gun and ammunition used in this particular study. Then similar studies would have to be conducted for every gun of different manufacture of the same caliber and for every different type of ammunition to be significant. Biasotti completed his Masters thesis, ―Bullet Comparison – A Study of Fired Bullets, Statistically Analyszed, ― in February 1955 and published his findings in the Journal of Forensic Sciences, Vol 4., No. 1, Jan. 1 1959, pp. 34-50. In 1997, forty six years after stating in his 1951 student progress report, Biasotti (in collaboration with John Murdock) decided that sufficient research had been done on both firearms and non-firearms toolmakrs to justify publishing the conservative quantitative criteria for striated toolmarks. (Listed under Question 18). The criteria was published in reference (A) below and remains the same in the current edition of Modern Scientific Evidence (reference (B) below). (A) Biasotti, A., Murdock, J., Chapter 23, ―Firearms and Toolmark Identification‖ from

Modern Scientific Evidence: The Law and Science of Expert Testimony, Vol. 2, West Pub. Co., 1997, pp. 124-155; and (B) Biasotti, A., Murdock, J., Moran, B., Firearms and Toolmark Identification. Chapter 35, Vol.4, pp 645-723 in Modern Scientific Evidence: The Law and Science of Expert Testimony (Faigman DL, Kay DK, Saks MJ, Sanders, J, Chen EK., eds, 2009-2010), St. Paul: Thompson-West.

76June 14, 2011 76

20. What studies have been performed to determine error rates in firearm and toolmark analyses? What studies have been performed to determine examiner error rates? What research exists which identifies rates for misidentifications and false exclusions?

See responses to Question #21.

77June 14, 2011 77

21. Do studies exist which demonstrate how often false negatives (e.g. a non-match is declared, when they match) are reported?

Error rates in actual casework are difficult to assess due to a lack of ―ground truth.‖ Proficiency tests are not administered in a consistent, controlled manner but do provide a large amount of data. Validity tests employ various levels of blindness and control, but are usually designed to create a ―worst case scenario‖ in which consecutively machined firearms and tools are used. Proficiency Tests Crime Laboratory Proficiency Testing Results, 1978*1991, II: Resolving Questions of Common Origin, Journal of Forensic Sciences, Vol. 40, No. 6, Nov. 1995, pp.1009-29.

Article examined the origins of crime laboratory proficiency testing and the performance of laboratories in the identification and classification of common types of physical evidence. Part II reviews laboratory proficiency in determining if two or more evidence samples shared a common source. Parts I and II together review the results of 175 separate tests issued to crime laboratories over the period 1978 to 1991.

CTS 1978 - 1991, Stephen Bunch summary and slight revision of Peterson & Markham F/T results. Internet Source: www.swggun.org/resources/admissibility/prof_test results081603.pdf Proficiency Test Results from Peterson and Markham Article - Firearms Source: ―Crime Laboratory Proficiency Test Results, 1978-1991, II: Resolving Questions of Common Origin,‖ Journal of Forensic Sciences, Vol. 40, No. 6, November 1995, pp. 1009 -1029. (12 separate tests involving between 42 and 173 laboratories.)

From Table 8, page 1019: Total comparisons = 2106 False identifications = 12 False eliminations = 17 True identification conclusions = 905 True elimination conclusions = 954 True identifications judged inconclusive = 43 True eliminations judged inconclusive = 175 Total true identifications = 905 + 43 + 17 = 965 Total true eliminations = 954 + 175 + 12 = 1141 Total identification conclusions offered = 905 + 12 = 917 Total elimination conclusions offered = 954 + 17 = 971 Total inconclusives = 43 + 175 = 218

Data Analysis – Firearms Test Sensitivity = true IDs offered/true IDs = 905/965 = 93.78% Test Specificity = true eliminations offered/true eliminations = 954/1141 = 83.61%

78June 14, 2011 78

http://www.swggun.org/resources/admissibility/prof_test%20results081603.pdf

False positive error rate (false or mis-identifications) = false positive responses/total true eliminations = 12/1141 = 1.05% False negative error rate (false or mis-eliminations) = false negative responses/total true identifications = 17/965 = 1.76% Inconclusive rate = 218/2106 = 10.35%

Proficiency Test Results from Peterson and Markham article - Toolmarks Source: ―Crime Laboratory Proficiency Test Results, 1978-1991, II: Resolving Questions of Common Origin,‖ Journal of Forensic Sciences, Vol. 40, No. 6, November 1995, pp. 1009 - 1029. (12 separate tests involving between 72 and 163 laboratories.)

From Table 13, page 1024: Total comparisons = 1961 False identifications = 30 False eliminations = 44 True identification conclusions = 646 True elimination conclusions = 755 + 53 + 44 = 852 True identifications judged inconclusive = 83 + 48 = 131 True eliminations judged inconclusive = 258 Total true identifications = 646 + 44 + 48 = 821 Total true eliminations = 852 + 30 + 258 = 1140 Total identification conclusions offered = 646 + 30 = 676 Total elimination conclusions offered = 852 + 44 = 896 Total inconclusives = 83 +258 + 48 = 389

Under toolmarks, the authors include a category of ―unjustified exclusions.‖ An example: two wires cut by different areas on the cutting edge of a single pair of wire cutters was marked by a participant as an elimination. While this mistake would be understandable if one merely considers microscopic correspondence and ignores the larger picture, it was properly categorized as an unjustified exclusion, and counted here as a false negative. In other cases, however, the responses were correct from a scientific perspective (only false positives and false negatives matter), but incorrect from a training and quality assurance perspective. For my purposes, the scientific propositions trump quality assurance considerations, and thus the remaining ―unjustified exclusions‖ were counted as correct responses. CTS 1992 – 2005, F/T results revisions by Douglas Murphy, Presentation at 2010 AFTE Training Seminar (also at www.swggun.org/resources/docs/CTSErrorRates.pdf) CTS Error Rates: 1992 – 2005 Firearms False Positive = 1.5% Firearms False Negative = 0.5% Toolmark False Positive = 1.7% Toolmark False Negative = 1.6%

The two preceding summaries were produced by applying standard error rate calculation methods to results reported by CTS, to include false positive and false negative rates, sensitivity and specificity.

79June 14, 2011 79

http://www.swggun.org/resources/docs/CTSErrorRates.pdf

Validity Tests Teale, Popular Science Monthly, February 1932, p.213

The author reports on studies by Calvin Goddard of markings produced by six consecutively manufactured firing pins and four consecutively manufactured gun barrels. These studies appear to be the first documented attempt to test the firearm examiner‘s ability to distinguish between consecutively manufactured tools. This is the traditional ―worst case scenario‖ testing that has been conducted by firearm and toolmark examiners to validate the ability to individualize tools. With regard to the firing pin study, Teale relates the following: ―To prove his case, Col. Goddard told me he sent to the factory where for the suspect‘s gun was manufactured and obtained half a dozen firing pins made on the same machine one after another‖ and ―the differences are apparent in the confirmation of the tips of six firing pins made successively on the same machine. Contact with the cartridge caps will leave identifying imprints.‖ With regard to the study of markings produced on consecutively manufactured gun barrels, Teale notes that ―Not long ago at the Springfield Armory, in Massachusetts, bullets were fired through four rifles that had been made one after the other on the same machine. The Markings on the bullets were so different that each bullet could be traced to the gun that fired it.‖

A short description of each of the following articles, see the SWGGUN ARK, Error Rates and Power Point slide #62 in SWGGUN ARK- Appendix I. Internet Source: www.swggun.org/swg/index.php?option=com_content&view=article&id=6:error-rate-resources&catid=9:ark&Itemid=18 Brundage, David J. ―The Identification of Consecutively Rifled Gun Barrels.‖ AFTE Journal, Vol. 30, No. 3, Summer, 1998, pp. 438-444. DeFrance, Charles S. and Michael VanArsdale. ―Validation Study of Electrochemical Rifling.‖ AFTE Journal, Vol. 35, No. 1, Winter, 2003, pp. 35-37. Fadul, T.G., ―An Empirical Study to Evaluate the Repeatability and Uniqueness of Striations/Impressions Imparted on Consecutively Manufactured Glock EBIS Gun Barrels‖, AFTE Journal, Volume 43, Number 1, Winter 2011, Pp. 37-44. Hamby J. E., Brundage D. J. , Thorpe J. W., ―The Identification of Bullets Fired from 10 Consecutively Rifled 9mm Ruger Pistol Barrels: A Research Project Involving 507 Participants from 20 Countries‖, AFTE Journal, Volume 41, Number 2, Spring 2009, pp. 99-110. Smith E., ―Cartridge Case and Bullet Comparison Validation Study with Firearms Submitted in Casework.‖ AFTE Journal, vol. 37 (2), Spring 2005, pp.130-135.

80June 14, 2011 80

http://www.swggun.org/swg/index.php?option=com_content&view=article&id=5:testability-of-the-scientific-principle&catid=9:ark&Itemid=18

http://www.swggun.org/swg/index.php?option=com_content&view=article&id=5:testability-of-the-scientific-principle&catid=9:ark&Itemid=18

Bunch, S.G. and Murphy D.P.. ―A Comprehensive Validity Study for the Forensic Examination of Cartridge Cases.‖ AFTE Journal, Vol. 35, No. 2, Spring 2003, pp. 201-203. Giroux B. N., ―Empirical and Validity Study: Consecutively Manufactured Screwdrivers‖, AFTE Journal, Volume 41, Number 2, Spring, 2009, Pp. 153-158. Lyons, D. J., ―The Identification of Consecutively Manufactured Extractors‖, AFTE Journal, Volume 41, Number 3, Summer, 2009, Pp.246-256. Thompson, Evan and R. Wyant, ―Knife Identification Project (KIP),‖ AFTE Journal, Vol. 35 (4), Fall 2003, Pp. 366 – 370. Christensen AM, Sylvester AD., Physical Matches of Bone, Shell and Tooth Fragments: A Validation Study. Journal of Forensic Sciences, 2008;53, Pp.694-698 Orench, Jose A., ―A Validation Study of Fracture Matching Metal Specimens Failed in Tension,‖ AFTE Journal, vol. 37 (2), Spring 2005, pp. 142-149. Grzybowski, R., Miller, J., Moran, B., Murdock, J., Nichols, R., and R. Thompson. ―Firearm/Toolmark Identification: Passing the Reliability Test Under Federal and State Evidentiary Standards.‖ AFTE Journal, vol. 35 (2), Spring 2003, pp. 209-241.

There is a particularly good discussion, with references listed below, of error rate on page 216 through 230 of this article. This description includes reported summaries of firearm and non-firearm proficiency tests, what is done in most forensic laboratories to ensure that the results of individual cases are correct as reported, and the error rate of individual examiners.

References for above article: Biasotti, A., Murdock, J., Moran, B., Firearms and Toolmark Identification. Chapter 35, Vol.4, pp 645-723 in Modern Scientific Evidence: The Law and Science of Expert Testimony (Faigman DL, Kay DK, Saks MJ, Sanders, J, Chen EK., eds, 2009-2010), St. Paul: Thompson-West. Moenssens, A., ―Meeting the Daubert Challenge to Handwriting Evidence – Preparing for a Daubert Hearing‖, abstract of a talk given at the Second Annual Symposium on the Forensic Examination of Questioned Documents at Albany, N.Y. on June 18, 1999. (see www.forensic-evidence.com) An earlier version of this also appears in the October, 1999 issue of the Forensic Science Communications, a peer reviewed quarterly journal published on the Internet by FBI Laboratory personnel (see http://www.fbi.gov/programs/lab/fsc/current/index.htm)

81June 14, 2011 81

http://www.forensic-evidence.com/

http://www.fbi.gov/programs/lab/fsc/current/index.htm

Peterson, J., and Markham, P., ―Crime laboratory testing results, 1978 – 1991, II: Resolving Questions of Common Origin.‖ Journal of Forensic Science, 1995: 40(6): 1009-1029.

82June 14, 2011 82

22. Are there studies in toolmarks that identify what information/circumstances may bias an examiner's conclusion?

Refer to answer under Question 23.

83June 14, 2011 83

23. Are there specific studies showing a difference in rate of inconclusive versus conclusive as a result of “contextual bias” information?

The following articles largely deal with fingerprints. But, to answer questions 22 & 23, this is the best information we have due to the close similarity to how fingerprint examiners and firearm examiners perform their tasks. These articles deal with fingerprints, but the types of influences present to a fingerprint examiner can generally be characterized as the same for a firearm examiner; however, please do not assume that if firearm and toolmark examiners were faced with the same induced biases that similar mistakes would occur. No such studies have been conducted for firearms and toolmark evidence, but they probably should be conducted.

Budowle, B., et al. (2009). ―A Perspective on Errors, Bias, and Interpretation in the Forensic Sciences, and Direction for Continuing Advancement.‖ Journal of Forensic Sciences, Vol. 54, No. 4. Pp 798-809.

Abstract: The forensic sciences are under review more so than ever before. Such review is necessary and healthy and should be a continuous process. It identifies areas for improvement in quality practices and services. The issues surrounding error, i.e., measurement error, human error, contextual bias, and confirmatory bias, and interpretation are discussed. Infrastructure is already in place to support reliability. However, more definition and clarity of terms and interpretation would facilitate communication and understanding. Material improvement across the disciplines should be sought through national programs in education and training, focused on science, the scientific method, statistics, and ethics. To provide direction for advancing the forensic sciences a list of recommendations ranging from further documentation to new research and validation to education and to accreditation is provided for consideration. The list is a starting point for discussion that could foster further thought and input in developing an overarching strategic plan for enhancing the forensic sciences.

Koehler, J. ―Fingerprint Error Rates and Proficiency Tests:What They Are and Why They Matter.‖ Citation: 59 Hastings L.J. 1077 2007-2008.

Introduction text of article: When a fingerprint examiner declares a match between a print from a known source and a latent print recovered from a crime scene,' his word may seal a defendant's fate like no other form of evidence save, perhaps, DNA. At trial the fingerprint examiner will offer little in the way of data, statistical tests, or uncertainty. Instead, he will say that latent print could only have been made by the source of the known print, that he is 100% certain, that he has never erred, and that the method he used to make this and other identifications has an error rate of zero.' In recent years, the broader scientific community has objected to this form of testimony. Critics charge that fingerprint analysis lacks an empirical foundation and that examiners make exaggerated claims that are likely to mislead jurors.

84June 14, 2011 84

Dror, I., Charlton, D., and Peron, A., ―Contextual Information Renders Experts Vulnerable to Making Erroneous Identification‖, Forensic Science International 2006, 156: 74-78.

Abstract: We investigated whether experts can objectively focus on feature information in fingerprints without being misled by extraneous information, such as context. We took fingerprints that have previously been examined and assessed by latent print experts to make positive identification of suspects. Then we presented these same fingerprints again, to the same experts, but gave a context that suggested that they were a no-match, and hence the suspects could not be identified.Within this new context, most of the fingerprint experts made different judgements, thus contradicting their own previous identification decisions. Cognitive aspects involved in biometric identification can explain why experts are vulnerable to make erroneous identifications.

Dror, I., Charlton, D., Hind, S. and Peron, A., ―When Emotions Get the Better of Us: The Effect of Contextual Top-Down Processing on Matching Fingerprints‖, Applied Cognitive Psychology 2005, 19: 799-809.

Abstract: Twenty-seven participants made a total of 2,484 judgments whether a pair of fingerprints matched or not. A quarter of the trials acted as a control condition. The rest of the trials included top-down influences aimed at biasing the participants to find a match. These manipulations included emotional background stories of crimes and explicitly disturbing photographs from crime scenes, as well as subliminal messages. The data revealed that participants were affected by the top-down manipulations and as a result were more likely to make match judgments. However, the increased likelihood of making match judgments was limited to ambiguous fingerprints. The top-down manipulations were not able to contradict clear non-matching fingerprints. Hence, such contextual information actively biases the ways gaps are filled, but was not sufficient to override clear bottom-up information.

Dror, I. and Rosenthal, R., ―Meta-analytically Quantifying the Reliability and Biasability of Forensic Experts‖, Journal of Forensic Science July 2008, 53(4): 900-903. Abstract: In this paper we employ meta-analytic procedures and estimate effect sizes indexing the degree of reliability and biasability of forensic experts. The data are based on within-expert comparisons, whereby the same expert unknowingly makes judgments on the same data at different times. This allows us to take robust measurements and conduct analyses that compare variances within the same experts, and thus to carefully quantify the degree of consistency and objectivity that underlie expert performance and decision making. To achieve consistency, experts must be reliable, at least in the very basic sense that an expert makes the same decision when the same data are presented in the same circumstances, and thus be consistent with themselves. To achieve objectivity, experts must focus only on the data and ignore irrelevant information, and thus be unbiasable by extraneous context. The analyses show that experts are not

85June 14, 2011 85

totally reliable nor are they unbiasable. These findings are based on the fingerprint expert‘s decision making, but because this domain is so well established, they apply equally well (if not more) to all other less established forensic domains. Dror, I. and Charlton, D. ―Why Experts Make Errors‖, Journal of Forensic Identification 2006, 56(4): 600-616.

Abstract: Expert latent fingerprint examiners were presented with fingerprints taken from real criminal cases. Half of the prints had been previously judged as individualizations and the other half as exclusions. We re-presented the same prints to the same experts who had judged them previously, but provided biasing contextual information in both the individualizations and exclusions. A control set of individualizations and exclusions was also re-presented as part of the study. The control set had no biasing contextual information associated with it. Each expert examined a total of eight past decisions. Two-thirds of the experts made inconsistent decisions. The findings are discussed in terms of psychological and cognitive vulnerabilities.

Dutton, G. (1998) ―The Importance of Being Impartial.‖ AFTE Journal 30(3): 523-526.

Text from Article: Most firearm examiners around the globe are paid their wage by their respective governments, whether this is at a local, state, or federal authority. Although this means examiners work for the prosecution "side" in the investigation of crime, it does not mean that the evidence examiners give should be weighted towards the prosecution case. The testimony as expert witnesses must be absolutely impartial. The firearms examiner that is not impartial is doing the field a great disservice and subject to possible ramifications from the AFTE code of ethics.

Gianelli, P. (2007). ―Confirmation Bias.‖ Criminal Justice, Vol. 22. pp 60-61. The author discusses the concepts of motivational and cognitive bias and how it relates to forensic examinations.

Hodge, E. (1988). ―Guarding Against Error.‖ AFTE Journal, Vol. 20, No. 3. pp 290-293.

The author discusses potential errors and omissions in casework. He further discusses five ways to reduce error: training, case organization, the removal of psychological pressures, checking your work, and using the partner system.

Koppl, R. and Whitman, G. (2010). ―Rational Bias in Forensic Science.‖ Law, Probability & Risk, Vol. 9. pp 69-90.

Abstract: The current organization of forensic science induces biases in the conduct of forensic science even if forensic scientists are perfectly rational. Assuming forensic examiners are flawless Bayesian statisticians helps us to

86June 14, 2011 86

identify structural sources of error that we might otherwise have undervalued or missed altogether. Specifically, forensic examiners‘ conclusions are affected not just by objective test results but also by two subjective factors: their prior beliefs about a suspect‘s likely guilt or innocence and the relative importance they attach to convicting the guilty rather than the innocent. The authorities—police and prosecutors—implicitly convey information to forensic examiners by their very decision to submit samples for testing. This information induces the examiners to update their prior beliefs in a manner that results in a greater tendency to provide testimony that incriminates the defendant. Forensic results are in a sense ‗contaminated‘ by the prosecution and thus do not provide jurors with an independent source of information. Structural reforms to address such problems of rational bias include independence from law enforcement, blind proficiency testing and separation of test from interpretation.

Paust, J. (1978). ―Dum-Dum Bullets, Law, and ‗Objective‘ Scientific Research: The Need for a Configurative Approach to Decision‖, Jurimetrics, Vol. 18. pp 68-278.

Text from Article: An examination of an article in Science magazine by Kenneth Hammond and Leonard Adelman on "Science, Values, and Human Judgment" seems to demonstrate, contrary to the views of its authors, that science, values and bias are necessarily integrated. In an attempt to "objectify" some recent research on the societal characteristics of hollow-point (or "dum-dum") handgun bullets, Messrs. Hammond and Adelman have defined and applied a seriously flawed, one-sided approach to decisions about weapons effects. They have allowed hidden choice points (about values, bias, what should be measured and how it should be measured) to impair their attempts at scientific research, objective inquiry and unprejudiced conclusions. In short, the values and biases of the scientists have been built into the definitional framework for their scientific inquiry and, thus, into the quantified results.

Risinger, D., Saks, M., Thompson, W., and Rosenthal, R. (2002). ―The Daubert/Kuhmo Implications of Observer Effects in Forensic Science: Hidden Problems of Expectation and Suggestion‖, California Law Review January 2002, 90(1):1-56.

The authors discuss the concept of observer effects in general as well as in forensic science as a whole. They also discuss ways in which to reduce observer effects.

87June 14, 2011 87

24. Does research exist which uses class characteristics to describe the relative rarity of source firearms based on the population of firearms that can be estimated?

No. Although the FBI has a General Rifling Characteristics (GRC) database, it is intended as an investigative tool and does not include population information.

88June 14, 2011 88

25. What statistical research has been conducted and applied to firearm and toolmark examinations? What statistical models for firearms and toolmarks have been published?

The following 14 articles have been abstracted elsewhere in this document in the response to other questions: Neel, Michael (et al.), A Comprehensive Statistical Analysis of Striated Tool Mark Examinations Part 1: Comparing Known Matches and Known Non-Matches, AFTE

Journal, Volume 39, No. 3, Summer, 2007, pp 174-196

Faden, D. (et al.), Statistical Confirmation of Empirical Observations, AFTE Journal, Volume 39, Number 3, Summer 2007, 211-220 Bachrach, Ben. ―Development of a 3D-Based Automated Firearms Evidence Comparison System.‖ Journal of Forensic Sciences, Vol. 47, No. 6, November, 2002, pp. 1253-1264. Biasotti, Alfred A. ―A Statistical Study of the Individual Characteristics of Fired Bullets.‖ Journal of Forensic Sciences, Vol. 4, No. 1, January, 1959, pp. 34-50. Intelligent Automation, Incorporated, ―A Statistical Validation of the Individuality of Guns Using High Resolution Topographical Images of Bullets ‖, National Institute of Justice Grant #2006-DN-BX-K030, October, 2010 Howitt D., Tulleners F., ―A Calculation of the Theoretical Significance of Matched Bullets,‖ Journal of Forensic Sciences, Volume 53, Number 4, July 2008, pp. 868-875. Neel M., and Wells M., ―A Comprehensive Statistical Analysis of Striated Tool Mark Examinations Part I: Comparing Known Matches and Known Non-Matches‖, AFTE Journal, Volume 39, No. 4, Summer 2007, pp. 176-198. May L., ―Identification of Knives, Tools and Instruments‖, Journal of Police Science , Volume 1, No. 3, 1930, pp. 247-248. Deinet, Werner. ―Studies of Models of Striated Marks Generated by Random Processes.‖ Journal of Forensic Sciences, Vol. 26 (1), Jan., 1981, pp. 35-50. Stone, Rocky, ―How Unique are Impressed Marks,‖ AFTE Journal, Vol. 35, No.4, Fall 2003, pp. 376-383. Collins, Eric R., ―How ―Unique‖ Are Impressed Toolmarks? – An Empirical Study of 20 Worn Hammer Faces,‖ AFTE Journal, Vol. 37 (4), Fall 2005, pp. 252-295.

89June 14, 2011 89



Chumbly, L. Scott, et al, ―Validation of Tool Mark Comparisons Obtained Using a Quantitative, Comparative, Statistical Algorithm‖ Journal of Forensic Sciences, Volume 55, Number 4, July 2010, pp. 953-961. Bachrach B., Jain A., Jung S., and Koons R.D., ―A Statistical Validation of the Individuality and Repeatability of Striate Tool Marks: Screwdrivers and Tongue and Groove Pliers‖, Journal of Forensic Sciences, Volume 55, Number 2, March 2010, pp 348-357. Bacharach, B., ―Statistical Validation on the Individuality of Tool Marks Due to the Effect of Wear, Environment Exposure and Partial Evidence‖, NIJ/NCJRS Document #227929, August, 2009. Wever, G., et al. (2011). ―A Comprehensive Statistical Analysis of Striated Tool Mark Examinations Part 2: Comparing Known Matches and Known Non-Matches using Likelihood Ratios.‖ AFTE Journal, Vol. 43, No. 2. pp 137-145.

Abstract: A potential model for increasing the objectivity in the interpretation of toolmarks is explored using consecutively matching striae (CMS) and Bayesian inference. Given the nature of the data, standard statistical thinking suggests that Bayesian inference is likely to be the most powerful method of interpretation. The unavoidable paucity of data for high CMS runs for the known non-match condition is handled using a small advance in modelling. The resulting likelihood ratios show some, but incomplete separation between the known match and known non-match conditions. Although promising, the resulting incomplete separation between known match and known non-match is thought to represent limitations of the CMS summary of the complete pattern and limitations of the modeling used.

Buckleton, J., et al. (2005). ―An Exploratory Bayesian Model for Firearm and Tool Mark Interpretation.‖ AFTE Journal, Vol. 37, No. 4. pp 352-361.

Abstract: An exploratory Bayesian model for the interpretation of striated tool marks has been developed. This model is based on a consecutive matching striae (CMS) summary of the visual image of a striated tool mark comparison. The results of applying this model to two data sets are encouraging.

Champod, C., et al. (2003). ―Firearm and Tool Marks Identification: The Bayesian Approach.‖ AFTE Journal, Vol. 35, No. 3. pp 307-316.

Abstract: We have read with interest the paper written recently by Bruce Moran in the AFTE Journal and the exchange of letters in INTERface. The author is a respected commentator in the field of firearm and tool mark examination. He challenges readers to demonstrate the practical applicability of Bayes theorem in firearms and tool mark examination. Mr Moran discusses an «objective» criterion such as consecutive matching striae (CMS) and it‘s applicability to firearms and

90June 14, 2011 90

tool marks examination. Notably he suggests the adoption of the consecutive matching striae numerical criterion for identification which (in the author‘s words) ―establishes a clear threshold between identification and non-identification in striated tool marks, because it is based on scientific principles and sound empirical research‖. He also declares that ―Bayes theorem is not appropriate as an effective means to interpret the identification science‖.

Deinet, W., et al. (2007). ―Comments on the Application of Theoretical Probablity Models including Bayes Theorem in Forensic Science Relating Firearm and Tool Marks,‖ AFTE Journal, Vol. 39, No. 1. Pp 4-7.

Abstract: The comparison of firearm and tool marks is of great importance in forensic science. Examiners magnify the marks using a microscope and decide, on the basis of their experience, whether the degree of similarity is sufficient for an identification. One aim of scientific research is to develop probability models to make objective and reproducible results possible. In this paper comments are made on the application of Bayes‘ rule. Additionally, the extent to which an objective description of forensic science evidence is possible by the application of probability models is discussed.

Taroni, F., et al. (1996). ―Statistics: A Future in Tool Marks Comparison?‖ AFTE Journal, Vol. 28, No. 4. pp 222-232.

This article discusses the uses of statistics and the possible use of statistics by the field of Firearm and Toolmark Identification.

Deschenes, M., et al. (1995). ―Statistics and Tool Marks Comparisons.‖ AFTE Journal, Vol. 27, No. 2. pp 140-141.

This paper suggests how statistics can be utilized in toolmark comparisons but also the limitations of using them.

Biedermann, A., Bozza, A., and Taroni, F. (2008). ―Decision Theoretic Properties of Forensic Identification: Underlying Logic and Argumentative Implications.‖ Forensic Science International, Vol. 177. pp 120-132.

Abstract: The field of forensic science has profited from recent advances in the elicitation of various kinds probabilistic data. These provide the basis for implementing probabilistic inference procedures (e.g., in terms of likelihood ratios) that address the task of discriminating among competing target propositions. There is ongoing discussion, however, whether forensic identification, that is, a conclusion that associates a potential source (such as an individual or object) with a given item of scientific evidence (e.g., a biological stain or a tool mark), can, if ever, be based on purely probabilistic argument. With regard to this issue, the present paper proposes to analyze the process of forensic identification from a decision theoretic point of view. Existing probabilistic

91June 14, 2011 91

inference procedures are used therein as an integral part. The idea underlying the proposed analyses is that inference and decision are connected in the sense that the former is the point of departure for the latter. As such the approach forms a coordinated whole, that is a framework also known in the context as ‗full Bayesian (decision) approach‘. This study points out that, as a logical extension to purely probabilistic reasoning, a decision theoretic conceptualization of forensic identification allows the content and structure of arguments to be examined from a reasonably distinct perspective and common fallacious interpretations to be avoided.

Meyers, C. (2002). ―Some Basic Bullet Striae Considerations.‖ AFTE Journal, Vol. 34, No. 2, pp 158-160.

Abstract: The purpose of this paper is to review some of the basics of striae formation on fired bullets and the often present problems in their replication and subsequent attempts at quantification. Methods of quantifying bullet striae along with a slightly different view of percent match will be mentioned.

Stone, R. (2004). ―A Probabilistic Model of Fractures in Brittle Metals.‖ AFTE Journal, Vol. 36, No. 4. pp 297-301.

Abstract: The examination and comparison of fracture evidence has often fallen within the purview of firearm/toolmark examiners. Little has been published concerning the individuality or probabilistic examination of this type of evidence. Traditionally, examinations have been based on ―common sense‖ and the ―feeling‖ or ―assumption‖ that ―two things that match this well couldn‘t have originated from separate objects or separate breaks‖. This study will focus on how such two and three dimensional fractures form in brittle metals and will present derived mathematical probability models for them. It also presents a model that, with validation, would allow examiners to quantify such fractures in actual casework.

Claytor, L.K., and Davis, A.L. ―A Valdiation of Fracture Matching Through the Microscopic Examination of the Fractured Surfaces of Hacksaw Blades.‖ AFTE Journal, Vol. 42, No. 4. pp. 323-332.

Abstract: During a fracture, an object separates into two or more parts; fracture matching is the association of those surfaces to each other. Forensic fracture matching relies on the principle that no two objects fracture identically. The authors hope to further support this principle by studying the topography of fractured hacksaw blades. Two consecutively manufactured hacksaw blades were each fractured eleven times and inter-compared. Two hundred fifty-three topographical comparisons were conducted between 44 fractured edges; each fracture produced two surfaces discernable from any other. In addition, a series of proficiency style tests were made from consecutively manufactured blades and

92June 14, 2011 92

sent to participants throughout the United States and abroad. A total of 66 answer sheets were returned, providing 330 test results for evaluation. A comprehensive list of 49 references is included.

Uchiyama, T. (1992). ―The Probability of Corresponding Striae in Toolmarks.‖ AFTE Journal, Vol. 24. No. 3. pp 273-290.

Abstract: Two significant and practical corresponding relationships used to compare and determine the common origin of striated toolmarks are the number and quality of corresponding striae. It is essential to know how many striae can correspond by chance in order to determine the discriminating power of these relationships. The probability of chance correspondence (i.e., % match), and the maximum number of consecutively matched striae was estimated for variation in the density of the striae, the critical coincidence ratio (CCR), and the width of the striae of the marks compared. The critical coincidence ratio is a measure of how each specific striae compared is counted as corresponding or not corresponding. The results generated by this simulator study will be used to evaluate the discriminating power of these relationships and allow the implementation of effective and efficient automated systems applied to the comparison of real striated toolmarks. Automated comparison systems applied to real striated toolmarks (ie., fired bullets) will in turn allow comparisons on a scale sufficient to establish and validate objective criteria for the comparison of striated toolmarks to determine common origin.

Kaye, D. (2010). ―Probability and Individuality in Forensic Science Evidence.‖ Brooklyn Law Review, Vol. 75, No. 4, pp. 1163-1185

Abstract: Day in and day out, the testimony of criminalists reflects the paradigm of positive, uniquely specific identification of fingerprints, DNA profiles, bullets, handwriting, and other trace evidence. Commentators from other disciplines have called for a ―paradigm shift‖ that would replace talk of individualization with statements of probabilities or would exclude certain testimony pending better research on the ability of analysts to perform as claimed. With rare exceptions, however, the courts have failed to perceive the gap between optimistic theory and hard proof, and they have accepted weak forms of validation. Now that Congress has called on the National Academy of Sciences to ―disseminate best practices and guidelines concerning the . . . analysis of forensic evidence,‖ a new opportunity to reassess the long-entrenched claims of individualization is at hand. This essay seeks to contribute to such a reassessment by examining the arguments of two of the most powerful critics of this aspect of forensic science, Professors Michael Saks and Jay Koehler. The ―individualization fallacy‖ they describe implicates issues in philosophy, logic, mathematics, psychology, and statistics. This essay argues that contrary to one possible reading of their work, there is no rule of logic or ontology that prevents individualization and that testimony as to uniqueness is acceptable in some situations. It suggests a

93June 14, 2011 93

combination of evidentiary rules and practices to avoid the excesses of the current form of testimony.

Kirk, P. and Kingston, C. (1964). ―Evidence Evaluation and Problems in General Criminalistics.‖ Journal of Forensic Sciences, Vol. 9, No. 4. Pp 434-444.

Authors discuss the ideas of objectivity and subjectivity and the role statistics plays in forming the foundation of identification conclusions. Although the articles addresses more general criminalistics issues, firearms identification fits within the broad range of topics being discussed.

Biasotti, A. (1964). ―The Principles of Evidence Evaluation as Applied to Firearms and Tool Mark Identification.‖ Journal of Forensic Sciences, Vol. 9, No. 4. pp 428-433.

The author discusses the current state of firearm and toolmark identification by a review of the literature. He also addresses the issue of more objective standards in the field to form a conclusion of identification. The author discusses probability and statistical models as a possible approach to more objective standards.

Gunther J.D., and Gunther C.O, "The Identification of Firearms", Wiley & Sons, Inc. 1935. pp 33-36.

The authors discuss a basic probability model for determining the individuality of a firearm. They looked at the random chances of particular individual / accidental characteristics in determining how ―unique‖ a particular firearm signature was. They also discuss the concepts of a mark being so unique that it will be differentiated from all other firearms.

Hatcher, J., Jury, F., and Weller, J. "Firearms Investigation Identification and Evidence." Small-Arms Technical Publishing Company, 1935. (reprint 2006 – Ray Riling Arms Books Company) pp 376-381.

The authors discuss basic concepts and ideas of probability models being used in firearms identification to determine how likely a particular firearm was responsible for firing a particular bullet / cartridge case. They base their discussion in this text on the theory of probabilities and clearly state that their estimates of a numerical probability is strictly theoretical.

94June 14, 2011 94

AFTE Response to Seven Questions Related to Forensic Science ...

Documents

Transcript of AFTE Response to Seven Questions Related to Forensic Science ...