Firearms and Toolmark Analysis

Presented by: Dr. Heike Hofmann

5/8/2020 forensicstats.org
 Research Area Objectives

Currently established comparison standard:

AFTE Theory of
identification

1. examine class characteristics

2. use microscopic analysis to assess detailed features

Identified Problems:
1. establishing error rates of identification process
2. subclass characteristics (determined by proficiency tests
in Europe) are a key risk factor for false identifications.
forensicstats.org | 2
CSAFE 1.0
Accomplishments

forensicstats.org | 3
 CSAFE 1.0
Accomplishments

1. Two automated matching algorithms:

bullets (Hare et al., 2017a; Hare et al., 2017b)
cartridge cases (Tai and Eddy, 2017)

2. Open source algorithms: cartridge3D, x3ptools, bulletxtrctr

3. Open data: 3d topographic high-resolution scans of bullet

lands (~25,000) and cartridge cases (~2,000), mostly uploaded to
the NIST Ballistics Toolmark Research Database Evaluation
(NBTRD).
forensicstats.org | 4
CSAFE 2.0
Objectives

forensicstats.org | 5
CSAFE 2.0 Projects and Lead Investigators

F&T I- Statistical and Algorithmic Approaches to Matching Bullets and Cartridges

Lead PI: Heike Hofmann, ISU

F&T II- Subclass Characterization and Analysis of Firearms

Lead PI: Keith Morris, WVU

F&T IV- Evaluating Foundational Validity of Toolmark Analysis

Lead PI: Maria Cuellar, UPenn

forensicstats.org | 6
 F&T I Statistical and Algorithmic
Approaches to Matching Bullets
and Cartridge Cases
Proposed Activities:
• Expand and refine matching algorithm: nontraditional rifling, new features based on
image
• Quantify factors affecting matching performance: combination of firearm/ammunition,
quantitatively assess quality of scans.
• Work with firearms examiners and crime labs to extend use of matching algorithms to
labs

Potential Impact:
• Providing objective quantitative assessments that examiners can use during testimony
• Providing empirical support for the validity if firearms and toolmarks evaluation through
objective algorithmic assessments
• Working with examiners to develop community confidence and trust in algorithmic
results forensicstats.org | 7
 F&T II Subclass characterization
and analysis of firearms

Proposed Activities:
• Characterization of the manufacturing processes and breech faces
• Collaborate with firearm examiners to identify areas of subclass on all breech faces and test fires
• Collection of reference collections from five forensic laboratories
• Automated comparison using NIST congruent matching cells (CMC) algorithm with and without
subclass characteristics present
• Reference collection of Contender G2 breech faces
• Creation of subclass markup GUI

Potential Impact:
• Test sets can be created from test fires
(both with digital scans and double-castings)
• Examiner accuracy testing of identifying subclass characteristics
• Performance of NIST CMC algorithm with subclass present
forensicstats.org | 8
 F&T IV Evaluating Foundational 3 D at a
Create new database of toolmarks (start in 2D, then

Validity of Toolmark Analysis 3.1 D at NBIDE:

We use t he dat
a descr ipt ion
• aFirearm brand
of bullet casing
3D) Toolmarks:
images collect ed and analyzed • inScrewdriver
•
brand
Vorburger et al. (2007),
Screwdriver # (of
shown in Figure
same
• Firearm # (of same
as a low-resolut ion snapshot . T he casings were collect ed from t est fires of 4 Ruger P95D, 4 Smit h & Wesson
Sauer P226 pist ols. Each was fired 3 t imes usingbrand)
9VE and 4 Sig brand) each of 3 di↵ erent brands of ammunit ion
PMC, Remingt on and Winchest er. T he t ot al number of t est• fires
• Ammunition brand Angle,
was 108.surface
T he casingsmaterial
were re-imaged b

Proposed Activities: (soft to hard), striation vs.

NIST at t heir Gait hersburg, Maryland campus, and t he dat a were made available as part of t he Ballist ic
• Iteration per
Toolmark Research Dat abase, an open-access research dat abase of fired bullet and cart ridge reference dat a
firearm/ammo imprint, etc.
1. Database: Create a database of high-quality toolmark images, both
T hese images are referred t o as t he NBIDE (NIST Ballist ics Imaging Dat abase Evaluat ion) dat a set . T he ful
combination
online.2
dat aset can be Iteration
found
• Iterations per
angle/surface/etc.
2D and 3D, using a factorial design, based on NBIDE firearms #1…
combination
database. Firearm #1 Firearm
#2…
2. Algorithm: Develop an algorithm to determine a score-based
likelihood ratio.
3. Validation: Validate algorithm by testing its external validity.

Potential Impact:
Many
F igur e 1: Dat asetdegrees
from a 2007ofNISTfreedom Screwdriver
st udy. T he dat a provides t he ground t rutstriation
h about whet(L)
her casing

• Start with simplest case to make progress in difficult field of

were fired from t he same gun, or from t he same type of gun (and ammunit ion). T he blue box delineat es on
in screwdriver
of t he 108 images mark
of casings from t he dat aset .
and impression marks (R).
Petraco (2011).
toolmark analysis, which has many types of tools and degrees of Baiker et al. (2016).

freedom. 3.2 D at a pr epar at ion

• Develop standard statistical methods for the analysis and

By using t he similarity score provided by Tai (2017), we separat ed t he dat a int o four set s. Figure 2 shows t ha
t he t rue non-mat ch similarity score dist ribut ion has a much lower mean and variance t han t he dist ribut ion fo
comparison of toolmarks. mat ches. T here is no clear separat ion between t he two. T his means t hat some mat ches do not produce ver
high similarity scores (t his is t he same reason why t he minimum values in Table 1 are similar). T herefore

• Expand the capacity of federal, state, and local labs to deal with
toolmark analysis.
it is int erest ing t o separat e t he similarity scores for four set s: t he high-similarity mat ches, low-similarit
mat ches, high-similarity non-mat ches, and low-similarity non-mat ches. To separat e t he pairs int o t hese set
we st udy t he quant iles of t he two dist ribut ions, shownforensicstats.org
in Table 1. | 9
We separat ed t he pairs int o t he four set s by using t he quant iles as follows: High-similarity mat che
Where we would need help
from you!

• Forensic examiners community:

• Standard operating procedures for assessing firearm and toolmark evidence of
labs
• Test fires of (some of) the reference collections – format needs to be
determined, but ideally we would like 4 test fires for each firearm/ammunition
combo
• AFTE studies (past and on-going): we would be excited to get materials for 3d
imaging!

• Forensic analysts:
• Help us in running (proprietary) algorithms on publicly available data and
make results available

forensicstats.org | 10