See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/240749019

Software selection: Towards an understanding of forensic software tool

selection in industrial practice

Article  in  International Journal of Technology Policy and Management · January 2005

DOI: 10.1504/IJTPM.2005.008633

CITATIONS READS

3 481

1 author:

Rory V. O’Connor
Dublin City University
230 PUBLICATIONS   2,278 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Overcoming Public Speaking Anxiety of Novice Software Engineers using Virtual Reality Exposure Therapy View project

KM in OSS View project

All content following this page was uploaded by Rory V. O’Connor on 25 November 2015.

The user has requested enhancement of the downloaded file.

Int. J. Technology, Policy and Management, Vol. 5, No. 4, 2005 311

Software selection: towards an understanding of

forensic software tool selection in industrial practice

Rory V. O’Connor
School of Computing
Faculty of Engineering and Computing
Dublin City University, Dublin, Ireland
E-mail: Rory.OConnor@computing.dcu.ie

Abstract: As a result of the increasing importance of the forensic examination

of computer systems, numerous forensic software tools have appeared in the
market. The selection of such tools is an important technical and strategic
business decision that may have serious legal implications. This paper
examines the main issues surrounding the selection of forensic software tools
for use by computer forensic investigators pursuing digital evidence as part of
an organisational, civil or criminal investigation. The primary objective of this
research was to develop a set of characteristics which highlight the main
technical and non-technical selection issues and their relative significance in
explaining the selection of a particular forensic software tool in industrial
practice. This paper presents the results of interviews with commercial forensic
investigators in order to assess their attitudes, opinions and experience in
making tool selection decisions. In addition a model is developed which
describes the main discriminating selection criteria.

Keywords: software selection; technology adoption; technology management;

forensic software tools; digital evidence.

Reference to this paper should be made as follows: O’Connor, R.V. (2005)

‘Software selection: towards an understanding of forensic software tool
selection in industrial practice’, Int. J. Technology, Policy and Management,
Vol. 5, No. 4, pp.311–329.

Biographical notes: Dr. Rory O’Connor is Lecturer in Software Engineering at

Dublin City University. He received a PhD in Computer Science from City
University, London and MSc from Dublin City University. His research
interests are centred on the processes whereby software intensive systems are
designed, implemented and managed, in particular, methods, techniques and
tools for supporting the work of software project managers and software
developers in relation to software process improvement and the management of
software development projects. He is also interested in technology adoption
issues and the process whereby tools and techniques are evaluated and selected
in a commercial setting.

Copyright © 2005 Inderscience Enterprises Ltd.

312 R.V. O’Connor

1 Introduction

The discipline of forensic computing is vital to law enforcement agencies, governments

and companies worldwide in the 21st century. The application of forensic techniques to
computer systems can be viewed as the ‘new kid on the block’ by comparison to more
commonly used techniques such as DNA analysis, fingerprinting and fibre analysis. The
term ‘Computer Forensics’ was coined back in 1991 by the International Association of
Computer Investigative Specialists (IACIS) and since then has become a popular topic in
computer security circles and in the legal community. Like any other forensic science,
forensic computing deals with the identification, preservation, extraction and
documentation of computer evidence as part of an investigation into a crime such as
computer intrusion, unauthorised use of computers and child pornography. Computer
forensic practice is in a period of redefinition (Stephenson, 2004) as it moves from being
associated with the examination of ‘conventional’ systems to devices such as routers,
Personal Digital Assistants (PDAs) and digital cameras (Stephenson, 2004; Rod, 2002).
Computer evidence is quite unique when compared with other forms of traditional
‘documentary evidence’. Unlike paper documentation, computer evidence is fragile and a
copy of a document stored in a computer file is identical to the original. Another unique
aspect of computer evidence is the potential for unauthorised copies to be made of
important computer files without leaving behind a trace that the copy was made. Like
other forms of evidence – hair, blood, eyewitness account, or paper documents – its form,
prevalence, and existence help clarify competing stories when re-enacting an event.
Furthermore, the reliability of both the physical and digital evidence can be scrutinised
by examining the forensic science involved in its identification, collection, preservation
and analysis.
Forensic science requires its practitioners to have the appropriate training and
education needed to perform the examination, and prove the rigor of the techniques and
communicate the results clearly to a court, which often contains a lay jury. Forensic
equipment, tools and techniques must have scientific validation and produce a
demonstrably accurate result.
To assist the computer forensic investigator, there are a wide variety of forensic
software tools in the marketplace, providing features such as forensic imaging, forensic
data recovery and forensic data analysis. Selecting a forensic software tool is difficult and
confusing for many organisations. There is a critical need for both law enforcement
agencies and corporations to make informed choices about acquiring and using forensic
software tools. Since forensic science is the application of a scientific discipline to the
law, the essence of all forensic disciplines concerns the principles applied to the
detection, collection, preservation and analysis of evidence to ensure its admissibility in
legal proceedings (Kenneally, 2002).
Forensic investigators may use tools, procedures and methods not readily available to
the public and therefore not readily understood and accepted. For an investigator’s
findings to be accepted they must be recognised by other experts within the field and
conform to national and international standards of practice. Computer forensic
investigators risk loss of credibility if doubt can be introduced into the appropriateness of
tools and/or actions deployed in the presented evidence (Armstrong, 2003), and as a
direct result there may be a failure to prosecute a case to the full extent of the law.
 Software selection: towards an understanding of forensic 313

Considering the significant issues regarding both the reliability and admissibility of
digital forensic evidence, a pivotal question for both the courts and the computer forensic
investigator is “what makes a ‘good’ forensic software tool?” and “how does one chooses
a ‘good’ software tool?”
To address these questions, this research examines the current state of practice with
respect to forensic software tool selection and examines the relevant literature to
determine a set of characteristics that may be used by computer forensic investigators to
guide the selection of a forensic software tool.
The purpose of this research is to provide a more comprehensive understanding of the
factors that are significant to the selection of a forensic software tool for use by a
computer forensic investigator in industrial practice. A primary objective of this research
is to develop a generalised set of characteristics that explains the effects of important
selection factors and the relative significance of these factors in explaining the selection
of a particular forensic software tool. Thus, the key research question is: “What are the
forensic software tool characteristics that contribute to the selection of a tool for a given
forensic investigation and what is the relative importance of these characteristics?”

2 Overview of forensic software tools

There are three primary reasons why specialised forensic software tools must be
employed in order to conduct a proper computer investigation (Patzakis, 2004):
• Proper acquisition and preservation of evidence
Electronic evidence is fragile by nature and can be easily altered or erased without
proper handling. For example, merely booting a subject computer into a Microsoft
Windows environment will alter critical date stamps, erase temporary data and cause
hundreds of writes to the hard disk.
• Authentication of collected data for court presentation
Computer forensics is largely based on the premise that the data recovered from a
system will ultimately be presented in a court of law. Therefore an important feature
is a verification process that establishes that the investigator did not corrupt or
tamper with the subject evidence at any time during the investigation.
• Recovery of all available data, including deleted files
In addition to the active data normally seen by the computer user, computer forensic
software tools must allow the examiner to recover all deleted files that have not been
completely overwritten, as well as other forms of unallocated or temporary data.
There are a number of forensic computer software tools of varying sophistication. These
tools generally differ in functionality, complexity and cost. In terms of functionality,
some tools are designed to serve a single purpose (Rod, 2002; Arthur and Venter, 2004)
while others offer a suite of functions. Therefore, the functionalities offered by a tool are
exactly what lead to its complexities. These complexities can either be related to design
and algorithmic complexity or ease-of-use; in some instances a tool can offer great
functionality but fall short because of a complex interface. Cost is another final
distinguishing factor, with some of the market-leading commercial products costing
thousands of Euros while other tools are free.
314 R.V. O’Connor

Generally, forensic software tools are designed to specialise in one or two specific
areas of forensic analysis of a computer system. The main areas are forensic imaging,
forensic data recovery and forensic data analysis. Some examples of market-leading and
free forensic software tools are:
• SafeBack – a commercial tool commonly used by law enforcement agencies
throughout the world. It is used primarily for imaging the hard disks of Intel-based
computer systems and restoring these images to other hard disks.
• EnCase – a commercial software package that enables an investigator to image and
examine data from hard disks, removable media and PDAs. Many law enforcement
agencies throughout the world use EnCase and this can be an important factor for
forensic investigators to consider where there is a possibility that an investigation
may be handed over to the police or used in a court of law.
• Paraben E-mail Examiner – a comprehensive e-mail examination tool which is
capable of examining and recovering active and deleted email messages from most
leading proprietary email programs as well as generic mailboxes.
• GetGif – a budget-priced tool which automatically extracts exact copies of graphic
file images and is used to find evidence in corporate, civil and criminal
investigations which involve computer graphic files, e.g., investigations which
potentially involve child pornography.
• The Coroner’s Toolkit – a collection of (essentially) free tools designed to be used in
the forensic analysis of a UNIX machine. This is specifically designed to be of use in
the investigation of a computer break-in and has tools to help reconstruct the
activities of an intruder.
There is much by way of published material (see Kenneally, 2001; Patzakis and
Limongelli, 2004 for example) dealing with examples of the usage of forensic software
tools in gathering digital evidence and subsequent court prosecutions in both civil and
criminal cases. In the current business climate such tools are also used in common
business practices. For example, (Patzakis, 2004) details one Fortune 500 company
where an employee’s hard disk is imaged upon resignation, termination or internal
transfer as a matter of standard procedure, to allow subsequent examination should it
need to take place.

3 The forensic software tool selection problem

In the last few years the forensic software tool market has undergone remarkable
changes. The number of tools has increased significantly, prices have declined
dramatically and such tools are now being used by a diverse group of users. Many of
these tools were developed to fit different user needs and designed to execute on a variety
of hardware platforms. Owing to the complexity of the product and the profusion of
alternatives a systematic process of selection can be formidable and expensive.
To further complicate the situation, selecting the best forensic software product
may not be the responsibility of a single individual, for group decision making is
common place in most organisations. A group approach to software selection offers many
benefits, including improved overall decision quality and decision-making effectiveness.
 Software selection: towards an understanding of forensic 315

Still the process of appraising forensic software tools or any other IT investment may be
a political process as decisions touch many people and groups. Clearly, software selection
is not a well-defined or structured decision problem. The presence of multiple criteria
(both managerial and technical) and the involvement of multiple decision-makers will
expand the decision from one to many dimensions, thus increasing the complexity of the
solution process. It seems obvious that we cannot solve the software selection problem by
simply grinding through a mathematical model or computer algorithm.

3.1 Standards
Forensic investigators may use tools which are not readily understood by the public and
the courts. It is therefore vital for an investigator to ensure that doubt cannot be
introduced into the appropriateness of tools deployed in the collection and presentation of
evidence (Armstrong, 2003).
There is a critical need in the law enforcement community to ensure the reliability of
computer forensic tools. The National Institute of Standards and Technology (NIST) in
partnership with law enforcement and other agencies has developed a programme for
testing computer forensic software tools. A goal of the Computer Forensic Tool Testing
(CFTT) project at NIST is to establish a methodology for testing computer forensic
software tools (Wick et al., 2004). CFTT provides a mechanism for users to determine if
a specific computer forensic tool meets their needs. It does this by:
• defining a means for classifying requirements for different types of tools (currently
disk imaging and hard disk write blocking tools)
• specifying these requirements
• defining tests to determine if these tools meet the requirements
In addition, a US Air Force programme1 was set up in 2001 to address the issue of
defining a framework for the area of digital forensics as a whole. However, to date
such programmes have investigated technical issues regarding testing and reliability
of a limited number of tools and have not addressed the meta-level question of
suitability of tools, the selection of tools and the justification/defence for the selection of
a particular tool.

3.2 Legal issues

In many countries there is a legal debate underway regarding the acceptance of digital
evidence as documentary evidence. In most countries, in the eyes of the law, there is a
big difference between the acceptance of digital evidence as documentary evidence
– where it is expected to ‘stand on its own’ and require no context or interpretation by
expert witnesses – and electronic evidence as supporting evidence (where independent
explanation of relevance is necessary). There is a debate whether the law should be
significantly altered in order to explicitly allow electronic evidence to be admitted under
documentary evidence rules.
316 R.V. O’Connor

Much is written on methodologies for searching and seizing a computer environment

(Patzakis and Limongelli, 2004). There is, however, a vast gap between civil and criminal
aspects of any investigation. In the case of a criminal investigation where law
enforcement is involved, it may be necessary and appropriate to freeze the entire
computer environment. In the case of a civil case within an organisation, it may not be
necessary to freeze the computer environment, merely ring-fence the activities of the
suspect to limit damage.
In most courts of law, evidence (physical or digital) must be reliable. However this
varies greatly from country to country. For example, in a US court, the reliability of
scientific evidence, such as the output from a forensic software tool, is determined by a
judge (not a jury) in a pre-trial ‘Daubert Hearing’. It is the judge’s responsibility to
determine whether the underlying methodology and technique used to identify the
evidence was sound and whether, as a result, the evidence is reliable. The Daubert
process identifies four general categories that are used as guidelines when assessing
a procedure:
1 Testing – can the procedure be and has been tested?
2 Error rate – is there a known error rate for the procedure?
3 Publication – has the procedure been published and subject to peer review?
4 Acceptance – is the procedure generally accepted in the relevant
scientific community?
Another area of contention is whether a person can be considered an expert solely based
on ability to use a software tool, without the ability to define how the tool works or
without reviewing the source code. For example, (Meyers and Rogers, 2004) cites two
relevant US-based examples: in Williford versus Texas, the court found that an expert
does not need to know the code of the software package nor the background processes;
and in State of Washington versus Leavell, the court found that an inanimate object (the
forensic software tool) cannot be an expert.
There are indications that evidence generated through the use of standard, generally
available software tools is easier to admit than evidence generated with custom software
tools (Kenneally, 2002). The rationale being that the capabilities of commercially
marketed software packages are well known and cannot normally be manipulated to
produce aberrant results, whereas custom software, on the other hand, must be carefully
analysed by an expert programmer to ensure that the evidence being generated by the
computer is in reality what it appears to be. Whilst this is a highly contentious argument,
it is an issue that the forensic computer industry must address.

4 Research method

When a user starts to think about whether to adopt a new tool, s/he will think about cost,
benefits and risks. This thinking may not be deep and rigorous, but it is clear from prior
research that a certain amount of such thinking takes place (Lethbridge, 2004). There is a
long history of general research into adoption or non-adoption of technical innovations.
Perhaps the most influential researcher is Rogers (2003) who points out that there
are numerous reasons why individuals do not adopt innovations, or adopt them slowly.
 Software selection: towards an understanding of forensic 317

One of his key points is that an individual’s adoption decision is based on his or her
perception of various factors rather than absolute truths about those factors. In Rogers’
model, adoption starts with the ‘knowledge stage’ in which people become aware of the
existence of the innovation. Then they move to the ‘persuasion stage’ in which they form
‘favourable or unfavourable attitude’ towards the innovation, perhaps influenced by
marketing or interaction with others. Only after forming a favourable attitude do they
move to the ‘decision stage’, where they consciously or subconsciously consider the
various factors that may lead to trial use or more intense use. In this paper we will focus
on the issues considered during the decision stage.
Extensive tool evaluation is generally a very time- and resource-intensive process.
Hence we segment our study of the tool selection process into three stages. The first stage
of this investigation is concerned with the state of best practices in computer forensic
investigations. As previously discussed, there is little by way of direct guidance in the
relevant literature regarding the selection of forensic software tools. However, the
literature and marketplace is worth examining in order to fully explore the general issues
surrounding forensic software tools.
The second stage of this study was to assess the state of practice by computer forensic
investigators in relation to tool selection. In order to explain the use of a particular tool
one must first understand the organisational context and the nature of the investigation
together with the intentions and actions of the computer forensic investigators involved.
Accordingly, this research investigated the underlying variables and the beliefs of
practising computer forensic investigators regarding the influences of these variables
upon the selection decision.
The third stage of this research involved bringing together the principal
characteristics identified in the previous two stages to form a generalised set of
characteristics that describes the primary forensic software tool selection criteria which
may be used to assist the various stakeholders in the selection process.

4.1 Research context

Due to the large number of tools available and wide range of application areas to which
they may be applied, we decided to limit the initial investigation to a subset of tool
categories to include two of the most frequently used tool types – disk imaging and string
searching. Whilst the purpose of this study is to explore a generalised set of forensic
software tool selection characteristics, initially the study was limited to the practice of
computer forensic investigations in Ireland in the first instance, as this is the country
where the researchers are based.

4.2 Industrial practice review

The second stage of this study was to assess the state of practice by computer forensic
investigators in relation to tool selection. Therefore, as part of this research a series of
interviews were conducted with a set of key individuals (expert computer forensic
investigators) in a number of commercial organisations involved in forensic computing
investigations. A total of ten individuals representing the five leading computer forensic
organisations based in Ireland participated. The participating organisations included
specialist forensic software firms, the forensic computing division of multinational IT
318 R.V. O’Connor

consulting firms and the forensic/security departments of large multinational technology

firms. In order to respect the privacy of the individuals and their employers, they will not
be named.
The interviews were conducted in-house and were two to four hours in duration. They
were conducted in two distinct phases. The approach taken during the first phase
involved utilising the depth (unstructured) interview technique (Jones, 1985), which
involves asking open-ended questions, listening to and recording the answers and then
following up with additional relevant questions. Far from being an ad-hoc talk-and-listen
session, this data gathering approach is recognised as providing a wealth of valuable
information, as it provides for greater flexibility where questions can be spontaneous and
responsive to the last things the interviewee has said (Coombes, 2001). The questions
asked were intended to elicit decision-making data in the following main areas:
• The forensic software tools currently known and in regular use by the participants
• The forensic software tools most frequently used in the firm
• The motivation behind the selection of these forensic software tools
• General software selection issues and policy from the firm’s perspective
• General software selection issues and policy from the individual’s perspective
• General software selection issues and policy from the client’s (external or internal
client) perspective
• The technical selection criteria (discriminating variables) from the issues previously
identified with regard to forensic software tools
• The non-technical (business) selection criteria (discriminating variables) from the
issues previously identified with regard to forensic software tools
• The impact of current (and future) legal requirements in relation to civil prosecution
based on digital evidence and usage of forensic software tools
• The practitioner’s views on proprietary versus open-source forensic software tools.
Table 1 lists the main forensic software tools used in the participating organisations. It is
worth noting that the majority of forensic investigators used at least one tool that is in
common use and has been the subject of court evidence by the main policing agency in
Ireland, thus allowing for ease of transfer of digital evidence, whereby the police can
subsequently validate all the actions of the forensic investigator.

Table 1 Main forensic software tools used

Category 1 Category 2
Disk imaging String searching
EnCase EnCase
FTK Imager DTSearch
Norton Ghost AccessData Forensic Toolkit (FTK)
DD/DCFLDD Paraben Email Examiner
 Software selection: towards an understanding of forensic 319

5 Selection criteria

During these interviews, four distinct areas of concern emerged as common themes:
• General managerial (non-technical) issues – applicable to both forensic and
non-forensic software tools
• General technical considerations applicable to all categories of forensic
software tools
• Specific technical considerations in relation to the two categories (disk imaging and
string searching) under consideration
• Legal issues related to subsequent civil prosecution based on digital evidence
obtained by a forensic software tool.
In-depth analysis of the outcomes of the interviews resulted in further decomposition of
the selection themes above to show the decision criteria and variables as identified by the
participants, as illustrated in Figure 1. The following sections will discuss these criteria.

Figure 1 Forensic software tool selection model

Forensic software tool

selection decision

Technical Managerial Legal

considerations considerations considerations

General technical Specific technical • Availability • Non-repudiation

considerations considerations • Purchase cost • Verifiability
• Cost of ownership • Repeatability
• Tool background
• Product maturity
• Vendor
• Accuracy Tool category
• Vendor support
• Audit trails • Training
• Forensic integrity
• Presentation/reporting
• Underlying technology String
Disk
• Usability/ease of use searching
imaging
• Reliability
• Speed

5.1 Legal considerations

There were three main legal issues that affected the decision as to which forensic
software tool to select and these are discussed below. It is worth noting that whilst
few of the cases that the participants were involved in ended up in a court of law, the
ability to pursue a case via the legal system was in most cases very important. The
motivation behind this lies in the fact that the majority of civil cases in an Irish context
320 R.V. O’Connor

involving forensically recovered digital evidence do not end up in the courts or result in
an out-of-court settlement (Ernst and Young, 2005), but the injured party values the
threat of a court prosecution.
• Non-repudiation
The issue of non-repudiation of digital evidence (the ability of a party to an offence
to deny having performed a particular action (McCullagh and William, 2002) was
considered to be of extremely high importance for criminal cases and very important
for civil cases.
• Verifiability
Forensic tools produce a demonstrably accurate result, as objection to authenticity
may involve questioning the reliability of the computer program that generated or
processed the computer evidence in question. In such cases the proponent of the
evidence must testify to the validity of the program utilised in the process (Patzakis
and Limongelli, 2004).
• Repeatability
Repeatability of an investigation or deduction of an action/data from an original
source was considered important, as a subsequent court action would require a
complete explanation or demonstration of how digital evidence was obtained.

5.2 Managerial considerations

There were eight main managerial (or non-technical) issues which affected the decision
as to which forensic software tool was selected and these are discussed below.
• Availability
The availability of the software was an issue in cases where specialist software tools
that were not possessed by the practitioner were required at short notice.
• Purchase cost
The initial purchase cost of the tool was a minor issue for most practitioners.
However, the subsequent cost of ownership was not.
• Cost of ownership
The costs associated with ongoing support and maintenance costs associated with
updates to the tool and subsequent retraining, etc. were seen as a potentially
important issue.
• Tool background
How the tool has evolved and what impact this has on both the application domain
and future tool evolution. This was considered by most to be moderately important.
 Software selection: towards an understanding of forensic 321

• Product maturity
Issues such as how stable the tool is in terms of its evolution and if it would
continue to evolve. The issue of continuing evolution (in terms of product
improvement/enhancement) was ranked as being moderately important.
• Vendor choice
The main issues in relation to vendors were the availability and choice of vendors
and the firm’s allegiance or strategic business alliance with any specific vendor. In
the main, this was not identified as being a particularly important issue.
• Vendor support
The quality of ongoing vendor support was of importance. Important indicators of
reliable vendors include: availability of immediate (24-hour) telephone support,
quality of consulting services, and quality of training services. In addition, several
participants took into account the endorsement and evaluation of the vendor and the
vendor’s product from other current users and industry sources.
• Training
The availability of professional (usually vendor-backed) training was seen as
an important issue. Specifically issues such as average training time, associated
costs, availability of suitable training materials and ease of access to training
were identified.

5.3 General technical considerations

There were eight main technical issues that were applicable to all categories of forensic
software tools that affected the decision as to which forensic software tool was selected.
These are discussed below.
• Accuracy
The accuracy of the results produced by a tool for a given forensic examination of a
system were regarded as being of extremely high importance by practitioners.
• Audit trails
The ability of the tool to produce a complete audit trail of all actions performed by
the forensic investigator was considered a reasonably important factor, as it allowed
third parties to subsequently validate all the actions of the forensic investigator.
• Maintaining the integrity of forensic process
Employing a proper forensic process is the foundation of all computer investigations
(Patzakis, 2003). The mishandling or compromise of digital evidence, either during
the collection or analysis process, will result in the evidentiary integrity of the data
being lost.
322 R.V. O’Connor

• Presentation/Reporting
The depth, level and nature of the reports produced from tools are considered very
important from two perspectives – that of the forensic investigator who requires
in-depth technical data for scrutiny and that of the non-technical audience (managers,
lawyers, judge, jury, etc.).
• Usability
Whilst usability has a broad definition covering ease of learning, ease of use,
flexibility of use, effectiveness of use and user satisfaction with a system, the
principal usability issues for computer forensic investigators were speed and
ease of learning.
• Reliability
The reliability of both the forensic software tool and its output has potentially serious
ramifications, particularly when most legal systems require very high levels of
assurances regarding evidentiary offers of proof.
• Underlying technology
Participants identified the need to understand the scientific principles underlying the
tool. It is therefore necessary to understand the underlying technologies behind the
various tools used and their ability to present scientifically valid information.
• Speed
This refers to the speed at which a tool operates and speed at which results may be
obtained from the forensic examination of a system. In particular, due to the growing
size of hard disk capacities, imaging and searching speeds were considered
particularly important.

5.4 Specific technical considerations

As previously discussed, due to the large number of tools available and the wide range of
application areas to which they may be applied, we decided to limit the initial
investigation to a subset of tool categories to include two of the most frequently used tool
types – that of disk imaging and string searching. There was a series of specific technical
issues identified in relation to the two categories under consideration and these are
discussed below.

5.4.1 Disk imaging tools

Disk imaging (mirror image backups) involves making a complete bit-by-bit duplicate of
all areas of a computer hard disk drive (or another type of storage media) where the
duplicate exactly replicates all sectors. Thus, all files and ambient data storage areas are
copied. Such duplicate disks are referred to as ‘evidence grade’ duplicates and they differ
substantially from traditional computer file backups and network server backups, which
only copy active files and not deleted files.
 Software selection: towards an understanding of forensic 323

The main technical issues applicable to this category of forensic software tool which
affect the selection decision are:
• A complete forensically accurate bit-stream duplicate (image) of a disk (or partition,
other media device) must be made
• The tool should be able to verify the integrity of the disk image
• The tool must not alter the original disk in any way
• The investigator must be able to prove the duplicate data hasn’t been modified from
the original acquisition
• The tool should be reliable in that it must work correctly each time it is used and an
identical image should be produced
• The tool should log all I/O errors.

5.4.2 String searching tools

Substantial amounts of forensic investigation work involves carefully targeted
examinations of computer hard drives using a process called ‘string searches’ which
involves the systematic probing of all directories and files on a given file system looking
for a specific sequence of words, letters or bytes. Typical uses of string searching forensic
software tools include: sorting through a large number of suspect computers prior to
copying in order to eliminate those containing no relevant information; looking for
references to specific types of crime on suspect computers, such as the presence of
common drugs terms or phrases used by paedophiles; and conducting initial high-speed
searching in order to test potential search criteria.
The main technical issues applicable to this category of forensic software tool which
affect the selection decision are:
• Verifiability – it is essential that the evidence collected is verifiable. Many forensic
software tools produce a digital signature in the form of a message digest that
ensures that the authenticity of the search results can be verified.
• Repeatability – repeatability of a search from the original source is an important
factor, as a subsequent court action would require a complete explanation or
demonstration of how digital evidence was arrived at.
• Integrity – the search process should cause no alterations to the original disk/media
being investigated.
• Speed of searching was identified as a high priority selection factor. This is
especially important given that the volume of data stored on the average computer
system is growing at an enormous rate. For example, DIBS Mycroft (a high-speed
search engine used in forensic analysis) searches a suspect computer’s hard disk for
evidence at the rate of about 5 megabytes per second. With an average PC hard disk
of 40GB, comprehensive searching is not a trivial matter.
324 R.V. O’Connor

• The tool should produce no false positives, i.e., a test should not report, incorrectly,
that it has found the target string where none exists in reality.
• The tool should produce no false negatives, i.e., a test should not report, incorrectly,
that the target string(s) were not detected when, in fact, they are present.
Practitioners reported that false negatives had a much more serious impact than false
positives. The rationale behind this being that a small amount of time had to be
invested in the clarification of a false positive, by comparison to the potential
damage of not identifying suspect data (false negative).
• The tool should be able to accomplish/implement all the features as ‘promised’ by
the product data sheet. For example, Boolean style search, stemming, indexing, etc.

5.5 Other issues

During the course of the interviews a number of other selection factors arose that were
not universally accepted as common selection issues but which merit mention. In
particular some of the practitioners admitted to using their own personal experiences
when selecting a particular tool for a given situation, or when a given outcome was
desired. In addition, tools which are perceived to be market leaders in their specialist
domains were often chosen for specific individual specialist tasks with little reflection on
the decision process, often for pragmatic reasons.
For all the participants there were little or no formal or semi-formal guidelines
regarding the evaluation or selection of forensic software tools, although some of the
participating organisations did have a formalised purchase policy with respect to
computer hardware and other forms of computer software tools. In these cases, this was
justified by the explanation that forensics is regarded as a new field and there was either
no internal agreed standard or no impetus to create such standards.
In addition, the participants also commented on the nature of the Irish forensic
computing profession and its impact on tool selection and usage. Due to the limited
number of professional forensic computing practitioners, the majority of the key
individuals in the Irish computer industry personally knew each other and used an
informal network to communicate and share experiences, including those relating to
forensic software tools.

5.6 Summary
Table 2 lists the decision criteria and variables as discussed above (and illustrated in
Figure 1) along with and their associated level of importance (Not important, moderately
important, important, very important) as ranked by the participants in this study.
 Software selection: towards an understanding of forensic 325

Table 2 Forensic software tool selection criteria and ranking

Theme Criteria Importance level

Technical Accuracy Very important
Audit trails Moderately important
Forensic integrity Very important
Presentation Very important
Underlying technology Moderately important
Usability Not important
Reliability Very important
Speed Important
Managerial Availability Not important
Purchase cost Not important
Cost of ownership Moderately important
Tool background Moderately important
Product maturity Moderately important
Vendor choice Moderately important
Vendor support Important
Training Important
Legal Non-repudiation Very important
Verifiability Important
Repeatability Important

6 Conclusions
6.1 Future research
There are a number of limitations in the current study. Future studies should include a
larger number of forensic practitioners and firms, a greater diversity of participants in
terms of country of origin and commercial versus law enforcement organisations
operating in more than one country. This increase in sample size would allow for the
development of a more comprehensive and significant set of selection factors. In
addition, future studies should be broadened to include a larger set of forensic software
tool categories that would allow for multiple selection characteristic sets.
Despite its limitations, this study makes an important contribution to our
understanding of forensic software tool selection by computer forensic investigators and
the selection issues in the current business environment.
In addition to the above, there are a number of key issues to be addressed in the future
with regard to both the development, evolution and selection of forensic software tools.
Three leading issues are software reliability, the open-source debate and the need for a
formalised decision model for the selection of forensic software tools. These issues will
be discussed in the following sections.
326 R.V. O’Connor

6.1.1 Software reliability

There is a growing tension between the need to present probative and visual evidence of
digital disputes and the legal standards for the admissibility of scientific and technical
evidence (Kenneally, 2001). In other words, the ubiquity of computer technology and the
pervasiveness of the data obtained bear a direct relation to software. Since software
provides the functional link between man and machine, unreliable software is sure to
have an effect on any activity in a high-tech society. The resolution of many disputes
rests with the reliability of the digital evidence derived from software tools, therefore,
software reliability is of profound significance to both the forensic software industry and
the legal system.
Some have observed that digital evidence may carry an aura of infallibility in
the public’s eyes, a fact that may facilitate settlements and discourage technical
challenges during litigation (Kenneally, 2001). Computer technology is afforded a
presumption of reliability because there is a common belief that machines are immune to
human frailties, desires and whims that can lead to erroneous information or
misinterpretation (White, 1996).
Until there is a universal ‘underwriters laboratory’ to denote third party certification
of reliability (Kenneally, 2001), or a common national and international framework from
which reliability can be determined, there will be continued debate and litigation
regarding the admissibility of digital evidence obtained from forensic software tools.

6.1.2 Open-source debate

Proprietary software tools (i.e., software whose underlying source code is not freely
available to be viewed or changed) stand in marked contrast to open-source software,
where access to the systems’ source code is the central defining point, as well as the issue
of reliability which is a significant issue behind the open-source movement.
There are indications that evidence resulting from proprietary software enjoys a
presumption of authenticity, while its customisable open-source counterpart faces a
higher hurdle (Kenneally, 2001; Carrier, 2002). Evidence generated through the use of
standard, generally available software is easier to admit than evidence generated with
custom software. The reason lies in the fact that the capabilities of commercially
marketed software packages are well known and cannot normally be manipulated to
produce aberrant results. Custom software, on the other hand, must be carefully analysed
by an expert programmer to ensure that the evidence being generated by the computer is
in reality what it appears to be. Non-standard or custom software can be made to do a
host of things that would be undetectable to anyone except the most highly trained
programmer who can break down the program using source codes and verify that the
program operates as required.
Some computer forensic investigators utilise custom software tools developed by the
investigating agency or a private company that are not commercially available to the
general public. The courts have addressed some of the issues concerning the type of
software involved where computer-generated evidence is at issue (Patzakis and
Limongelli, 2004). Such cases provide a presumption of authenticity for evidence
resulting from commercially available tools over custom tools. There have been cases
where the courts have actually required that any computer-generated evidence be a
product of a ‘standard’ tool in order to admit such evidence. For example, USA versus
 Software selection: towards an understanding of forensic 327

Greathouse presented a twist in computer forensic case law: rather than the typical
situation in which the defence challenges the prosecution’s use of a particular tool, the
defence argued instead that the prosecution should have used a specific market-leading
tool (EnCase) (Kenneally, 2001).
There remains an open issue for forensic software investigators. Should evidence
derived from proprietary software be scrutinised just because the source code is made
secret? Closed mechanisms are inherently incompatible with the reliability requirements
of digital evidence such as embodied in the Daubert test, and as such create a dilemma
for judges when the instrument’s reliability is legitimately called into question.

6.1.3 Formalised decision model

In our increasingly electronic society, digital evidence promises to continue to saturate
crime scenes and civil disputes, thus rendering computer forensics an increasingly vital
discipline in the resolution of disputes. A potential danger is that computer forensics will
be driven by industry and market forces that lose sight of the need for scientific
underpinning regarding computer forensic tools (Kenneally, 2002). An important step is
to increase confidence in the use of forensic software tools at all levels from the selection
of forensic software tools to their formal testing.
Accordingly, a more formalised approach to arriving at the selection decision using
the identified characteristics would be more desirable (Armstrong, 2003; O’Connor,
2004). Therefore an ultimate aim of this research could be a formalised decision/selection
framework for use by computer forensic investigators in law enforcement agencies,
governments and companies in the selection of the most appropriate forensic software
tool for a given situation. In the future we intend to draw upon the decision of sciences
literature to identify a more appropriate formalised decision-making process. It is
considered that the problems presented in this paper represent a multiple attribute
decision-making problem (O’Connor, 2004) and techniques such as Analytic Hierarchy
Process (AHP) (Hwang and Yoon, 1981) may form the basis for the future evolution of
this research. Previous studies in related IT domains such as software project selection
(Muralidhar et al., 1990), multimedia authoring tool selection (Lai et al., 1999) and
expert system shell selection (Kim and Yoon, 1992) have successfully applied AHP.

6.2 Summary
This paper has reported on the issues facing computer forensic investigators in law
enforcement agencies, governments and companies in the selection of the most
appropriate forensic software tool for a given situation. It has discussed the issues of
reliability and admissibility of digital forensic evidence gathered using forensic software
tools and has highlighted the need for both the courts and computer forensic investigators
to justify answers to the questions of “what makes a ‘good’ forensic software tool?” and
“how to make a ‘good’ choice of forensic software tool?”
This research set out to provide a more comprehensive understanding of the
factors that computer forensic investigators and their firms consider significant to
the selection of a forensic software tool for use in a computer investigation in
industrial practice and thereby address the key research question: “What are the
forensic software tool characteristics that contribute to the selection of a tool for a
given forensic investigation?”
328 R.V. O’Connor

The significance of this research is in broadening the understanding of forensic

software tool selection in industrial practice by focusing on the issues of concern to
computer forensic investigators. A model is presented which describes the main
discriminating selection criteria and the associated issues are explored, thus providing a
better understanding of the factors that are pertinent to the successful selection of forensic
software tools.

Acknowledgements

The author acknowledges the assistance of the students of the MSc in Security and
Forensic Computing class of 2004–2005 and the cooperation of the forensic computing
practitioners and organisations who participated in this study. The author also
acknowledges the assistance of Professor Kurt Engemann and the anonymous reviewers
for their helpful comments in the preparation of this article.

References
Armstrong, C. (2003) ‘Developing a framework for evaluating computer forensic tools’,
Proceedings of Evaluation in Crime and Justice Conference, Canberra, Australia, March.
Arthur, K. and Venter, H. (2004) ‘An investigation into computer forensic tools’, Proceedings 4th
Annual Information Security South Africa Conference, July.
Carrier, B. (2002) ‘Open source digital forensics tools: the legal argument, Research Report,
@Stake Inc., October.
Coombes, H. (2001) Research Using IT, Palgrave.
Ernst and Young (2005) Computer Forensic Case Studies, http://www.ey.com/global/content.nsf/
Ireland/tsrs_computer_forensics_case_studies_overview, Ireland, (retrieved January 2005).
Hwang, C.L. and Yoon, K. (1981) Multiple Attribute Decision Making: Methods and Applications,
Springer Verlag.
Jones, S. (1985) ‘Depth interviewing’, in R. Walker (Ed.) Applied Qualitative Research, Gower.
Kenneally, E. (2001) ‘Gatekeeping out of the box – open source as a mechanism to assess
reliability for digital evidence’, Virgina Journal of Law and Technology, Vol. 6, No. 3.
Kenneally, E. (2002) ‘Computer forensics – beyond the buzzword’, Login, August, Vol. 27, No. 4.
Kim, S. and Yoon, Y. (1992) ‘Selection of a good expert system shell for instructional purposes in
business’, Information and Management, Vol. 23, No. 5.
Lai, V., Trueblood, R. and Wong, B. (1999) ‘Software selection: a case study of the application of
the analytical hierarchical process to the selection of a multimedia authoring system’,
Information and Management, Vol. 36, No. 4.
Lethbridge, T. (2004) ‘Value assessment by potential tool adopters: towards a model that considers
costs, benefits and risks of adoption’, Proceedings of 4th International Workshop on
Adoption-Centric Software Engineering, May.
McCullagh, A. and William, C. (2002) ‘Non-repudiation in the digital environment, First Monday,
Vol. 5, No. 8.
Meyers, M. and Rogers, M. (2004) ‘Computer forensics: the need for standardisation and
certification’, International Journal of Digital Evidence, Vol. 3, No. 2.
Muralidhar, K., Santhanam, R. and Wilson, R. (1990) ‘Using the analytic hierarchy process
for information system project selection’, Information and Management, Vol. 18, No. 2,
pp.87–95.
 Software selection: towards an understanding of forensic 329

O’Connor, R. (2004) ‘A decision framework for forensic software tool selection’, in K. Engemann
and G. Lasker (Eds.) Advances in Decision Technology and Intelligent Information Systems,
IIAS, Vol. V.
Patzakis, J. (2003) ‘Maintaining the digital chain of custody’, Infosecurity Europe
Conference, April.
Patzakis, J. (2004) ‘Computer forensics as an integral component of the information security
enterprise’, Guidance Software White Paper, www.guidancesoftware.com/corporate/
whitepapers (retrieved December 2004).
Patzakis, J. and Limongelli, V. (2004) EnCase Legal Journal, Guidance Software, December.
Rod, M. (2002) ‘Options in computer forensic tools’, Computer Fraud and Security, November,
No. 11, pp.8–11.
Rogers, E.T. (2003) Diffusion of Innovations, Free Press.
Stephenson, P. (2004) ‘The right tools for the job, digital investigation’, The International Journal
of Digital Forensics and Incidence Response, Vol. 1, No. 1, pp.24–27.
White, L. (1996) ‘Maladjusted contrivances and clumsy automation: a jurisprudential
investigation’, Harvard Journal of Law and Technology, Vol. 9, No. 2.
Wick, C., Avramov-Zamurovic, S. and Lylem, J. (2004) ‘Hard disk interface used in computer
forensic science’, Proceedings of the IEEE Instrumentation and Measurement Technology
Conference, May.

Note
1 Digital Forensics Research Workshop, www.dfrws.org

View publication stats