FIELD METHODS

10.1177/1525822X04266503
Ice / TECHNOLOGICAL ADVANCES IN OBSERVATIONAL DATA COLLECTION ARTICLE

Technological Advances in
Observational Data Collection:
The Advantages and Limitations of
Computer-Assisted Data Collection

GILLIAN H. ICE
Ohio University

Quantitative behavioral observation can add a great deal of depth to anthropologi-

cal research. The development of laptops, handheld computers, and observational
software has given researchers enhanced flexibility in sampling strategies and assis-
tance in training and establishing reliability; it has also greatly reduced the amount
of time required for data entry. With technological advances, more researchers
should be able to incorporate quantitative behavioral analysis into their projects.
Although computer-assisted data collection offers many advantages, it is not without
pitfalls and limitations. One of the primary disadvantages is the potential loss of data
resulting from battery failure, computer error, or operator error. This article
reviews the basic elements of observational data collection and considers the advan-
tages and disadvantages of computer-assisted quantitative observational data col-
lection for human field research, with a focus on The Observer software and
associated hardware from Noldus Informational Technology (© 1995–2003).

Keywords: behavior technology; data collection; ethogram; real-time

observation

Quantitative behavioral observation can add a great deal of depth to anthro-

pological research. With the exception of primatology, quantitative behav-
ioral observation is not commonly used in anthropological research, yet it
has numerous applications in the fields of ethnography, human ecology, and
human biology. Whether conducting ethnographic, ethological, or human
biological research, observational data are extremely laborious to collect.
The mode of recording and sampling strategy can greatly affect the required
training time and person hours required for data entry and analysis. When
observational data are collected with paper and pencil, complex sampling
strategies, particularly continuous sampling (real time), are challenging to
conduct.

Field Methods, Vol. 16, No. 3, August 2004 352–375

DOI: 10.1177/1525822X04266503
© 2004 Sage Publications
352
 Ice / TECHNOLOGICAL ADVANCES IN OBSERVATIONAL DATA COLLECTION 353

The development of laptops, handheld computers, and observational soft-

ware has given researchers enhanced flexibility in sampling strategies and
assistance in training and establishing reliability; it has also greatly reduced
the amount of time required for data entry. With technological advances,
more researchers should be able to incorporate quantitative behavioral analy-
sis into their projects. Although computer-assisted data collection offers a
great deal of advantages, it is not without pitfalls and limitations. One of the
primary disadvantages is the potential loss of data resulting from battery fail-
ure, computer error, or operator error. This article reviews the basic elements
of observational data collection and considers the advantages and disadvan-
tages of computer-assisted quantitative observational data collection for
human field research, with a focus on The Observer software and associated
hardware from Noldus Informational Technology (1995–2003).
Several software options currently exist for observational data collection
(see appendix), although they differ tremendously in design, flexibility, and
power. For those without computer programming skills, the most widely
available programs are produced by Noldus (The Observer), Education Con-
sulting, Inc. (BEST), and Psychsoft (Systematic Observation). There are also
a number of producers of proprietary software (Kahng and Iwata 1998). With
one notable exception, the proprietary programs are primarily used by the
creators, are difficult to get information about, and have minimal technical
support. MOOSES software, created by Jon Tapp and Joe Wehby at
Vanderbilt University, has been used widely, particularly by education and
disabilities researchers. Technical support is provided at cost, but the
software itself is free and available on the Internet.
All the software listed above are designed for observer-based (rather than
participant-based) quantitative behavioral observation. They are not appro-
priate for use with participant diary data nor are they equipped for qualitative
observational data. With the exception of The Observer from Noldus, the
software has primarily been designed by educational researchers and thus
has some limitations in flexibility for anthropological and ethological
research. In this review, I discuss issues in designing a protocol for observer-
based quantitative observation and review Noldus software, with which I
have the most experience. The software listed above has been previously
reviewed elsewhere (Kahng and Iwata 1998; Sidener, Shabani, and Carr in
press). Those interested in software for conducting interviews, qualitative
data, or diary data should refer to the following references: Johannes et al.
(2000); Greene (2001); Gravlee (2002); Fletcher et al. (2003).
Noldus is based in the Netherlands, with offices in Germany and the
United States. Noldus offers three primary products: The Observer,
EthoVision, and Ultra Vox. EthoVision, an automated behavioral tracking
354 FIELD METHODS

program, and Ultra Vox, an automated vocalization tracking program, are

primarily used by animal biologists. The Observer is used in a number of
human and nonhuman animal settings. The Observer, an observational soft-
ware program, can be used for live or videotaped data-collection protocols.
The Observer is not appropriate for collecting qualitative ethnographic data,
nor is it suitable for participant-recorded behavioral diaries. For those col-
lecting quantitative observational data, it allows great flexibility and power
in collecting data using a variety of sampling strategies. The Observer can be
used to collect observational data from a desktop, laptop, and a variety of
handheld computers.

OBSERVATIONAL DATA COLLECTION:

PROCESS AND DEFINITIONS

Observation, particularly when performed in a naturalistic setting, is very

time consuming and expensive and requires an extensive amount of training.
Behavioral observation can be collected in a laboratory or naturalistic/field
setting. By and large, anthropologists collect data in naturalistic settings,
which allows more realistic data collection but also limits the data collection
technology that can be used.
There are numerous considerations when deciding on an observation pro-
tocol. These include coding technique, types of measures (latency, fre-
quency, duration, or intensity), sampling procedure (continuous vs. inter-
val), collection technique (quantitative vs. qualitative), and method of
analysis. Overall, the use of computer-assisted technology allows greater
flexibility and power in the development and implementation of an
observational protocol.
When planning an observational study, one needs to develop an
“ethogram,” or a catalog of behaviors/facial expressions that describe the
typical behaviors of a group (Martin and Bateson 1993). There are two
approaches to coding: social scheme (functional scheme [e.g., sad, angry]) or
physical scheme (empirical description of expressions [e.g., forehead wrin-
kled]; Lerner 1979; Bakeman and Gottman 1997). Both schemes require
inference. Social schemes require inference before analysis; physical
schemes require interpretation during or after analysis. For coding purposes,
you should not be especially concerned with being truly objective but rather
that the observations are reproducible (i.e., good interobserver reliability).
Codes should be distinct, or precisely defined, homogeneous, and simple to
identify but detailed enough to answer the question. In some cases, you may
 Ice / TECHNOLOGICAL ADVANCES IN OBSERVATIONAL DATA COLLECTION 355

want mutually exclusive categories, but this is not always necessary. How-
ever, non–mutually exclusive categories are more of a challenge to analyze.
When deciding on a coding scheme, you may use a splitting or lumping
approach. In some circumstances, you may want a very detailed coding
scheme. In other cases, you may want more inclusive categories for analysis.
You may want to use a more detailed scheme as a first step and group behav-
iors or expressions into larger categories during analysis. To develop a cod-
ing scheme, it is important to do some initial unstructured observation. The
coding scheme should be tested before beginning the study, or you should
not count the first few observations in analysis to avoid “drifting” or
changing of the scheme after research has started (Martin and Bateson 1993).
There are several kinds of measures you can collect. The measure of inter-
est will determine sampling method.

1. Latency: The time from a specified event to onset of behavior.

2. Frequency: The number of occurrences per a specified amount of time.
3. Duration: The length of behaviors.
4. Intensity: There is no universal definition on intensity; you must make judg-
ments about what constitutes important components of intensity (typically,
people use a scale of 1–n, [e.g., intensity or local rate: the number of acts per
unit time performing a discrete activity, e.g., the number of times yelling over
five-minute intervals]).
5. Sequence: The sequence of behaviors. Sequence can be recorded as real time
or as the pattern of behaviors.

Once you determine the measure of interest, you must choose a sampling
strategy. There are two important components of sampling: who is being
sampled and how the data are collected. There are several general
approaches, as follows:

1. Ad libitum: There is no systematic constraints on when or what is recorded.

You can follow one individual or a group. Generally, one records behaviors
that are most obvious. Data collected in this manner cannot be quantified
(Lerner 1979).
2. Focal person: One individual is followed at a time (continuous or specified
time intervals). This is the most common approach in human behavioral
research.
3. Scan sampling: At specified time intervals, a group of individuals are
scanned, and the behavior of each individual is recorded. This approach may
be particularly useful in research on social interactions or group-based
interventions.
4. Combined approach: Simultaneous focal person and scan sampling. This
technique allows examination of interactions and possible influence of others
on the behavior of the individual.
356 FIELD METHODS

5. Behavior sampling (all occurrence sampling, event recording): Record each

observed occurrence of a particular behavior (with or without regard to the
actor).

Observations can be initiated by a timed interval or the occurrence of an

event. Sampling is generally divided into continuous event recording versus
timed sampling. With event recording, an observer records the time and
duration (onset/offset) of a behavior or expression of interest. This method
allows the determination of true frequencies and duration. Without the aid of
a computer, continuous event sampling is extremely difficult to conduct
because maintaining the start and stop of a variety of behaviors simulta-
neously with a stopwatch is challenging. With timed sampling, on the other
hand, observations are based on a specified time interval; thus, duration and
frequencies are estimated. The accuracy of such estimates will depend on
procedure, particularly the size of time intervals. A timed sampling proce-
dure is far easier to conduct with a paper and pencil as only a timer is needed
to cue an observation, which can easily be recorded on a check sheet or table.
There are two basic methods of timed sampling: instantaneous and one-
zero. With both, one collects data at specified time intervals. With instanta-
neous sampling, the behavior of interest is coded at a point in time (is it occur-
ring or not, or what behavior is occurring). With one-zero sampling, the
observer records if a behavior occurred within an interval of interest. For
example, if a time interval is five minutes, the observer records whether an
event has occurred within the five minutes. The frequency within the time
interval is not recorded; thus if a behavior occurred ten times, it is recorded
the same as if it had only occurred once. Instantaneous sampling produces the
proportion of sample points when a behavior is occurring. Continuous sam-
pling allows the analysis of latency, frequency, duration, and sequence of
behaviors. Timed sampling can only provide estimates of frequency and
does not allow for the analysis of latency, duration, or sequence. Continuous
sampling, however, is tiring for observers and may result in greater error.
Observational data can be collected with pencil and paper, an event
recorder, or a computerized recorder. For continuous sampling, computer
recording is the preferred method to reduce observer burden and increase
accuracy. When using a timed sampling procedure, any recording method
can be used. Pencil and paper are preferred when one does not have a speci-
fied ethogram (e.g., data on activities) or when there is an infinite number of
states. Computerized data recording reduces the amount of time needed to
analyze data since the data can be loaded directly into a spreadsheet or
database program.
 Ice / TECHNOLOGICAL ADVANCES IN OBSERVATIONAL DATA COLLECTION 357

One of the biggest challenges to observational data is collecting reliable

data. As the complexity of the sampling and coding scheme increases, so
does the time needed for training and establishment of reliability. Calculating
interrater reliability is a tedious and time-consuming task when observa-
tional data are collected by pencil and paper. The ability to conduct reliability
analysis with ease is one of the greatest advantages of computer-assisted data
collection. Interrater reliability can be calculated quickly and therefore can
assist in training.

DESIGNING THE OBSERVATIONAL PROTOCOL

WITH THE OBSERVER SOFTWARE

The Observer software is a flexible program that allows a researcher to

develop an observational protocol, called a “configuration,” using a variety
of sampling strategies. The Observer is menu-driven, Windows-based soft-
ware. Noldus has made continuous improvements in its software. Observer
4.1 provided significant improvement over 3.0, namely, a large overhaul in
user interface and enhancement in documentation. The primary improve-
ments from 4.1 to 5.0 are related to video-based observation; however, there
are new features in the configuration, data analysis, and back-up procedure.
In addition, there is a new feature, a report generator, which provides a
detailed description of the protocol, data files, and analysis summaries.
The Observer comes as a base package (Observer Basic) for a desktop
computer with the design and analysis modules as well as an observational
module. In addition, The Observer Mobile is used with handheld devices.
The Observer 5.0 requires, at minimum, PentiumII (266 MHz) with 32 MB
of RAM and at least 50 MB of free space. It runs on Windows 98, Windows
2000, and Windows XP. To protect the license, Noldus has a hardware key
that is fitted into a printer or USB port. The software comes on a CD-ROM.
Academic prices range from $2,000 for The Observer Basic software to
$5,500 for The Observer Video-Pro system.1 Additional fees are required for
use on multiple computers.
The primary Observer user interface is a “workspace,” which contains
related projects (Figure 1). There are three primary modules: configuration,
observation, and analysis. The first step to using the software is to develop a
configuration, which allows the researcher to set the sampling strategy, pro-
gram independent variables, identify participants, and set up an ethogram.
There are three options for sampling strategy (called “recording method” in
The Observer): “continuous recording”; “interval: states”; and “interval:
358 FIELD METHODS

FIGURE 1
Observer Workspace

events.” With continuous sampling, one or more individuals are observed

and the onset and offset of behaviors, both events and states are recorded.
The duration of observation can be predetermined or open-ended. Interval:
states is equivalent to instantaneous sampling. A focal sample or scan sam-
pling strategy can be used with this method. The interval: events method is
equivalent to one-zero sampling. Besides the inherent disadvantages of one-
zero sampling, it does not allow the researcher to assign modifiers to behav-
iors (described below). With both interval methods, the time period for the
interval is set in the configuration.
Independent variables to be used as grouping variables in the analysis can
be specified in the configuration. Independent variables are variables that
remain constant throughout the observation period, such as location of obser-
vation, environmental conditions, or attributes of the participants being
observed. I have not used this particular setting as the data that I might be
interested in using for grouping are not readily accessible during observation
(i.e., participant attributes) and are entered into other programs; however,
this feature would useful in a number of other protocols. As long as you can
 Ice / TECHNOLOGICAL ADVANCES IN OBSERVATIONAL DATA COLLECTION 359

link the observational data files with other data collection files, setting inde-
pendent variables is unnecessary. It would be helpful for recording features
specific to the observational time period or location, such as temperature or
setting.
For protocols that require the observation of multiple participants, sub-
jects can be identified within the configuration. Subjects can be recorded as
“actors,” “receivers,” or both. Within the configuration, subject’s name or
identification, description, and code are specified.
The ethogram is divided into behavioral classes and behavioral elements.
Behavioral classes are a group of related behaviors. For example, in my
research on persons with dementia, I have five behavioral classes: emotion,
posture, activity, agitation/inappropriate behaviors, and location. Behavioral
elements are individual behaviors per class. For example, in my ethogram,
there are three elements to posture: sitting, standing, and reclining. The
behavioral elements within a behavioral class must be mutually exclusive
and exhaustive. If there are behaviors that you want to collect simulta-
neously, you must define them in separate classes. For example, I might
record that a participant is receiving personal care (activity class) and refus-
ing such care (agitation) at the same time. Creating an exhaustive list of
mutually exclusive behaviors can be challenging and requires extensive
unstructured observation and pretesting of configurations prior to the
beginning of data collection.
Behavior can be recorded as events or states. States have a defined begin-
ning and end and a measurable duration. Events take only an instant, such as a
vocalization or fast body movement. Only frequency of events can be
counted. If using continuous sampling, a clear beginning and end to a state
must be reliably recognized. Transitional behaviors may be particularly dif-
ficult to record reliably. For example, Lawton, Van Haitsma, and Klapper
(1996) noted that the onset and offset of transferring a frail elder between two
locations has poor reliability, leading them to eliminate this behavior from
the configuration. Similarly, a common behavior that is examined in research
on agitation in people with dementia is wandering. As it is impossible to
determine someone’s motivation for walking (to go somewhere or simply to
wander) until after the act has been completed, I wasn’t able to include this
behavior in my configuration.
In the configuration, you list the behaviors, provide a description, a code,
modifiers, and label as a state or event (see Figure 2). Behavioral modifiers
can be used in combination with behavioral elements. These are used to spec-
ify the nature of a behavior. For example, you might want to have a behav-
ioral element of play, with a modifier of being alone or with others. Within
the configuration, you can set default behaviors within each class, which is
360 FIELD METHODS

FIGURE 2
Observation Classes and Elements

handy if your participants are frequently in one state. In some hardware plat-
forms, the observation always begins in the default categories, which I found
handy. This is not the case with the Pocket PC. If you are recording data on
more than one participant by continuous sampling, you must define chan-
nels. “Channels are unique combinations of subjects and behavioral classes.”
(Noldus Information Technology 2003:100). I have not used this feature;
however, collecting continuous data on multiple participants adds signifi-
cantly to the complexity of the protocol and likely increases the possibility
for errors.
Once you have completed the configuration, there is a feature to review
and check for errors. Once you use a configuration to collect data, the config-
uration is locked and cannot be revised without separating the data files from
the configuration. While it is possible to edit the configuration and re-add the
older data files, the revised configuration needs to be compatible with the old
data files, or it cannot be added to the project. This should not be a problem as
long as sufficient time has been taken to test the configuration prior to the
beginning of a study.
 Ice / TECHNOLOGICAL ADVANCES IN OBSERVATIONAL DATA COLLECTION 361

CONDUCTING THE OBSERVATION:

THE HARDWARE

Observational data can be collected on a variety of hardware platforms,

including a desktop or laptop computer and a variety of handheld devices
(see Table 1 for a comparison of handheld devices). To conduct observations
with a desktop or laptop, the user simply goes to the observational module
(Figure 3). I will focus on the Psion Workabout and iPAQ Pocket PC as these
are the two handheld devices that I have used. With the exception of the
Workabout, all handheld devices run on Windows CE. Unfortunately, they
have not designed compatible software for a Palm platform. As the Palm
platform is more commonly used and has greater software variety overall,
this is a disadvantage. Those wanting to use computer-assisted interviews on
a Palm platform (e.g., with Entryware software [Gravlee 2002]) will need
multiple handhelds.

Desktop/Laptop
Collecting data via a desktop or laptop computer obviously requires a
fixed location. If using a continuous event recording protocol, the observa-
tion must take place in a limited area (e.g., a lab) so that the participant(s) can
always be viewed from a fixed point. Scan sampling would be feasible in a
more open area if you were interested in the use of space or group behavior,
such as what is used in behavioral mapping. With VideoPro, you can theoret-
ically conduct continuous event recording if you are willing to spend the time
taping and then coding on the computer (Figure 4). I feel that following
someone around with a video camera is overly invasive. Furthermore, taping
at one point and coding later can significantly add to the cost and time of a
project. As the observation occurs within The Observer Basic, file transfer is
not an issue.

Psion Workabout
The Workabout weighs 325g (dimensions: 18.9 × 9.2 × 3.5 cm) and runs
on a 3V lithium backup battery and two 1.5V AA batteries or a rechargeable
Psion Nickel-Cadmium battery pack. The Psion battery can be recharged on
a docking station2 or AC Mains adapter with an LIF converter. It has 2 MB
RAM and two Solid State Disk (SSD) drives that provide up to 16 MB addi-
tional memory. The Workabout comes with one SSD that contains The
Observer software. It runs on a different operating system (EPOC) than other
handhelds currently supported by Noldus. The Workabout is considered one
362
TABLE 1
Handheld Computers Supported by The Observer

Manufacturer Compaq Hewlett-Packard Hewlett-Packard Dell Panasonic Psion

Model iPAQ 3800/3900 iPAQ Pocket PC h2210 iPAQ Pocket PC h1930 Axim X5 Toughbook 01 Workabout MX

Operating system Pocket PC 2002 Pocket PC 2003 Pocket PC 2003 Pocket PC 2003 Pocket PC 2002 native (EPOC)

Data input pen/touch screen, pen/touch screen, pen/touch screen pen/touch screen, pen/touch screen keyboard
optional external optional external optional external and keyboard (57 keys
clip-on/foldable clip-on/foldable clip-on/foldable alphanumeric)
keyboard keyboard keyboard

Screen size color, 3.5" color, 3.5" color, 3.5" color, 3.5" color, 3.5" B/W LCD

Type TFT LCD TFT LCD TFT LCD TFT LCD TFT LCD

Resolution 240 × 320 240 × 320 240 × 320 240 × 320 240 × 320 240 × 100
a
Environment in use 0°C–40°C, 0°C–40°C, 0°C–40°C, unspecified 20°C–50°C, 20°C–60°C,
10%–90% RH 10%–90% RH 10%–90% RH 30%–80% RH IP54 up to 95% RH IP54
a
Environment storage unspecified 20°C–60°C 20°C–60°C unspecified 25°C–60°C, 25°C–60°C
30%–90% RH
 Dimensions (mm) 84 × 15.9 × 134 76.4 × 15.4 × 115.4 69.8 × 12.8 × 113.3 81.5 × 18 × 128 98 × 41 × 173 92 × 35 × 189

Weight (g) 184 144,2 124 196 490 325

Battery life 10 12 8 unspecified 8–24 10–12

b
indication (hrs)

Internal memory 64 MB with SD slot 64 MB with SD slot 64 MB with SD slot 32/64 MB with 32/64 MB with 2 MB with
& CF slot SD slot & CF slot SD slot & CF slot 2 SSD disc slots

Communication USB & optional USB & Bluetooth USB USB or serial, USB & infrared, Serial
Bluetooth optional Bluetooth and optional
or wireless LAN wireless LAN

Audio feedback Yes Yes No Yes Yes No

Remarks Designed for Suitable for field

durability use; shock resistant
NOTE: The table shows the main specifications of the handheld devices supported by The Observer Mobile at this moment, as specified by their manufactur-
ers. For detailed specifications and up-to-date prices, refer to the manufacturers’ Web sites or contact Noldus.
a. Dell does not give these specifications; they state the unit works under “normal circumstances.”
b. Depends on usage conditions; data should be saved to alternate memory device whenever possible.

363
364 FIELD METHODS

FIGURE 3
Observational Module in the Observer Basic

of the more rugged models, and the manual states that it is splash proof and
can withstand a 1-meter drop onto concrete. The LCD screen is relatively
small and is in gray scale with a backlight for low light environments. The
majority of the Workabout is a keyboard that is laid out alphanumerically
rather than as a standard keyboard. There is a shift and a “Psion key,” which
allows the use of additional characters as codes. The Workabout is comfort-
able to hold in one hand, freeing up the other to enter keystrokes, or held
between both hands, using your thumbs to enter keystrokes. The current U.S.
academic price of The Observer Mobile 5.0 with Psion Workabout is
$4,225.00. This includes The Observer Basic 5.0 software and one handheld.
To begin observation, the configuration is transferred via a “Psion 3
Link.” Unfortunately, to communicate between the Workabout and a PC, a
variety of settings must be changed on the Workabout. These settings must
be reset every time you reenter the communications mode on the Workabout.
Although this is just a minor inconvenience, it would be nice if the settings
could be saved. After the configuration is loaded, you can begin observation.
At the start of each observation session, the user enters the observer program,
 Ice / TECHNOLOGICAL ADVANCES IN OBSERVATIONAL DATA COLLECTION 365

FIGURE 4
Recording Observations with Observer Video Pro

loads the configuration by entering the appropriate file name to be used, and
presses a key to start the session. After the data file is named, observation can
begin. The observation module on the Workabout is similar to that of the
desktop; however, the screen is obviously smaller (see Figure 5).
The screen layout can be modified to a small extent but generally notes the
time of the start of the observation, the current time, and the end time. It also
displays which behavior is active for each behavioral class. Unfortunately,
the screen is not large enough to display all of the behavioral classes. View-
ing the current behaviors is an important check for the observer, who may
forget to “turn off” a behavior. In my research, we had a number of errors,
particularly during training, as the majority of the time our participants dis-
played no agitative behavior; therefore, under the agitation category, most of
the time they were in “non,” or no-agitative behavior. Since you do not actu-
ally turn off a behavior (the transition to a new behavior turns off the old
behavior), this was a common error. If the current behavior was displayed on
the screen, it would allow an instant check. I made this error less frequently
366 FIELD METHODS

FIGURE 5
Psion Workabout Screens

when using the PocketPC, which displays all classes. Of course, with the
Workabout, you can view the current behaviors by hitting, “psion-I,” but this
extra step requires more effort for the observer. One significant limitation of
the Workabout is the inability to easily edit while observing.
 Ice / TECHNOLOGICAL ADVANCES IN OBSERVATIONAL DATA COLLECTION 367

The Workabout (and the Pocket PC) records time in seconds and connects
behaviors to time elapsed since the beginning of observation, not wall-clock
time. This is fine if the researcher only plans to analyze the behavioral data;
however, it makes matching to other types of data more complicated. I have
participants wear ambulatory blood pressure monitors, and I collect salivary
cortisol samples twice during observation. To match the observational data
to the blood pressure data (which is taken every fifteen minutes), either the
observational data have to be converted into clock time or the blood pressure
data have to be converted to elapsed time since the beginning of the observa-
tion. This extra step is not hard; it is just a nuisance. I hope that in future ver-
sions, Noldus will provide an option for wall-clock time.
Overall, I found using the Workabout easy. It’s comfortable to hold and
easy to use once the codes are mastered. As with any hardware, the more
codes that need to be remembered, the longer the required training time.
Besides the small display screen, the biggest disadvantage of the Workabout
is that it is easy to inadvertently hit the off key. If the off key is hit or the
Workabout is dropped, you return to the same screen, but the time is reset to
zero and the program does not function properly. You must exit the observa-
tion and start with a new file. Then, after this data file is transferred to your
PC, you must edit the file to remove the zero time from the end of the obser-
vation. Unfortunately, it took some time for the research team to realize what
was happening, and we lost a small portion of data. Even if you catch this
immediately, you then have to merge files prior to analysis. In addition, the
keys are not protected for transport. This means that the Workabout can be
turned on during transport. I carried them in a Baumometer box, and
although this worked, it was not the best solution, particularly since I always
brought a back-up in the event of battery failure.
While the rechargeable batteries have a ten to twelve-hour battery life, I
had some problems with the battery packs. Two of my battery packs failed to
recharge; the technical support offered some solutions but I was unable to use
them. Of course, you can use AA batteries as an alternative, but since I had
invested in the charger, I found this frustrating. I also had a Workabout that
continuously blew a fuse. The technical support office never did figure out
why this was, but they replaced the Workabout and the battery pack. The last
problem that I had was loss of data from one four-hour observation because
the backup battery died. This was because the file was saved on the internal
drive. If data are stored on the SSD, then this problem is avoided. Unfortu-
nately, Noldus only provides one SSD that contains the program. If you do
not have too many files stored on the Workabout, then this is not a problem;
however, I recommend purchasing an extra SSD for each Workabout.
368 FIELD METHODS

The Pocket PC
The Pocket PC weighs 144.2g (76.4 × 15 mm) and runs on a rechargeable
lithium-ion battery. The battery is charged when the Pocket PC is on the
docking cradle. The iPAQ has 64 MB of internal memory and an SD and CF
slot. The iPAQ has a plastic cover that opens to the side. I liked the cover for
the protection it afforded during transport, but I found it uncomfortable to
hold. If the cover is removed, it is less awkward to hold. While I would not
want to use the Pocket PC without the stylus, it adds another piece of equip-
ment to hold and possibly lose—this is potentially problematic if working in
a remote area. Perhaps it was because of familiarity, but I found that the
Workabout without a stylus to be more ergonomically sound. The Observer
Mobile 5.0 with HP iPAQ costs $4,020.00. This includes The Observer Basic
5.0 software, Pocket Observer 2.0 software, HP iPAQ h2215 handheld com-
puter (with AC adapter, battery, cradle), and a 128-MB memory stick.
To begin observation, the configuration is transferred via Microsoft
Active Sync while the iPAQ is on the docking cradle. The configuration then
must be loaded into The Observer. If using only one configuration, this only
has to be done once as opposed to the Workabout for which you have to load
the configuration for every new session. You start observation and you must
enter the current states (defaults are not entered). The screen is a good size
and is in color. A number of different options are available to set up the
screen. The codes are displayed on the bottom of the screen (see Figure 6). To
turn on a behavior, you simply tap the screen with the stylus. I was able to fit
forty-nine three-letter codes at a reasonable font size. However, with more
codes, they might not all fit on the screen for easy viewing. During observa-
tion, you can either view the sequence of observations (like the Workabout)
or display the active behaviors by each class.
Although there are no data to support this, I suspect that remembering
codes to be able to recognize them on a screen is easier to do than recalling
three-letter codes from memory. I did not train anyone on the Pocket PC, but
the training with the Workabouts took approximately three months. I suspect
that the Pocket PC would require slightly shorter training time. The Pocket
PC also has a number of other programs, adding to the overall utility of the
iPAQ over the Psion Workabout. Because it is menu driven, it is probably
easier for people with limited computer skills to use compared to the
Workabout.
There are several helpful features on the Pocket PC. Data can easily be
edited during observation. You can place time-stamped markers to describe
activities that do not fit the configured codes. You can also add a note with
time to add more extensive field notes. The marker is listed in the event code
 Ice / TECHNOLOGICAL ADVANCES IN OBSERVATIONAL DATA COLLECTION 369

FIGURE 6
Pocket PC Observation Screen

as a coded line, while the note is not. There are several options that you can
use to conserve power, and you can check the status of your battery during
observation. One of the features is to turn the power off after a certain amount
370 FIELD METHODS

of time without use of the iPAQ. As I often sat watching people sleep for long
periods of time between active periods, I appreciated this opportunity to save
battery power. If the power turns off automatically or if you inadvertently
turn the iPAQ off, the clock keeps running and the observation continues as
normal when you turn the power on. I had no situations where I had to create
more than one file per observation time.
Overall, I found the Pocket PC easy to use and in many ways preferable to
the Workabout. It is definitely less rugged than the Workabout. Although the
battery life is twelve hours, you must have access to electricity to charge the
battery. As this is problematic in many field settings, the Pocket PC might not
be ideal for anthropological research. My project was set in the United States,
but I lost battery power while the Pocket PC was not being used in the field
bag. I had not thought about placing it on the charger since it was not in use.
As a result, I lost a significant amount of data. Unfortunately, the default for
saving data on the Pocket PC is the main memory. These data files will be lost
if power is lost. You can save to something called the “iPAQ file store,” but
this is not indicated in the manual and I did not find this out until after I had
lost data. I strongly recommend that you save to this alternative location, and
I hope that Noldus will set this to the default location in future versions.

TRANSFERRING AND ANALYZING DATA

To transfer data files to The Observer basic from the Workabout, you use
the same connector as used for transferring the configuration. Again, you
have to reset the communication settings. Once the data are transferred, they
are added to the project on your workspace. To transfer files from the Pocket
PC, you simply set the iPAQ in the cradle and hit “Microsoft Active Sync.”
These files are saved to a folder and then added to the project. You can also
retrieve files using a menu feature in The Observer Basic on your PC.
There are three analysis features in The Observer: reliability, elementary
statistics, and lag sequential analysis (see Figure 7). Before analysis, you can
view your data in the time-event view. You can view the observation in a
table or plot. The format of the time-event table was changed from version
3.0 to 4.0/5.0. For matching with the physiological data, the table layout in
version 3.0 was easier to use; however, the data are not easy to connect to
blood pressure data in any of the versions. This must be done by hand.
As many researchers also collect physiological data, I suspect Noldus will
work to make this more user friendly. The time-event plot displays each class
of behaviors in a bar with different colors for different behaviors. I have
found no use for this feature. Prior to viewing the data in the time-event view
 Ice / TECHNOLOGICAL ADVANCES IN OBSERVATIONAL DATA COLLECTION 371

FIGURE 7
Analysis Screen

or analyzing the data, you must create a data profile. This extra step allows
you to select a set of data files to be analyzed repeatedly, but I found the extra
step to be a nuisance. In the data profile, you can select an option to add all
new data files to the profile. As a result, if you are analyzing data or creating
time event views each time you add files (which is helpful for data manage-
ment), you have to create new data profiles each time. In version 3.0, there
was no data profile step, and you just selected the observation as you used the
time event view or data analysis. In the data profile, you can select individual
data files that you want to be treated as one observation. This was also easier
to do in version 3.0. Of course, you can just rerun the analysis on all files in
the data profile each time you add an observation, but this is time consuming,
particularly for slower computers.
The biggest improvement from version 3.0 to the later versions is in the
analysis modules. The reliability feature is easy to use and provides per-
centage agreement, index of concordance, Cohen’s Kappa, and Pearson’s
Rho. Version 3.0 only calculated percentage agreement. Furthermore, there
are four methods of analysis: duration based, frequency based, duration/
372 FIELD METHODS

sequence based, and frequency/sequence based. The method chosen will

depend on the purpose of the data collection (see Jansen et al. [2003] for a
good review of methods and The Observer software). This analysis feature
alone makes the use of computer-assisted data collection superior as the time
and effort required to calculate these by hand would be beyond most
researchers.
Within elementary statistics, The Observer calculates the total number of
times a behavior is observed, latency, rate, total duration (in seconds and %
of observation), mean duration, minimum duration, standard deviation, stan-
dard error and confidence interval of duration, median duration, and mean
total of observations. The desired statistics can be selected in the analysis set-
tings. These settings can be saved in a statistics profile, allowing the user to
set up a variety of settings for different purposes. Lag sequential analysis
allows the analysis of behavioral patterns. The analysis results are displayed
in a spreadsheet and are easily exported or copied into Excel or a statistical
package. The output is greatly improved from version 3.0, which did not
allow easy transfer.
Overall, The Observer 5.0 is more user friendly than the earlier versions.
The documentation and help module within The Observer have also greatly
improved. Probably the best feature of The Observer software and associated
hardware is the technical assistance. Assistance is available on-line, by e-
mail, and by phone. All of the Noldus staff with whom I have worked have
gone above and beyond the basics to help me as necessary. Furthermore, they
clearly listen to their clients and try to continuously improve software to fit
user needs. Noldus also supports a listserve for people to exchange ideas and
ask for help by other users. As the majority of users work with animals, I have
not found this useful, but I can see its potential.
My major complaint about the computer-assisted data collection in gen-
eral is the potential for data loss. I lost approximately twenty-four hours of
observations using the Pocket PC because of transfer errors from the Pocket
PC to my PC, user errors, software errors, or computer errors. The actual
cause was never determined. I had a tremendous amount of help from several
people in technical assistance, although we were never able to recover this
data. In general, the software is designed to avoid this kind of error, so I can-
not rule out user error. Had I saved the data in my Pocket PC to the iPAQ file
store, of course, I could have reloaded the files, but this was not possible
because of the power failure discussed above. This should not necessarily
deter potential users, as data loss can be avoided with diligent battery charg-
ing and backup. Those who use the program in remote areas may want to opt
for the more rugged handhelds and those that have replaceable rather than
rechargeable batteries. Extra storage disks or a field laptop for frequent trans-
 Ice / TECHNOLOGICAL ADVANCES IN OBSERVATIONAL DATA COLLECTION 373

fer and backup of data are essential too. Furthermore, computer-assisted

technology likely reduces overall missing data, data errors resulting from
complex protocols, and errors resulting from data transfer from paper to
computer (Johannes et al. 2000; Gravlee 2002; Fletcher et al. 2003). This is
particularly important since the errors produced in these ways may be im-
measurable, whereas data loss resulting from battery loss at least is known.

CONCLUSION
Quantitative behavioral data can substantially add to research in anthro-
pology. They enable researchers to test a number of hypotheses that are
weakly tested by interview data. However, behavioral data are labor inten-
sive to collect and, without computer-assisted collection, very labor inten-
sive to process. The advances in technology, particularly those provided by a
responsive company such as Noldus, should encourage more anthropolo-
gists to add quantitative behavioral observation to their field techniques.
Researchers who are considering adding observation will need to allow
enough time for extensive field testing and training, so using these kind of
data would be challenging for dissertation projects. It may also be challeng-
ing for researchers without access to sufficient seed grants to pretest and pilot
such a protocol. Some researchers may want to consider the use of videotap-
ing behaviors, if time in the field is limited. Training could then occur in
one’s home institution. Alternatively, researchers can start with a paper-and-
pencil approach in the first field season and develop a computer-assisted
program for the following season.

APPENDIX
Observation Software Information

• Behavioral Evaluation Strategy and Taxonomy (BEST)

Educational Consulting, Inc.: http://www.skyware.com.
Reviewed by Sidener, Shabani, and Carr (In press).
• Multiple Option Observation System (MOOSES):
John Tapp & Associates: http://kc.vanderbilt.edu/jont/mintro.html
Phone: 615-443-0535.
• The Observer
Noldus Information Technology: http://www.noldus.com/
751 Miller Drive, Suite E-5
Leesburg, VA 20175-8993
Phone: 703-771-0440
Toll-free: 1-800-355-9541
Technical Support: 866-860-3580
374 FIELD METHODS

Fax: 703-771-0441
E-mail: info@noldus.com
• Systematic Observation Software (SOS)
Psychsoft: http://members.aol.com/Psychsoft/index.html

NOTES
1. Note that these prices apply to North America only and are subject to change (e.g., when
the exchange rate changes).
2. The docking station can charge four Workabouts at the same time. It cannot, however, be
used to transfer files contrary to the promotional materials.

REFERENCES
Bakeman, R., and J. Gottman. 1997. Observing interaction. Cambridge, UK: Cambridge Uni-
versity Press.
Fletcher, L. A., D. J. Erickson, T. L. Toomey, and A. C. Wagenaar. 2003. Handheld computers. A
feasible alternative to paper forms for field data collection. Evaluation Review 27 (2): 165–
78.
Gravlee, C. C. 2002. Mobile computer-assisted personal interviewing with handheld computers:
The Entryware System 3.0. Field Methods 14 (3): 322–36.
Greene, P. D. 2001. Handheld computers as tools for writing and managing field data. Field
Methods 13 (2): 181–97.
Jansen, R. G., L. F. Wiertz, E. S. Meyer, and L. P. J. J. Noldus. 2003. Reliability analysis of
observational data: Problems solutions, and software implementation. Behavior Research
Methods, Instruments & Computers 35 (3): 391–9.
Johannes, C. B., S. L. Crawford, J. Woods, R. B. Goldstein, D. Q. Tran, S. Mehrotra, et al. 2000.
An electronic menstrual cycle calendar: Comparison of data quality with a paper version.
Menopause 7 (3): 200–8.
Kahng, S., and B. A. Iwata. 1998. Computerized systems for collecting real-time observational
data. Journal of Applied Behavior Analysis 31:253–61.
Lawton, M. P., K. Van Haitsma, and J. Klapper. 1996. Observed affect in nursing home residents
with Alzheimer’s disease. Journals of Gerontology : Psychological Sciences and Social Sci-
ences 51 (1): 3–14.
Lerner, P. 1979. Handbook of ethological methods. New York: Garland STPM Press.
Martin, P., and P. Bateson. 1993. Measuring behavior. Cambridge, UK: Cambridge University
Press.
Noldus Information Technology. 2003. The Observer reference manual 5.0. Wageningen, the
Netherlands: Author.
Sidener, T. M., D. B. Shabani, and J. E. Carr. In press. A review of the Behavioral Evaluation
Strategy & Taxonomy (BEST) Software application. Behavioral Interventions.

GILLIAN H. ICE, PhD, MPH is an assistant professor of biological anthropology and

gerontology in the Department of Social Medicine at Ohio University College of Osteo-
pathic Medicine. Her research interests include human variation in stress and aging. She
 Ice / TECHNOLOGICAL ADVANCES IN OBSERVATIONAL DATA COLLECTION 375

conducts research with elders in long-term care settings and with grandparents in Kenya
who are caring for orphaned grandchildren. Recent publications include “Factors Influ-
encing Cortisol Level and Slope among Community Dwelling Older Adults in Minne-
sota” (Journal of Cross-Cultural Gerontology, in press), “Diurnal Cycles of Salivary
Cortisol in Older Adults” (with A. Katz-Stein, J. Himes, and R. L. Kane, Psycho-
neuroendocrinology, 2004), and “Blood Pressure Variation in the Institutionalized
Elderly” (with G. D. James and D. E. Crews, Collegium Antropologicum, 2003).