Journal of the Korean Physical Society, Vol. 56, No. 3, March 2010, pp.

10031010
Hidden Markov Model and Self-organizing Map Applied to Exploration of
Movement Behaviors of Daphnia magna (Cladocera: Daphniidae)
Yuedan Liu and Tae-Soo Chon

Department of Biological Sciences, Pusan National University, Busan(Pusan) 609-735

Sang-Hee Lee
Division of Industrial Mathematics, National Institute for Mathematical Sciences, Daejeon 305-340
(Received 18 September 2009, in nal form 23 December 2009)
Response behaviors of indicator species have been used for monitoring environmental distur-
bances. Markov processes were applied to elucidate behavioral changes of animals under chemical
stress. Movement of Daphnia magna in two dimensions was continuously observed before and after
treatments with an insecticide, diazinon, at low concentrations. The Self-organizing map (SOM)
was initially used to train complex movement data to classify dierent movement states. Subse-
quently, a hidden Markov model (HMM) was applied to sequence the movement states identied
by the SOM. The transition probability matrix (TPM) and the emission probability matrix (EPM)
were eciently estimated by using the HMM. Markov processes were separately observed before
and after the treatments, and the changes in the response behaviors of the indicator organisms were
demonstrated under stressful conditions.
PACS numbers: 02.50.Ga, 02.50.-r
Keywords: Hidden Markov model, Self-organizing map, Transition probability, Emission probability, Move-
ment pattern, Behavioral monitoring
DOI: 10.3938/jkps.56.1003
I. INTRODUCTION
The automatic detection of response behaviors of an-
imals has been considered as an ecient tool for bio-
monitoring in aquatic ecosystems. Behavioral changes
would be a suitable indicator system for detecting vari-
ous chemical agents [1]. Previous studies [24] reported
that movement behaviors of indicator species were ac-
cordingly patterned and that the patterned movements
appeared to change in a time sequence. The frequencies
of some selected patterns would be dierent before and
after the treatments [5]. We intended to elucidate the
changes in movement patterns as the time progressed and
check if Markov processes could govern the occurrence of
movement patterns of indicator specimens. Considering
that the observed data are prone to various experimen-
tal conditions and noise, we evaluated the stochastic pro-
cesses in movement patterns by using the hidden Markov
model (HMM) [6,7].
Application of the HMM is biologically attractive con-
sidering the fact that the underlying state of the ani-
mal can be modeled, along with analyses of the observ-
able behaviors. The HMM can be used to infer opti-
mal hidden states in organisms from the observed se-

E-mail: tschon@pusan.ac.kr
quence while conventional time series solutions can only
predict observable data. Two aspects are linked in illus-
trating behavioral states. Initially, the internal states of
the organisms are generated. Subsequently, the internal
states are projected outward to produce movement pat-
terns in the spatial and the time domains. The HMM
poses the question of optimal inference of a causal model
from the observable data [7]. If data for events (e.g.,
observable parameters) and states (e.g., behavior pat-
terns) are available from experiments, the advantage of
the HMM could be the provision of state sequence (e.g.,
transition probability) and the realization of event-state
relationships (e.g., emission probability).
In this study, we aimed to reveal state changes in
movement behaviors of an indicator species, Daphnia
magna, before and after chemical treatments. We ini-
tially obtained information on the behavioral states in a
heuristic manner by training the experimental data with
a Self-organizing map (SOM) [8]. The SOM classied
the movement patterns and was further capable of rec-
ognizing the movement patterns of new data segments.
By recognition, the SOM could provide the experimental
transition probability matrix (TPM). Subsequently, ob-
servable information (e.g., speed) was measured for the
corresponding data segment, and the categorized values
of the selected parameter were considered as events to al-
-1003-
-1004- Journal of the Korean Physical Society, Vol. 56, No. 3, March 2010
low emission probability matrix (EPM) to be produced.
Two computational methods, the HMM and the SOM,
in combination, were demonstrated in objectively char-
acterizing complex behavioral data to elucidate stochas-
tic processes in the response behaviors of the indicator
species.
II. MATERIALS AND METHODS
1. Test Organisms and Observation System
The stock population of the experiment species, Daph-
nia magna, was obtained from the Toxicology Research
Center, Korea Research Institute of Chemical Technol-
ogy (KRICT, Daejeon, Republic of Korea) and was cul-
tured in M4 medium [9]. Diazinon was selected as the
test chemical. The chemical has been regarded as one of
the major sources of insecticides that contaminate natu-
ral waters. Since diazinon aects the nervous system of
organisms [10], behavioral changes would be easily ob-
servable after the treatments. At a concentration of 1.0
ppb, diazinon (O, O-diethyl O-2-isopropyl-4-methyl-6-
pyrimidyl thiophosphate, 95 ng/l 5%, Sigma-Aldrich
Co) dissolved in dimethyl sulfoxide (DMSO; C
2
H
6
SO,
Sigma-Aldrich Co.), was directly added to the water in
the aquarium accommodating the individual specimens
of D. magna. The level of 48h LC
50
was 2.39 ppb [11].
Test organisms at one day old were randomly chosen
and were placed individually in a test chamber (60 mm
50 mm 10 mm). After the test specimens had been
acclimated to the observation system for 30 minutes be-
fore testing, they were individually observed from the
top view for 48 hours: for 24 hours before the treat-
ments, and subsequently for 24 hours after the treat-
ments. Fifty-ve individuals were observed in total. Five
individuals, however, showed continuous turning in ei-
ther the right or the left direction during the observa-
tion period in an unusual way. Considering that there
were reports in which individual organisms of D. magna
turned to the left or the right evenly [12], we excluded
these ve individuals from the data analysis. Distur-
bance to water in the observation area was minimized
during the monitoring period.
The movement trajectory of D. magna were recorded
by using an observation system consisting of an obser-
vation aquarium, a camera, a timer, an A/D interface,
and a software for an image recognition system (Fig. 1).
With a CCD TV camera (Kukjae Electronics Co., IVC-
841) the movement tracks of the test organisms were
scanned continuously in two-dimension by using a video
overlay board (Skydigital) in 0.25 s intervals during
the observation period. Subsequently, the captured im-
age was sent to the image recognition system in order to
locate target specimens in spatial and time domain [4,5].
Fig. 1. Movement tracks of Daphnia magna treated with
diazinon at 1.0 ppb: (a) before treatment; (b) after treat-
ment.
2. Self-organizing Map (SOM)
The movement tracks were classied by using the SOM
[8] by training the parameters of the movement segments.
The SOM performs a non-linear projection of data onto a
space in low dimension (conventionally two) and provides
a patterned map of input data without prior knowledge
[8]. The output layer consists of M N computation
nodes (i.e., 14 10 in this study) on a two-dimensional
grid (Fig. 2). The Euclidian distance (d
j
(t)) at node j
on the SOM between weight at iteration time t and the
input vector was calculated through learning processes:
d
j
(t) =
P1

i=0
[x
i
w
ij
(t)]
2
(1)
where x
i
is the value of parameter i, w
ij
(t) is the weight
between parameter i and the node j on the SOM, and P
is the number of the parameter.
The best matching neuron, which has the minimum
distance, is chosen as the winner. For the best matching
neuron and its neighborhood neurons, the new weight
vectors are updated as
w
ij
(t + 1) = w
ij
(t) +a(t)[x(t) w
ij
(t)] (2)
where t is the iteration time, and (t) is the learning rate.
The learning rate accordingly decreases as the system
Hidden Markov Model and Self-organizing Map Applied to Exploration of Movement Behaviors Yuedan Liu et al. -1005-
Fig. 2. The map trained by using the SOM for pattering movement segments of D. magna: (a) Six clusters classied by
the SOM (Empty circles represent the movement segments before the treatments while solid circles stand for the movement
segments after the treatments. The size of circles indicates the number of grouped segments in relative terms.) (b) Cluster
distances according to the Wards linkage method and (c) Prole of the parameters matching to clusters based on the trained
SOM. The values in the vertical bar in the top row indicate normalized parameters while the values, -1 and +1 in the
vertical bar in the bottom row present the left and the right directions, respectively.
converges. A detailed description regarding application
of the SOM to behavioral data is given in Ref. 8.
We randomly chose 10,000 continuous movement seg-
ments from 50 individuals before and after the chemical
treatments; then, we calculated the locomotory param-
eters of each movement segment. Based on preliminary
studies and previous reports [3,4], the following 13 pa-
rameters were selected for each 0.75 s movement seg-
ment with 3 sub-segments in 0.25 s interval: speed 1 -
3 (mm/s, speed of three sub-segments), average speed
(mm/s, averaged speed of three sub-segments), loco-
motory rate (mm/s, movement distance without stop
time), stop number (n, stop number during 0.75 s), an-
gular speed 1 - 2 (rad/s, angular change between sub-
segments), acceleration 1 - 2 (mm/s
2
, speed change be-
tween sub-segments), angular acceleration (rad/s
2
), and
meander 1 - 2 (rad/mm, angle change per movement dis-
tance between sub-segments).
By using the parameters extracted from the movement
segments as input data, we used the SOM to pattern
the movement segments (0.75 s). When the input data
were fed to the networks, all weight vectors between the
input parameter and the computation nodes (equation
(2)) were computed. The weights of the best-matching
unit and neurons close to it were adjusted towards the
input vector through an interactive calculation in the
SOM lattice. Thus, the similarity between the movement
segments is reected on the SOM map. In order to reveal
the degree of association between the SOM units, we
utilized the Ward linkage clustering [13].
3. Hidden Markov Model (HMM)
The most common HMM structure is a doubly
stochastic, rst-order model for representing time-
varying systems [7]. Rabiner [14] described an HMM
being characterized by ve elements: a number of states
(N), a number of events (K), an initial state distribu-
-1006- Journal of the Korean Physical Society, Vol. 56, No. 3, March 2010
tion (), state transition probabilities (A), and discrete
output probabilities (B). With the HMM formulation,
transitions between the behavioral states (i.e., movement
patterns) can be predicted from the state transition ma-
trix (A) and the state-dependent observation matrices
(B). The observation matrices dene the context sensi-
tive nature of the behavioral states and can be quantied
using a Bayesian algorithm.
According to Rabiner [14] and Jackson [15], a discrete
HMM can be summarized as
(1) A number of states N, x {1, . . . , N};
(2) A number of events K, k {1, . . . , K};
(3) Initial state probabilities, = {
i
} = {P(x
1
= i)}
for 1 i N;
(4) State-transition probabilities, A = {a
ij
} = {P(x
t
= j | x
t1
= i)} for 1 i, j N;
(5) Discrete output probabilities, B= {b
i
(k)} = {P(o
t
= k| x
t
= i)} for 1 i N and 1 k K,
where in (5), P(o
t
=k| x
t
=i) indicates the probability of
observation o
t
at time t to be the event k on the condition
that the state variable x
t
at time t is equal to i.
Given acceptable estimates of these quantities, solu-
tions are well known for (1) generating or predicting
observation and state sequences from the model by us-
ing Monte Carlo sampling methods, (2) determining the
most likely behavioral state sequence given the model
and an observation sequence (Viterbi algorithm), and
(3) updating the model estimate given new observations
(Baum Welch algorithm). More details about the Viterbi
or Baum Welch solutions are presented in Refs. 7 and 14.
We examined the possibility of six hidden states
identied by the SOM. In addition to dening the states,
we subsequently measured an observable parameter for
each movement. We selected the speed because speed
generally represents the activity of D. magna [16]. The
measured speed was divided into 9 classes, and the cate-
gorized speed was considered as an event. Based on the
solutions presented in Ref. 14, we calculated the esti-
mated TPM (between dierent states) and EPM (be-
tween states and events) by using the HMM toolbox
(MATLAB 7.8, The MathWorks, R2009).
III. RESUTLS
1. Movement patterns
Overall, the observed specimens were usually more ac-
tive before the treatments with diazinon (Fig. 1(a)). Al-
though there were individual variations in activity, the
movement tracks of the untreated D. magna appeared
to be smooth and linear, spanning a large area of the
arena used for observation. After the treatments, typi-
cal movements (Fig. 1(b)) were also observed although
response behaviors were diverse. The treatments caused
the test organisms to produce irregular short turns along
with shaking. The degree of activity and the range of
Fig. 3. Movement patterns of D. magna in 0.75 s segments
based on the trained SOM: (a) Continuous right movement
(P1) (b) right-left movement (P2) (c) stop (P3) (d) continu-
ous left movement (P4) (e) left-right movement (P5), and (f)
forward movement (P6).
swimming correspondingly decreased. The chemical ef-
fects were also shown by changes in speed and angular
speed. After the treatments, the speed decreased from
4.48 1.34 mm/s to 2.83 0.75 mm/s (paired-sample t
test, t = 8.94, DF = 49, P < 0.001) while angular speed
increased from 1.25 0.29 rad/s to 3.24 1.24 rad/s
(paired-sample t test, t = -11.29, DF = 49, P < 0.001).
The movement segments (in 0.75 s interval) were pat-
terned by using the SOM with the parameters extracted
from the movements stated above. Figure 2(a) shows
the clustered SOM based on the clustering dendrogram
(Fig. 2(b)). The movement segments before (empty cir-
cles) and after (solid circles) the treatments were accord-
ingly grouped on the SOM (Fig. 2(a)). Six movement
patterns were identied according to the Ward linkage
clustering [13] (Fig. 2(b)). The clusters accordingly cor-
responded to the following movement patterns: contin-
uous right (Fig. 3(a), cluster I, P1) and left (Fig. 3(d),
cluster VI, P4) movements, zigzag movements covering
right-left (Fig. 3(b), cluster II, P2) and left-right (Fig.
3(e), cluster V, P5), stops (Fig. 3(c), cluster III, P3), and
fast forward movements (Fig. 3(f), cluster VI, P6).
The proles of the parameters can also be correspond-
ingly visualized according to the location of the clusters
on the SOM (Fig. 2(c)). Cluster I on the upper left area
of the map (Fig. 2(a)), for instance, was mainly charac-
terized with relatively high speed, low stop number, and
continuous right turns. Cluster IV showed active move-
ments similar to cluster I, except for directional change.
Cluster II, right-left movements, was observed at the
mid-left area of the map and was presented as relatively
low speed and higher number of the stops, while the sym-
metric left-right zigzag movements in cluster V showed
parameters similar to those in cluster II, except for the
direction. Cluster VI showed most active movements in
forward direction with high speed and a minimal level
of stop and meander. In contrast, stops in cluster III
covered the segments with the most non-active move-
ment pattern with a maximal level of stop number and
Hidden Markov Model and Self-organizing Map Applied to Exploration of Movement Behaviors Yuedan Liu et al. -1007-
Table 1. Transition of movement patterns of D. magna
based on training using the SOM: (a) number of recognized
patterns before treatment (b) number of recognized patterns
after treatment (c) transition probability matrix before treat-
ment and (d) transition probability matrix after treatment.
(a)
H
H
H
H
H
t 1
t
P1 P2 P3 P4 P5 P6 Total Percentage
P1 616 245 84 122 151 655 1873 18.7%
P2 178 297 72 233 111 355 1246 12.5%
P3 105 76 329 127 88 185 910 9.1%
P4 125 152 99 621 269 756 2022 20.2%
P5 213 111 36 144 434 310 1148 11.5%
P6 540 230 134 465 277 1154 2800 28.0%
(b)
H
H
H
H
H
t 1
t
P1 P2 P3 P4 P5 P6 Total Percentage
P1 616 245 84 122 151 655 1873 18.7%
P2 148 438 767 272 323 160 2108 21.1%
P3 378 17 705 283 154 322 1859 18.6%
P4 78 97 222 318 198 274 1187 11.9%
P5 290 368 614 120 854 58 2304 23.0%
P6 229 171 203 259 161 198 1221 12.2%
(c)
H
H
H
H
H
t 1
t
P1 P2 P3 P4 P5 P6 Total
P1 0.329 0.131 0.045 0.065 0.081 0.350 1.00
P2 0.143 0.238 0.058 0.187 0.089 0.285 1.00
P3 0.115 0.084 0.362 0.140 0.097 0.203 1.00
P4 0.062 0.075 0.049 0.307 0.133 0.374 1.00
P5 0.171 0.089 0.029 0.115 0.348 0.248 1.00
P6 0.193 0.082 0.048 0.166 0.099 0.413 1.00
(d)
H
H
H
H
H
t 1
t
P1 P2 P3 P4 P5 P6 Total
P1 0.190 0.179 0.199 0.062 0.148 0.222 1.00
P2 0.070 0.208 0.364 0.129 0.153 0.076 1.00
P3 0.203 0.009 0.379 0.152 0.083 0.173 1.00
P4 0.066 0.082 0.187 0.268 0.167 0.231 1.00
P5 0.126 0.160 0.267 0.052 0.371 0.025 1.00
P6 0.188 0.140 0.166 0.212 0.132 0.162 1.00
low levels of speed and locomotory rate. These clusters
shown on the SOM were generally in accordance with
experience by observers. We considered these patterns
to be behavioral states of the specimens, and used the
trained SOM to recognize new data sets of movement.
Table 1(a) shows a summary of the overall recognized
patterns before the treatments (10,000 segments). The
dominant patterns were P6, P1, and P4; especially P6
holds 28.0% of the total number of segments. The con-
tinuous left (P4) and continuous right (P1) movements
were also dominant before the treatments while P3, stop,
occurred with minimal frequency. After the treatments
(10,000 segments), the proportion of the movement pat-
terns changed substantially (Table 1(b)). The segments
keeping the same direction in movement sequence (P1,
P4, and P6) decreased while the zig-zag type movements
(P2 and P5) increased after the chemical treatments. It
was notable that the proportion of forward movement
(P6) decreased distinctively from 28.0% to 12.2% after
the treatments. The stop pattern (P3), however, sub-
stantially increased. This change in pattern frequency
indicated that the chemical eects were accordingly pro-
jected onto the spatial-time domain of the movement
data. The paired-sample t test accordingly showed sta-
tistical signicance in each pattern at P <0.001 (t values:
22.76 (P1), -29.45 (P2), -36.38 (P3), 26.42 (P4), -31.58
(P5), and 39.91 (P6)) before and after the treatments.
2. Transition and Emission Probabilities
After normalization of the observed sequence between
dierent movement patterns in two time sequences, the
transition probabilities were obtained based on the ex-
perimental data before (Table 1(c)) and after (Table
1(d)) the treatments. We considered this to be the exper-
imental transition probability matrix (TPM). Transition
probabilities of the same patterns tended to remain in
the higher range before and after the treatments. Some
dominant transitions between dierent patterns, how-
ever, were also observed; for instance, P1 P6 and P4
P6 before the treatments, and P2 P3 and P5 P3
after the treatments.
Symmetry and asymmetry of transition probabilities
also existed in the experimental TPM. Before the treat-
ments (Table 1(c)), the transition probability of P1
P2, for instance, was 0.131, and that of the reverse
transition P2 P1 was in a similar range, with 0.143.
The symmetric transitions were also observed in other
pairs of transitions, such as P1 P4 and P2 P5.
Asymmetric transitions, however, additionally occurred
in the experimental TPM. For example, the transition
probability of P4 P6 was 0.374 while that of the re-
verse way was substantially lower with 0.166. It is no-
table that the transition probabilities related to P6 were
mostly asymmetric (Table 1(c)). Symmetry in transition
probabilities appears to be at higher frequencies after the
treatments (Table 1(d)).
Substantial changes in transition probabilities oc-
curred after the treatments (Table 1(d)). The propor-
tion of continuous movement patterns (e.g., P1) de-
-1008- Journal of the Korean Physical Society, Vol. 56, No. 3, March 2010
Table 2. Experimental emission probability matrix of the movement patterns of D. magna based on recognition by the SOM:
(a) before treatment; (b) after treatment.
(a)
P
P
P
P
P
P
P
State
Event
V1 V2 V3 V4 V5 V6 V7 V8 V9
P1 0.000 0.017 0.153 0.422 0.244 0.112 0.052 0.000 0.000
P2 0.012 0.157 0.497 0.215 0.091 0.026 0.003 0.000 0.000
P3 0.637 0.266 0.078 0.016 0.002 0.000 0.000 0.000 0.000
P4 0.000 0.008 0.117 0.335 0.322 0.200 0.018 0.000 0.000
P5 0.000 0.157 0.364 0.228 0.146 0.059 0.031 0.008 0.008
P6 0.000 0.000 0.000 0.000 0.016 0.155 0.356 0.185 0.288
(b)
P
P
P
P
P
P
P
State
Event
V1 V2 V3 V4 V5 V6 V7 V8 V9
P1 0.000 0.012 0.258 0.487 0.201 0.036 0.004 0.000 0.000
P2 0.010 0.429 0.469 0.080 0.010 0.001 0.000 0.000 0.000
P3 0.787 0.193 0.013 0.005 0.001 0.001 0.000 0.000 0.000
P4 0.001 0.022 0.310 0.403 0.210 0.047 0.006 0.001 0.000
P5 0.002 0.332 0.457 0.157 0.044 0.008 0.000 0.001 0.000
P6 0.000 0.000 0.000 0.000 0.009 0.322 0.304 0.104 0.261
Table 3. Estimated transition probability matrix of the
movement patterns of D. magna based on recognition by the
SOM: (a) before treatment; (b) after treatment.
(a)
H
H
H
H
H
t 1
t
P1 P2 P3 P4 P5 P6
P1 0.290 0.112 0.065 0.065 0.071 0.396
P2 0.131 0.172 0.081 0.263 0.081 0.273
P3 0.135 0.081 0.351 0.176 0.054 0.203
P4 0.057 0.073 0.047 0.295 0.130 0.399
P5 0.168 0.110 0.077 0.200 0.439 0.006
P6 0.194 0.084 0.026 0.178 0.123 0.395
(b)
H
H
H
H
H
t 1
t
P1 P2 P3 P4 P5 P6
P1 0.142 0.194 0.200 0.097 0.161 0.206
P2 0.124 0.159 0.310 0.159 0.204 0.044
P3 0.207 0.004 0.378 0.147 0.100 0.163
P4 0.076 0.096 0.185 0.268 0.115 0.261
P5 0.140 0.134 0.273 0.058 0.360 0.035
P6 0.205 0.179 0.086 0.232 0.126 0.172
creased while the proportion of zigzag patterns (e.g.,
P2) increased under chemical stress. The symmetry in
transition probability of P1 P2 was broken af-
ter the treatments: P1 P2 was 0.179 while the re-
verse transition, P2 P1, was 0.070. A substantial
number of asymmetric transition probabilities before the
treatments were reversed after the treatments: for in-
stance, the transition probabilities between P5 P6
were (0.248, 0.099) before the treatments (Table 1(c))
while the probabilities changed to (0.025, 0.132) after
the treatments (Table 1(d)). Some transition probabil-
ities, however, remained in the similar range after the
treatments. For example, the transition probability be-
tween P1 P4 kept a symmetric transition similar
to the case before the treatments.
In each segment of movement, the event with dier-
ent categories of speed was also recorded to produce the
EPM before the treatments (Table 2(a)). Among six
movement patterns, the high speed range, V5-V9, mainly
occurred in P6 while the low speed range, V1-V5, was
observed mostly in P3. This was understandable because
P6 indicates a continuous forward movement while P3
represent a stop pattern. Another continuous directional
movement pattern, P1, covered a relatively higher range
of speed, V2-V7. The reverse was observed with P4.
The zig-zag movements, P2 and P5, also showed a sim-
ilar range of speed. The six patterns could be ordered
by speed as P6 > P1 = P4 > P2 = P5 > P3 (Tukey
test, F = 5393, DF = 5, 9994), P < 0.001) before the
Hidden Markov Model and Self-organizing Map Applied to Exploration of Movement Behaviors Yuedan Liu et al. -1009-
Table 4. Estimated emission probability matrix of the movement patterns of D. magna based on the hidden Markov model:
(a) before treatment; (b) after treatment.
(a)
P
P
P
P
P
P
P
State
Event
V1 V2 V3 V4 V5 V6 V7 V8 V9
P1 0.000 0.024 0.166 0.414 0.183 0.136 0.077 0.000 0.000
P2 0.010 0.172 0.525 0.192 0.071 0.020 0.010 0.000 0.000
P3 0.676 0.270 0.027 0.027 0.000 0.000 0.000 0.000 0.000
P4 0.000 0.000 0.078 0.342 0.326 0.238 0.016 0.000 0.000
P5 0.000 0.168 0.342 0.252 0.116 0.071 0.032 0.006 0.013
P6 0.000 0.000 0.000 0.000 0.010 0.113 0.400 0.161 0.316
(b)
P
P
P
P
P
P
P
State
Event
V1 V2 V3 V4 V5 V6 V7 V8 V9
P1 0.000 0.013 0.297 0.503 0.155 0.032 0.000 0.000 0.000
P2 0.026 0.474 0.439 0.061 0.000 0.000 0.000 0.000 0.000
P3 0.797 0.195 0.004 0.004 0.000 0.000 0.000 0.000 0.000
P4 0.000 0.025 0.287 0.344 0.293 0.051 0.000 0.000 0.000
P5 0.000 0.378 0.459 0.134 0.029 0.000 0.000 0.001 0.000
P6 0.000 0.000 0.000 0.000 0.000 0.391 0.318 0.079 0.212
treatments.
After the treatments, the categories in high speed gen-
erally decreased (Table 2(b)) compared with the values
before the treatments. However, some transition proba-
bilities were conservative: probabilities of P1 and P4 and
of P2 and P5 remained in similar ranges. The order of
movement patterns was also the same as the case before
the treatments: P6 > P1 = P4 > P2 = P5 > P3 (Tukey
test, F = 4348, DF = 5, 9994), P < 0.001).
3. Evaluation of TPM and EPM
The estimated TPM and EPM were accordingly ob-
tained by using the HMM. Initially, the experimental
TPM and EPM were provided as inputs to the HMM.
Based on the Baum-Welch algorithm [14], the estimated
TPMs (Table 3) and EPMs (Table 4) were accordingly
obtained before and after the treatments. The values for
the estimated TPMs and EPMs were overall in a range
similar to the experimental TPMs (Chi-square goodness
of t, DF = 35, P > 0.999 with
2
= 0.166 before the
treatments and
2
= 0.054 after the treatments) and
EPMs (Chi-square goodness of t, DF = 53, P > 0.999
with
2
= 0.180 before the treatments, and with
2
=
0.196 after the treatments).
Subsequently we evaluated the accuracy of the esti-
mated TPMs and EPMs. By using the generated event
sequence, we produced the generated states along with
the time progress. Concurrently, the sequence state
was produced from the Viterbi algorithm in the HMM.
When the generated states and the states estimated from
Viterbi algorithm [14] were matched (HMM toolbox,
MATLAB 7.8, The MathWorks, R2009), the accuracy
was 82.5% before the treatments and 79.2% after the
treatments. The results indicate that the sequence of
movement patterns before and after the treatments could
be explained by using Markov processes.
IV. DISCUSSION AND CONCLUSIONS
Because of the complexity residing in a huge amount
of data in two-dimensional movement, nding an e-
cient method to determine the pattern changes in move-
ment behavior with conventional methods was not easy.
Notably, the TPM in stochastic processes provides use-
ful information on transition probabilities regarding this
type of complex behavioral data. Symmetric and asym-
metric transition probabilities were clearly presented in
the TPMs, and changes in the transition probabilities
were eciently illustrated (Table 1(c) - 1(d)). Among
the symmetric transition probabilities, some probabili-
ties were conserved while other probabilities changed. Il-
lustrations of changes in the transition probabilities will
be helpful for further characterizing the movement mech-
anisms of indicator species.
Asymmetric transition probabilities also showed char-
-1010- Journal of the Korean Physical Society, Vol. 56, No. 3, March 2010
acteristic results. Some selected asymmetric transition
probabilities were reversed after the treatments. For in-
stance, the transition probabilities for P5 P6 and P6
P5 were 0.248 and 0.099, respectively, before the
treatments (Table 1(c)). However, the transition prob-
abilities were reversed, 0.025 and 0.132, respectively, af-
ter the treatments (Table 1(d)). The reverse change
in asymmetry after the treatments could also be use-
ful for checking the physiological behavior causality rela-
tionships. The asymmetric transitions that were not re-
versed after the treatments mostly changed to symmetric
transitions up to eight pairs, including P1 P3, P1
P5, P3 P4, P3 P4, etc (Table 1(c) -
1(d)). This increase in symmetry additionally indicated
that the control ability of the indicator organisms was
somewhat loosened after the treatments.
Since the individuals are reared in a limited area in the
arena, the individuals sometimes stayed near the bound-
ary area. Behaviors near the boundary area could be
dierent from the movements of the individuals in the
middle of the arena. In this study, however, a relatively
low proportion of the observation time was spent near
the boundary area. Considering that the boundary area
could be determined as 2.5 mm inside the boundary [17],
the specimens usually stayed less than 5% of the total
observation time, on average, in the boundary area. In
the future, however, a denition of behavioral states near
boundary area should be checked further, because behav-
ioral states in the boundary area may be characterized
dierently [18].
In conclusion, Markov processes were identied in
short-time movement segments of indicator species and
will be useful for detecting the eects of toxic chemi-
cals. Behavioral states obtained by using the SOM were
accordingly classied in a heuristic manner, and subse-
quently, the movement patterns identied by using the
SOM were veried with the HMM to elucidate Markov
processes under both normal and stressful conditions.
Data mining and stochastic processes in combination
could be eciently used to illustrate the behavioral pro-
cesses and to monitor toxic chemicals in the environment.
ACKNOWLEDGMENTS
This paper was supported by Korean Council for Uni-
versity Education, grant funded by Korea Government
(MOEHRD) for 2008 Domestic Faculty Exchange.
REFERENCES
[1] H. Dutta, J. Marcelino and C. Richmonds, Arch. Int.
Physiol. Bio. 100, 331 (1992).
[2] T.-S. Chon, N.-I. Chung, I.-S. Kwak, J.-S. Kim, S.-C.
Koh, S.-K. Lee, J.-B. Leem and E. Y. Cha, Environ.
Monit. Assess. 101, 1 (2005).
[3] Y.-S. Park, N.-I. Chung, K.-H. Choi, E. Y. Cha, S.-K.
Lee and T.-S. Chon, Aquat. Toxicol. 71, 215 (2005).
[4] C. W. Ji, S. H. Lee, K-H. Choi, I.-S. Kwak, S. G. Lee, E.
Y. Cha, S.-K. Lee and T.-S. Chon, Int. J. Ecodyn. 12, 1
(2007).
[5] I.-S. Kwak, T.-S. Chon, H.-M. Kang, N.-I. Chung, J.-
S. Kim, S. C. Koh, S.-K. Lee and Y.-S. Kim, Environ.
Pollut. 120, 671 (2002).
[6] L. R. Rabiner and B.-H. Juang, Encyclopedia of Lan-
guage & Linguistics (Elsevier: Oxford, 2006).
[7] A. Franke, T. Caelli and R. Hudson, Ecol. Modell. 173,
259 (2004).
[8] T. Kohonen, Self-Organizing Maps (Springer, Berlin,
2001).
[9] B. P. Elendt and W. R. Bias, Water Res. 24, 1157 (1990).
[10] J. T. Hamm and D. E. Hinton, Aquat. Toxicol. 48, 403
(2000).
[11] D. E. Burkepile, M. T. Moore and M. M. Holland, Bull.
Environ. Contam. Toxicol. 64, 114 (2000).
[12] N. D. Dees, S. Bahar, R. Garcia and F. Moss, J. Theor.
Biol. 252, 69 (2008).
[13] J. H. Ward, J. Am. Stat. Assoc. 58, 236 (1963).
[14] L. R. Rabiner, Proc. IEEE 77, 257 (1989).
[15] P. Jackson, HMM Tutorial, from the website
http://www.ee.surrey.ac.uk/Teaching/Courses/eem.ssr/
(September 15, 2009).
[16] G. Wolf, P. Scheunders and M. Selens, Comp. Biochem.
Physiol. A 120, 99 (1998).
[17] R. Jeanson, S. Blanco, R. Fournier, J.-L. Deneubourg,
V. Fourcassie and G. Theraulaz, J. Theor. Biol. 225,
443 (2003).
[18] R. M. A. Hayman, J. G. Donnett and K. J. Jeery, J.
Neurosci. Methods 167, 184 (2008).