Computerized Medical Imaging and Graphics

PERGAMON
Computerized Medical Imaging and Graphics 24 (2000) 283296 www.elsevier.com/locate/compmedimag

A self-learning segmentation frameworkthe Taguchi approach

Ding-Horng Chen, Yung-Nien Sun*
Department of Computer Science and Information Engineering, National Cheng-Kung University, No 1 University Road, Tainan, Taiwan, ROC Received 4 November 1999; accepted 26 January 2000

Abstract The detection of object boundary is an interesting and challenging task in computer vision and medical image processing. The active contour model (snake model) has attracted much attention for object boundary detection in the past decade. However, due to the lack of understanding on the effect of different energy terms to the behavior of related objective functions for an image, the assignment of weights for different energy terms in this model is usually fullled empirically. Few discussions have been brought out specically for assigning these weights automatically. In this paper, a novel self-learning segmentation framework, based on the snake model is proposed and applied to the detection of cardiac boundaries from ultrasonic images. The framework consists of a learning section and a detection section, and provides a training mechanism to obtain the weights from a desired object contour given manually. This mechanism rst employs Taguchi's method to determine the weight ratios among distinct energy terms, followed by a weight renement step with a genetic algorithm. The rened weights can be treated as the a priori knowledge embedded in the manually dened contour and be used for subsequent contour detection. Experiments with both synthetic and real echocardiac images were conducted with satisfactory outcomes. Results also show that the present method can be used to analyze successive images of the same object with only one training contour. Finally, the validity of the weight determining process was veried by the analysis of variance method (ANOVA). q 2000 Elsevier Science Ltd. All rights reserved.
Keywords: Snake model; Taguchi's method; Genetic algorithm; ANOVA

1. Introduction Object segmentation is a critical task in the early stage of vision and medical image processing applications. In most cases, such a task serves as a primary pre-processing step to distinguish the foreground object from its background. Object segmentation is generally classied into two main categories, i.e. region-based methods [1,2], and boundarybased methods [3,4]. In region-based methods, they testify the homogeneity of the target object and generate the object shape, using region growing and merging techniques. The major drawback of this type of method is that irregular boundaries and cavities inside an object are often generated. On the contrary, boundary-based methods, which only deal with the information along the object boundary, can give much better boundary smoothness without cavitation problems. The usual tools being employed in boundary-based methods include, local ltering approaches such as Canny edge detector [5], Sobel edge operator, or energy minimization mechanisms like the
* Corresponding author. Tel.: 1 886-6-2757575, ext. 62520; fax: 1 886-6-2747076. E-mail address: ynsun@mail.ncku.edu.tw (Y.-N. Sun).

active contour model (i.e. snake model) [6] and balloon models [7]. In this paper, we focus on the active contour model, which has been the most frequently used approach on contour segmentation, due to its exibility and practicality [3,816]. This model formulates the boundary detection problem as an energy minimization process for an energy function, and simulates the contour behavior by the manipulation of different energy terms. The conventional snake model makes an assumption that the denition of the energy terms can describe the dynamic behavior of the deformed contour. Therefore, the minimization of the energy function can achieve an optimal contour, as dened by the preferred image properties. In general, an energy function consists of three major feature forces, namely, the internal force, the external force (also known as image force), and the constraint force. The internal force limits the contour to be a smooth curve. The image force is a function that corresponds to certain characteristic functions of a contour point. The constraint force can be included to restrict the contour to a specic shape expectation. One of the major problems in the snake model is that it is hard to determine the appropriate ratio (i.e. weights) between different forces in the total energy function of the

0895-6111/00/$ - see front matter q 2000 Elsevier Science Ltd. All rights reserved. PII: S 0895-611 1(00)00023-9

284

D.-H. Chen, Y.-N. Sun / Computerized Medical Imaging and Graphics 24 (2000) 283296

Fig. 1. The system diagram of the proposed framework.

model, due to the lack of understanding on the effects of each force on the energy function. Therefore, few convincing or systematic ways to acquire reasonable weights that reect the contribution of the corresponding energy terms are available, and such weights are generally assigned empirically or an arbitrary global-weight (i.e. a same weight for all feature forces) is used. However, in cases where there are non-homogenous image properties around the vicinity of the deformed contour, empirical or global-weight assignment could render the previously mentioned basic assumption of the snake model invalid, and the desired contour usually cannot be achieved. Although satisfactory weights can be obtained eventually, through tedious and timeconsuming manual trial-and-error processes, there is no standard procedure to prove that the weights thus obtained are optimum ones. In this paper, we propose a novel algorithm, which can be used to automatically detect the contour of a series of dynamic images from a single object, or the contour for a set of similar images. The present proposed method contains two sections, i.e. the training section and the detection section, as illustrated in Fig. 1. The training section aims to extract the embedded internal knowledge (as represented by the weights of feature forces) of an image contour, through an image outline provided by an expert, using Taguchi's method and the Genetic Algorithms (GA). The detection section then utilizes these weights to detect the contour of desired objects. The present method rst trains the computer to learn from knowledge embedded in a training contour of an image provided by an expert who is familiar with such images (for instance, a cardiac surgeon for a ultrasonic echocardiac image). It rst adopts the expert's sketch as the initial contour, then use this contour as a template in the snake model, and attempts to analyze the embedded information of the sketch, by assigning proper weights for different

feature forces. Subsequently, image features forces (including the internal force, external force, and constraint force) are calculated, and a set of weights is assigned to each force, to reect its signicance. Taguchi's method is used in the present method to reduce the variable complexity in the experiment design, and act as a standard procedure to determine the initial weights, which are then rened by the GA. The rened weights can be regarded as the a priori expertise, viz. the high level knowledge, acquired from the human sketched contour. In the detection section, the greedy method was adopted to obtain the contour, using the a priori knowledge. By iterative minimization of the weighted energy function along different assigned searching lines, the optimally t contour positions can be located. In the following parts of this paper, Section 2 introduces the theoretical background and the detailed algorithm, including a demonstration on the extraction of desired contours for ultrasonic cardiac images. The experimental results and their discussions are presented in Section 3, with the conclusions being given in Section 4. 2. The self-learning snake model Details of the present method are described in this section, which include the basic snake model, the learning section and detection section. The active contour model (or snake model) which can locate the position of a smooth contour in a 2D image was rst introduced by Kass et al. [6]. However, its simplicity and exibility in implementation found wide applications in both 2D and 3D imagery [17,18]. In this paper, we demonstrate with 2D illustration, a new self-learning mechanism for the snake model. This new development teaches the snake model to carry out border detection tasks according to the given training.

D.-H. Chen, Y.-N. Sun / Computerized Medical Imaging and Graphics 24 (2000) 283296

285

is given either by a human sketch or by pre-processing. A search is made on a set of searching lines (or curves) that extend from the initial contour, to locate the optimal contour that has the minimum energy according to the given energy denition. These searching lines (curves) are usually orthogonal to the initial contour and locate uniformly along the arc length. 2.2. Training section Let C be the training contour given by an expert on an image. In the present proposed framework, the training contour provides the bases to extract human knowledge. The contour is generally sampled with some control points, on which the search of the optimal contour position is performed. Let Pi, 1 # i # n be the ith control point of the training contour. For each control point, m features such as intensity, gradient magnitude, or curvature, are dened to represent the local image properties. Thus, the m feature values (i.e. u ij, 1 # i # n; 1 # j # m for each control point Pi in the training contour C could be written in a matrix form: I H I H Q1 u11 u12 u1m g f g f fQ g fu u2m g g f 2 g f 21 u22 g f g f 3 Qf . gf . . g f . g f . . g f . g f . ] . g e d e d Qn un1 un2 unm The feature value u ij represents the local image properties of the control point Pi on the training contour C. Fig. 2 shows the illustrative sketch of control points on the training contour. 2.2.1. The energy function For a training contour C, its energy function can be dened as: E AQ 2 w 1 iv si 1 w2 iv si 1 Pvs ds
H 2 HH n m i1 j1

Fig. 2. The illustrative sketch of the training contour C.

2.1. The snake model The snake model is dened with a function E to represent the energy of a contour, as dened by: E int vs 1 Eext vs 1 Econ vs ds 1 Esnake vs
V

where v(s) is a point on the curve with C, Eint, Eext, and Econ representing the previously mentioned internal, external and constraint forces, respectively, and V is the supporting domain. In the energy minimization process of the snake model, the internal, external and constraint forces play different roles in the contour detection process. The internal force provides the a priori knowledge about the contour behavior such as smoothness and continuity. The external (image) force represents the local features of the image function such as edges, lines, regions or textures. The constraint force restricts the deformation of the contour to specic geometrical properties, e.g. degree of curvature, of the image plane. To clarify the concept in more detail, the energy function E can be rewritten as: v U 3 Ev 1
0

Evs ds

a ij uij

1
0

where H

where v H and v HH denote the rst and the second derivatives of v, respectively. The derivatives retain the curve smoothness and P is the potential function associated with the external forces, with P usually being designed according to the desired properties in contour extraction. For example, if we want the contour to approach the edge points, the edge gradient can be included as one of the potential elds. Sometimes, the constraint force is also employed to limit the evolution of the contour to certain geometric shape conditions. The contour deformation is achieved by using an energyminimizing approach and begins with an initial contour that

g f g f fA g fa a2m g g f 2 g f 21 a22 g f g f Af . gf . . g f . g f . . g f . g f . ] . g e d e d an1 an2 anm An and m aij 1; 1 # i # n; () is the scalar product operaj1 tor. The weight a ij reects the signicance of the feature force u ij. The values of u ij such as intensity, gradient magnitude, curvature or other image properties, are normalized to the same scale. Therefore, feature values with higher weights have a more signicant inuence in the total contour energy. In practice, we usually dene the preferred

A1

a11

a12

a1m

286

D.-H. Chen, Y.-N. Sun / Computerized Medical Imaging and Graphics 24 (2000) 283296

Table 1 An instance of the orthogonal array L25(5 6) A 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 a a a a a b b b b b c c c c c d d d d d e e e e e B a b c d e a b c d e a b c d e a b c d e a b c d e C a b c d e b c d e a c d e a b d e a b c e a b c d D a b c d e c d e a b e a b c d b c d e a d e a b c E a b c d e d e a b c b c d e a e a b c d c d e a b F a b c d e e a b c d d e a b c c d e a b b c d e a

approach to select the most useful features from a given set of image features. 2.2.2. Initial estimation of weights by Taguchi's method As the feature values u ij are xed for a given training contour C, the initial population of GA is used as the starting point to approximate the exact weights, according to the proposed paradigm. Since we are aiming at obtaining local adaptive weight assignment (i.e. different weights for the same feature force at different control points), a large number of experimental factor levels will result, which will pose a demanding challenge for computation. For instance, if there are n control points, each with m feature values, then m n combinations in selecting the initial population will have to be tested. Such a large number of experiments will not only involve too much computation time, but also makes the optimal selection of model parameters infeasible. Thus, a proper estimation of initial weight is necessary before GA treatment. As Taguchi's method provides a simplied, robust, and reproducible means to handle multiple factor selection problems, it is thus used for the initial estimation of the weights a ij. Taguchi's method was rst developed for quality engineering purposes [19,20]. However, the robust optimization scheme makes it possible to be applied to many other applications. During the design stage of an experimental scheme, the most important task is to control the variability of different experimental factors. The technique of laying out conditions of experiments involving multiple factors, is also known as factorial design of experiment, and was rst introduced by Fisher in the 1920s [21]. Although a full factorial experimental design can describe all possible conditions for a given set of factors, this will result in a large number of experiments. To make an experimental design easier to implement, a compact design that involves a manageable number of experiments is needed. Taguchi's method successfully resolves two difculties in compacting experimental design. First, orthogonal arrays (OAs) [22] that represent the possible experimental conditions were introduced. Secondly, a standard procedure to analyze the results performed by the adopted experimental design is provided. Taguchi's method also compares the main effects of each experimental factor and analyzes the performance of each factor to acquire the best experimental conguration. Finally, the ANVOA technique can then be used to evaluate the outcome of experimental results to obtain a better experimental conguration. The rst step of designing an experiment with known number of factors in Taguchi's method is to select a most suitable OA, which is designed to cover all the possible experiment conditions and the factor combination. In the present study, since there are six factors, represented by A, B, C, D, E and F; with ve levels (a, b, c, d and e) for each factor, the orthogonal array L25(5 6) that involves 25 trial runs is selected to perform the initial weights estimation of the segmentation framework. Table 1 shows an instance

image properties with higher feature values. The energy terms we used in this study include: 1. Intensitythe intensity of the object was higher than the background, higher intensity values will increase the corresponding feature values. 2. First-order derivativeresists the contour from stretching. 3. Second-order derivativeresists the contour from bending. 4. Curvatureis used to preserve smoothness in object shape. 5. Gradientdenotes the edginess of the boundary. High gradient implies high possibility of being boundary points. In this study, the Sobel edge operator is used to compute the gradient magnitude. 6. Template energythe term used to dene the template energy is usually problem-dependent [1014]. It denotes a prototype of the object shape present in the image and can be viewed as the constraint force in the snake model. In this study, the ratio of the radius on the control point to the overall average radius is calculated as the template energy. In fact, there are still many other alternatives to dene image properties for the desired object contour. For instance, texture, intensity contrast, or some specic ltering output may also be good image features in the energy computation. The following section presents a systematic

D.-H. Chen, Y.-N. Sun / Computerized Medical Imaging and Graphics 24 (2000) 283296

287

Fig. 3. The owchart of GA.

of the L25(5 6) OA, with each row representing a trial condition with factor levels indicated by the letters in the row. The vertical columns of OA correspond to the factors specied in the study. The factor levels a, b, c, d and e are the physical experimental values of each factor, which are being used to perform the trial runs. However in the present study, since there are no physical or pre-dened conditions that can be utilized, all feature values are normalized to the same scale, and the factor levels around 1, i.e. a 0:25; b 0:5; c 1:0; d 1:25 and e 1:5; are assigned as the given experiment conditions. The factor levels are then substituted into a ij in the energy optimization scheme according to Eq. (4). Taguchi's method is then used to calculate the main effect of uij at different factor levels a ij (i.e. factor levels a, b, c, d and e as dened above), and the combination with the highest energy is chosen. However, the outcome of Taguchi's method only provides a single value for each factor level. In the present study that involves multiple control points, such single values (equivalent to the weight of the respective feature force) will have to be assigned to all control points, i.e. global-weight assignment. Therefore, such outcomes can only be used as the initial states of the GA treatment at subsequent stages. The detailed implementation of Taguchi's method can be found in related references [19,20,22]. 2.2.3. Rened weights by GA In this stage, GA is carried out to ne-tune the globalweights obtained by Taguchi's method. GA was inspired by the natural genetics and rst proposed by Holland [23]. The most powerful feature of GA is its highly parallel and adaptive property compared with other optimization techniques. These features make GA particularly useful in handling

simultaneous optimizations of a whole population, while avoiding local minimum solutions. Generally speaking, GA employs an evolutionary strategy that incorporates a selection and search mechanism to achieve a near optimal solution of a given objective function. The objective function is being approached iteratively, with each iteration during the optimization process known as a generation. In GA treatment, populations of binary strings are rst chosen as the initial solutions with respect to the objective function. These binary strings represent the candidate solutions by encoding its numerical values to 0s and 1s. Given a pre-dened integer range, the candidate solutions are mapped linearly to the range with a xed number of binary bits, which are used to construct the binary strings that carry the solution of the desired objective function, and are regarded as genes as in genetics. To evaluate the tness of the genes, an objective function, called tness function is used to evaluate each binary string (actually representing a candidate solution), and a tness value that reects the performance of the solution is assigned to each string. Before the tness value reaches a pre-decided termination criterion, the looping is repeated. The termination criterion generally varies with the given problems. The GA contains three types of operations, namely, selection, crossover and mutation. The selection operation simulates nature's survival-of-the-ttest phenomenon, and can be carried out by setting appropriate criteria for selecting individual genes. Crossover simulates the gene interchange in genetics. Suppose the binary string length is l, an integer ranging from 1 to l 2 1 is randomly selected as a crossover point. Further, a probability value is used as the crossover rate to indicate the necessity of performing the crossover. When the randomly selected integer is greater than the crossover rate, crossover operations will take place. Given a pair of binary strings, the crossover operation exchanges the bit content beyond the crossover point, to generate a new binary string. Mutation operations can also proceed after crossover. This operation ips the bit content of the binary string from 1 to 0, and vice versa, thereby providing a mechanism to keep the solution away from the local minimum. A probability value called mutation rate is given to determine the necessity of performing mutation. Similar to crossover, mutation will proceed only if the value of a random picked integer is greater than the mutation rate. The owchart of GA is sketched as in Fig. 3. Details of GA can be found in a related reference [23]. At the beginning of the GA process, the weights acquired by Taguchi's method are used as the initial population of GA, and the tness function is dened as: 1 5 1 1 eE m m where E n i1 j1 aij uij ; j1 aij 1; 1 # i # n: GA evaluates the weights that are bound in a given interval around the initial population. After the previously

288

D.-H. Chen, Y.-N. Sun / Computerized Medical Imaging and Graphics 24 (2000) 283296

Fig. 4. The conceptual sketch for the contour detection.

Fig. 5. The synthetic image for verifying the effectiveness of the proposed method. (a) The original image. (b) The training contour with manual drawn circles. (c) The normalized feature values on the searching lines.

D.-H. Chen, Y.-N. Sun / Computerized Medical Imaging and Graphics 24 (2000) 283296

289

Fig. 6. The detection section: (a) the initial contour and the result contour (innerthe initial contour, outerthe result); (b) the energy prole of the result contour after 9 iterations; (c), (d), (e) and (f) the deformation process of the rst, second, third and fth iteration, respectively.

mentioned training process is completed, a set of weights that represent the signicance of the feature force values is obtained, i.e. I H I H a11 a12 a1m A1 g f g f fA g fa a2m g m g f 2 g f 21 a22 g f g f aij 1; 1 Af . gf . g and . g f . g f . . g j1 f . g f . ] . e d e d an1 an2 anm An #i#n (6)

It is worth noting that the acquired weight set is only a possible solution in the solution space, with respect to the tness function. The solution space spanned by the tness function is an n-dimensional hyperspace and each weight set acquired by the GA mechanism approaches to a local minimum over the hyperspace. Thus, each weight set is actually a testing point to nd the real minimum of the tness function. By the central limit theorem, a large number of nondegenerate identically distributed random variables will approach to a normal distribution. Therefore, the mean and variance will converge after being tested several times. In the present method, the average of the resulting

290

D.-H. Chen, Y.-N. Sun / Computerized Medical Imaging and Graphics 24 (2000) 283296

Fig. 7. The comparative studies. (a) The result of the weight set obtained by the proposed method. (b) The result of the equally-weighted method.

weights obtained from each test is chosen to be the representative weight for the features. For each control point in the training contour, a distinct weight is obtained and assigned to the corresponding feature. These weights are the indicators for the inuence of the features in energy optimization. 2.3. Detection section In this section, we use the trained weights obtained in the training section to detect the object contour. The weighting parameters, which represent the knowledge embedded in the sketched contour provided by an expert, indicate the relative signicance of feature forces in the energy function system. In this study, the conventional greedy method is used to minimize the value of the resulting energy function. Given a target image T, let C0 be the initial contour in T. C0 is equally sampled with n control points Pi, 1 # i # n: For a specic control point Pi, let xi be any one of the candidate points lying on the searching line Li passing through the control point and normal to C0. Let a ij, 1 # j # m be the trained weights for the i-th control point and j-th feature, where m is the number of features. Fig. 4 shows the searching line on the object contour.

The intermediate contour is located by looking for the point with the highest energy along the searching lines, as dened by Eq. (4). With such points being detected on all searching lines, a new intermediate contour can be constructed. After a number of iterations, the intermediate contour will converge to the desired contour. 3. Results and discussions Experiments were carried out with both synthetic and multi-plane transesophageal echocardiac images to testify the present method. The ANOVA analysis is also used to evaluate the successfulness and validity of these experimental results. Further, application of the present method to detect the boundaries of a series of ultrasonic echocardiac images, without repetitive training, was also demonstrated, to show the suitability of the present method for treating sequential images that retain similar image properties or for treating similar images. Re-training is only necessary when a sudden change in sequential images occurs. 3.1. Experiments with synthetic image A synthetic image with one oval shape object in the

Fig. 8. The other synthetic image experiments. (a) and (b): the different initial contour settings do not affect the resulting contours.

D.-H. Chen, Y.-N. Sun / Computerized Medical Imaging and Graphics 24 (2000) 283296

291

Fig. 9. The real image experiment. (a) The original image. (b) The training contour drawn by the expert. (c) The resulting contour detected by the proposed method.

center surrounded by a varying background is created and tested as shown in Fig. 5. The object and its surrounding background both have gradually changing intensity, but the directions of their intensity change are opposite. The image is blurred with a smoothing lter and Gaussian noise added. Therefore, the intensity changes between the object and its background are not homogenous along the boundary. Under such circumstances, it is difcult to dene energy terms in the traditional snake model. Further, manual assignment of weights with respect to different energy terms is complicated and tricky. Thus, a simple weight assignment will be hardly suitable to all changes in local properties along the boundary. The training contour in Fig. 5(a) is sketched manually and sampled with 20 equally spaced control points as shown in Fig. 5(b). The features being considered for the image

include intensity value, gradient magnitude, rst derivative value, second derivative value, curvature, and template energy. All these feature values are normalized to the same scale, and Fig. 5(c) shows plots of the normalized irregular and complicate feature prole along the searching lines. The goal of the present method is to nd suitable weights among different energy terms and blend these feature proles into an integrated one, in order to fulll the task of contour detection in complex environments. The OA dened in Table 1 is used to compute the initial weights by Taguchi's method. Once the initial weights are obtained, GA is applied to rene these weights of the given energy functions. There are two conditions to end the training section, one is to simply stop GA iteration after a xed number of generations; the other is to measure the difference of the tness values between two successive generations,

292

D.-H. Chen, Y.-N. Sun / Computerized Medical Imaging and Graphics 24 (2000) 283296

Fig. 10. The real image experiment (part two). (a) The third original image of the cardiac sequence. (b) Without re-training, the same initial contour still produces a reliable resulting contour using the trained weight acquired by Fig. 9.

and stop the iteration when their difference is smaller than a given threshold value. In the synthetic image experiment, the weight training operation was stopped after 100 iterations. In the synthetic image experiment, after training with the expert's sketched contour on an image, the same image was used to verify the detection performance of the present method, using the greedy method to detect the target contour. Fig. 6(a) shows both the irregular initial contour and the nal oval shape contour. Fig. 6(b) shows the energy proles after 9 iterations. It is obvious that the resulting energy prole becomes more regular by adopting the weights acquired in the training section, thus showing that with the present method, contour detection can be easier and
Table 2 ANOVA results on validity of differences observed. Yes: F p . Fa ; No: F p # Fa Point 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Fp 3.971884098 2.266573918 2.611998377 4.496893489 4.729724441 3.145555259 1.22628374 3.830804254 2.355824833 3.996787785 3.793322368 3.703114097 2.761463569 4.918862555 3.41773355 3.290474207 3.445763895 3.009224704 3.682369806 2.165461 F0.05 2.244703978 2.244703978 2.244703978 2.244703978 2.244703978 2.244703978 2.244703978 2.244703978 2.244703978 2.244703978 2.244703978 2.244703978 2.244703978 2.244703978 2.244703978 2.244703978 2.244703978 2.244703978 2.244703978 2.244703978 F0.025 2.609809258 2.609809258 2.609809258 2.609809258 2.609809258 2.609809258 2.609809258 2.609809258 2.609809258 2.609809258 2.609809258 2.609809258 2.609809258 2.609809258 2.609809258 2.609809258 2.609809258 2.609809258 2.609809258 2.609809258

more precise. The deformation process is also demonstrated, with the rst, second, third and the fth iteration results shown in Fig. 6(c)(f), respectively. Despite the image properties of the boundary prole not being homogeneous, the present method can still successfully converge to the desired contour. To demonstrate the effectiveness of the present method, a comparative study on two sets of weights was conducted on the same image using the same initial contour, as depicted in Fig. 7. The rst set of weights was obtained by the present method, and the second set simply employed arbitrary global-weights. Results of these two experiments after 9 iterations (Fig. 7), show that the global-weight approach

a 0:05
Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No

a 0:025
Yes No Yes Yes Yes Yes No Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No

D.-H. Chen, Y.-N. Sun / Computerized Medical Imaging and Graphics 24 (2000) 283296

293

Table 3 The Taguchi experiment using the OA dened in Table 1. The asterisked rows indicate the equally-weighted case (experiment 1), the best case (experiment 6), and the worst case (experiment 13), respectively Exp # 1p 2 3 4 5 6p 7 8 9 10 11 12 13 p 14 15 16 17 18 19 20 21 22 23 24 25 Gradient 0.1667 0.0909 0.0625 0.0476 0.0385 0.1176 0.1176 0.1176 0.1176 0.1176 0.1667 0.1667 0.1667 0.1667 0.1667 0.2105 0.2105 0.2105 0.2105 0.2105 0.2500 0.2500 0.2500 0.2500 0.2500 Intensity 0.1667 0.1818 0.1875 0.1905 0.1923 0.0588 0.1176 0.1765 0.2353 0.2941 0.0556 0.1111 0.1667 0.2222 0.2778 0.0526 0.1053 0.1579 0.2105 0.2632 0.0500 0.1000 0.1500 0.2000 0.2500 Curvature 0.1667 0.1818 0.1875 0.1905 0.1923 0.1176 0.1765 0.2353 0.2941 0.0588 0.1667 0.2222 0.2778 0.0556 0.1111 0.2105 0.2632 0.0526 0.1053 0.1579 0.2500 0.0500 0.1000 0.1500 0.2000 First 0.1667 0.1818 0.1875 0.1905 0.1923 0.1765 0.2353 0.2941 0.0588 0.1176 0.2778 0.0556 0.1111 0.1667 0.2222 0.1053 0.1579 0.2105 0.2632 0.0526 0.2000 0.2500 0.0500 0.1000 0.1500 Second 0.1667 0.1818 0.1875 0.1905 0.1923 0.2353 0.2941 0.0588 0.1176 0.1765 0.1111 0.1667 0.2222 0.2778 0.0556 0.2632 0.0526 0.1053 0.1579 0.2105 0.1500 0.2000 0.2500 0.0500 0.1000 Template 0.1667 0.1818 0.1875 0.1905 0.1923 0.2941 0.0588 0.1176 0.1765 0.2353 0.2222 0.2778 0.0556 0.1111 0.1667 0.1579 0.2105 0.2632 0.0526 0.1053 0.1000 0.1500 0.2000 0.2500 0.0500 Fitness 7.991325 8.356503 8.493444 8.565176 8.609318 10.19836 6.902686 8.041593 7.103388 8.892057 9.676892 8.790809 5.978417 7.412301 8.443164 7.351849 8.370871 9.971259 7.314132 6.524881 7.024855 8.545224 7.74775 8.71582 6.475992

could not reach the desired contour. Further, it was found that for the global-weight approach, increasing the number of iterations still could not lead to a nal converging solution for the detected contour, due to the presence of large amount of local minima when approaching the nal contour. In contrast, the present method successfully achieves the desired contour only after nine iterations. Therefore, the results show that the assignment of global-weights will not provide a satisfactory solution. It was also shown in later experiments, that the present method is also applicable for detecting contours of real cardiac images. Fig. 8 shows another two synthetic image experiments with different initial contours using the present method. The resulting contours are not affected by different initial contours, showing the robustness of the training paradigm. 3.2. Experiment with real images A series of multi-plane transesophageal echocardiac images are used in this part of the experiment. The training process was carried out after a manually drawn standard training contour was prepared. The weights are derived rst by Taguchi's method and then rened by GA optimization. Again, the features used in this case are intensity, gradient magnitude, rst and second derivative values, curvature, and template energies. The rened weights were obtained by GA with 100 iterations. Fig. 9(a) is the

rst image in the cardiac image sequence. The non-homogeneous image properties along the cardiac image contour, makes human decision for proper weights in the convention snake model very difcult. Fig. 9(b) is the heart shape training contour drawn by an expert, with the centroid indicated by the small inner circle. To prove that the characteristics of the human sketch contour could be learned during the training section, the present method was applied to the same image, using an arbitrary initial contour. From the resulting detected contour Fig. 9(c), it can be found that the detected contour is almost identical with the training contour. This demonstrates that by proper weight assignment, the present method can enable the traditional snake model to adopt to image characteristics along the cardiac contour. Fig. 10(a) is the third image of the given echocardiac image sequence. Using the same initial contour, and trained with the sketch on the rst image in Fig. 9(b), the resulting contour of Fig. 10(a) is shown in Fig. 10(b), which exhibits a high degree of agreement with the image. Therefore, results in Fig. 10 also show that, the present segmentation framework is both selfadaptive and re-usable, even though the test image is not the original one. It was also found that the trained weights could be reused for several continuous frames in the same sequence of echocardiac motion images. Around two or three training processes were required in the boundary detection of a complete cardiac sequence of 12 images.

294

D.-H. Chen, Y.-N. Sun / Computerized Medical Imaging and Graphics 24 (2000) 283296

Fig. 11. The comparison test. (a) The best case (experiment 6 in Table 3). (b) The worst case (experiment 13 in Table 3). (c) The equally-weighted case (experiment 1 in Table 3).

3.3. Verication of the experiments As mentioned previously, the present method assumed that the weights within the snake model represent the prior knowledge of the boundary behavior and thus should be distinct according to the feature computation. In this section, both theoretical proof and visual inspections were applied to verify such assumptions. 3.3.1. The ANOVA model The results so far agree with the assumption that the weight of each feature force at different control points should be different. To prove that such an agreement is not due to noise or statistical error, the ANOVA technique was used to analyze some of the results. ANOVA is a statistical technique, which is used to test the differences in mean

values between two or more categories of independent variables. In this part of the study, a test image and a training contour sampled with 20 control points was rst chosen (Fig. 9(b)). The feature values of the control points on the boundary were then calculated and the initial weights computed by Taguchi's method. These initial weights were applied to GA for 50 iteration cycles, and the average weights thus obtained were chosen as the rened trained weights. These rened trained weights represent the signicance of different energy features. To prove the validity of the rened trained weights, two hypothesis H0 and H1 were put forward as follows: H0 : m11 m12 m1m mnm ; H1 : not all mij ; 1 # i # n; 1 # j # m are equal;

D.-H. Chen, Y.-N. Sun / Computerized Medical Imaging and Graphics 24 (2000) 283296

295

where uij is the rened weight obtained by GA. The appropriate decision rule in the single-factor ANOVA model to control the condence level (a ) is as follows: If F p # F1 2 a; conclude H0; If F p . F1 2 a; conclude H1, where F p is calculated in the ANOVA model. In testing H0 versus H1, there are two possible types of errors: H0 can be falsely classied as H1 or H1 can be falsely classied as H0. The rst error type is called Type I error or a false alarm and the second error type is called Type II error or a miss. The ANVOA analysis is performed with 50 weight sets. To avoid the Type I error, we applied the F statistics to assert our assumption, with a 0:05: Table 2 shows the ANOVA table with 20 control points. We can nd that only 2 control points disagree with the assumption H1 when a 0:05: This result implies the assumption being made, was valid and reasonable. 3.3.2. Visual inspection In this section, we like to show the visual difference between the weight set obtained by the proposed method and by setting global-weights. We perform the verication test with the initial weights acquired by Taguchi's method. Table 3 shows the experiment performance using Taguchi's OA dened in Table 1. The best tness occurs in experiment 6 and the worst case occurs in experiment 13. The global-weight experiment was conducted in experiment 1 according to the OA L25(5 6). Given the same initial contour, the outcome of the best and worst tness experiments, together with the global-weight experiment were compared, to assert the present assumption. Fig. 11(a) shows the result by using the best weight set obtained by Taguchi's method. The detected contour is almost the same as the desired one provided by experts. On the contrary, the results of the worst case and the global-weight experiment shown in Fig. 11(b) and (c), respectively, show that neither experiment gave satisfactory results. These strongly support the assumption in the present proposed method, i.e. the a priori knowledge about the shape can be expressed by the distinct weight assignment. Further, the present proposed method can also improve the segmentation result more efciently and robustly. 4. Conclusions A novel self-learning segmentation framework based on the snake model is proposed, and details of the related algorithm described. The present method dealt with the most important and difcult problem of the conventional snake model, namely the weight assignment of different feature forces. Experiments using both synthetic and real echocardiac images were made to evaluate the performance of this proposed method. It was shown that the present method could extract the embedded knowledge of a contour

provided by an expert, and successfully assign the weights for different energy terms within the snake model, as evidenced by the correct determination of boundary contours for images with inhomogeneous background intensity change. Further, it was also demonstrated that after training with one contour, the present method could be applied to determine the boundaries of successive images of the same object, thus showing its learning capability to adapt for local image properties by adjusting the weights of feature forces. The self-learning mechanism signicantly improved contour detection in comparison with the traditional snake model and had obviously enabled the snake model to learn to trace an object contour as humans do. All these results, together with the verications by the ANOVA method shows that, the basic assumptions for the present method are valid and provide a reliable and robust alternative for weights assignment in a snake model for image boundary determination. It is also envisaged that the present method can be extended to other applications, such as deformable template matching and object recognition. Acknowledgements This work was supported in part by grant NSC 88-2213E006-035 from the National Science Council of Taiwan, ROC. References
[1] Beveridge JR. Segmenting images using localizing histograms and region merging. Int J Comput Vision 1989:2. [2] Adams R, Bischof L. Seeded region growing. IEEE Trans Pattern Anal Machine Intell 1994;16(6):6417. [3] Chakraborty A, Staib LH, Duncan JS. Deformable boundary nding in medical images by integrating gradient and region information. IEEE Trans Medical Imaging 1996;15(6):85970. [4] Geman D, Geman S. Stochastic relaxation, Gibbs distribution and Bayesian restoration of images. IEEE Trans Pattern Anal Machine Intell 1984(6):72141. [5] Canny JF. A computational approach to edge detection. IEEE Trans Pattern Anal Machine Intell 1986;8(6):67998. [6] Kass M, Witkin A, Terzopoulus D. Snake: active contour models. Int J Comput Vision 1988(1):31231. [7] Cohen LD. On active contour models and balloons. CVGIP: Image Understanding 1991;53(2):2118. [8] Cootes TF, Taylor CJ, Cooper DH, Graham J. Active shape models their training and application. Comput Vision, Graphics Image Processing: Image Understanding 1995:3859. [9] Chiou GI, Hwang J. A neural network-based stochastic active contour model (NNS-SNAKE) for contour nding of distinct features. IEEE Trans Image Processing 1995;4(10):140716. [10] Jain AK, Zhong Y, Lakshmanan S. Object matching using deformable templates. IEEE Trans Pattern Anal Machine Intell 1996;18(3):267 78. [11] Olstad B, Torp AH. Encoding of a priori information in active contour models. IEEE Trans Pattern Anal Machine Intell 1996; 18(9):86372. [12] Christensen GE, Rabbitt RD, Miller MI. Deformable templates using large deformation kinematics. IEEE Trans Image Processing 1996;5(10):143547.

296

D.-H. Chen, Y.-N. Sun / Computerized Medical Imaging and Graphics 24 (2000) 283296 Ding-Horng Chen received his BS degree in Mathematics in 1989 and his MS degree in Information Engineering in 1993, all from National Cheng-Kung University, Taiwan. Currently he is a PhD candidate at the Department of Computer Science and Information Engineering, National Cheng-Kung University. His research interests focus on computer graphics, medical image processing, image processing and computer vision. He is a member of IEEE, SPIEthe International Society of Optical Engineering and the Chinese Association of Image Processing and Pattern Recognition.

[13] Jolly MD, Lakshmanan S, Jain AK. Vehicle segmentation and classication using deformable templates. IEEE Trans Pattern Anal Machine Intell 1996;18(3):293308. [14] Tagare HD. Deformable 2-D template matching using orthogonal curves. IEEE Trans Medical Imaging 1997;16(1):10817. [15] Storvik G. A Bayesian approach to dynamic contours through stochastic sampling and simulated annealing. IEEE Trans Pattern Anal Machine Intell 1994;16(10):97686. [16] Gunn SR, Nixon MS. A robust implementation: a dual active contour. IEEE Trans Pattern Anal Machine Intell 1997;19(1):6368. [17] Terzopoulos D, Witkin A, Kass M. Symmetry-seeking models for 3-D object reconstruction. Int J Comput Vision 1987;1(3):21121. [18] Terzopoulos D, Witkin A, Kass M. Contraints on deformable models: recovering 3-D shape and non-rigid motion. AI J 1988(36):91123. [19] Taguchi G. System of experimental design, UNIPUB. New York: Kraus International Publications, 1987. [20] Roy R. A primer on the Taguchi method. New York: Van Nostrand Reinhold, 1990. [21] Fisher RA. Design of experiments. Edinburgh: Oliver & Boyd, 1951. [22] Taguchi G, Konishi S. Orthogonal arrays and linear graphstools for quality engineering. Dearborn, MI: American Supplier Institute, Inc, 1987. [23] Holland JH. Adaptation in natural and articial systems. Ann Arbor, MI: University of Michigan Press, 1975.

Yung-Nien Sun received his BS degree from National Chiao-Tung University, Hsin-chu, Taiwan, Republic of China, in 1978 and his MS and PhD degrees from University of Pittsburgh, Pittsburgh, Pennsylvania, in 1983 and 1987, respectively. He was an Assistant Scientist with the Brookhaven National Laboratory, New York from 1987 to 1989, and he is currently Professor at the Department of Computer Science and Information Engineering, National Cheng-Kung University, where he joined in 1989 as an associate professor. He has been working on image processing and computer vision since 1982 and has published more than 100 papers, half of them in refereed journals. His current research interests are in medical image analysis, computer graphics, and virtual reality. He is a member of IEEE, Sigma-Xi, the Chinese Association of Image Processing and Pattern Recognition, and the Chinese Association of Biomedical Engineering.