Morphological Filtering - An Overview

CIRCUITS SYSTEMS SIGNAL PROCESS VOL. 11, NO.
1, 1992
AN OVERVIEW OF MORPHOLOGICAL FILTERING*

Jean S e r r a 1 a n d L u c Vincent 1'2
Abstract. This paper consists of a tutorial overview of morphological filtering, a

theory introduced in 1988 in the context of mathematical morphology. Its first section is devoted to the presentation of the lattice framework. Emphasis is put on the lattices of numerical functions in digital and continuous spaces. The basic filters, namely the openings and the closings, are then described and their various versions are listed. In the third section morphological filters are defined as increasing idempotent operators, and their laws of composition are proved. The last sections are concerned with two special classes of filters and their derivations: first, the alternating sequential filters allow us to bring into play families of operators depending on a positive scale parameter. Finally, the center and the toggle mappings modify the function under study by comparing it, at each point, with a few reference transforms.
1. Mathematical morphology for complete lattices

1.1. Introduction W e define a m o r p h o l o g i c a l filter as an o p e r a t o r ~b, acting on a c o m p l e t e lattice Y [4], which: (i) preserves the o r d e r i n g < of Y-, i.e.,
x _< Y ~ ~ ( x ) _< ~(Y), x, Y ~ s-,
(ii) is i d e m p o t e n t , i.e.,
4,(4,(x)) = ~ ( x ) , x ~ y.
* Received January 2, 1991. i Centre de Morphologie Math6matique, l~cole Nationale Sup6rieure des Mines de Paris, 35 rue Saint-Honor+, 77305 Fontainebleau Cedex, France. 2 Present address: Harvard University, Division of Applied Sciences, Pierce Hall, 29 Oxford Street, Cambridge, Massachusetts 02138, USA.
48
SERRA AND VINCENT
The first condition, which is called growth or increasingness, just means that, since we deal with lattices, we decide to focus on the transformations which preserve one of the basic lattice features (just as, in vector spaces, attention is paid to the operators which commute with addition and scalar product, e.g., convolutions). The second condition responds to the fact that an increasing operation is nonreversible and looses information. Idempotence stops such a simplifying action at its first stage. The above axiomatics, due to Serra in 1982 [24], did not arise at once, ex nihilo. They result from a long series of interactions between theory and practice. In addition, they have opened the way to ~new types of filters that could probably never have been discovered through experimentation. The schematic diagram shown in Figure 1 gives an overview of the connections between ideas and their chronology. During the late seventies, practitioners generalized in two ways the concept of a morphological opening (or closing), initially designed for sets. Firstly, they applied it to functions (Meyer, Sternberg) and to planar graphs (Lantu6joul). Secondly, they started composing closings with openings. The
Figure 1. Milestones for the morphological filtering theory: Moore: a (.~1920); Matheron: b, c [14]; d [15]; g, h, i, j [25]; Serra: e, f, j, k [25]; 1 [17]; m [27]; Sternberg: e [30]; Meyer: 1 [17]; Ronse and Heijmans: o [21]; specified filters: rank opening (Ronse [20]), segment-based filters (Meyer and Serra [17]); orientationdependent filtering (Kurdy and Jeulin [9]); filters for graphs (Vincent [322); multispectral filtering (Vitria [35]); etc.
AN OVERVIEWOF MORPHOLOGICAL FILTERING 49
first extension led to basing the theory on complete lattices, whereas the second one is at the origin of the morphological filtering axiomatics, proposed in 1982. The same year and one year later, Matheron established a series of major theoretical results: lattice of the filters, v - and /x -filters, strong filters, middle element, the four envelopes . . . . . and extended these results to increasing (but not necessarily idempotent) operators. The part of Matheron's theory which is presented in this overview corresponds to Section 3. Matheron's concept of middle element suggested to Serra in 1986 the ideas of morphologiocal center and activity lattice [25, Chapter 8]. A further step led the same author to leave increasingness and to introduce the notion of toggle operations [27]. The approach allows in particular the association of optimization criteria to morphological transformations. It is presented below in Section 5. Several other pieces of theory exist, that we do not develop here. We may quote, among others, the relationships between filtering and connectivity preservation [25, Chapter 7], an instructive approach to multispectral filtering due to Vitria r35], and the properties of sequential monotone convergence established by Heijmans and Serra [7]. The pedagogical purpose of this document imposes restricting such derivations and pruning the text of many technicalities. For the same reason, special effort has been put on figures and on practical comments. 1.2. Algebraic framework of the complete lattices Inclusion is a set-oriented notion. The scenes under study may be modeled by sets, but also by gray-tone functions, by multispectral functions, and by graphs, each of them acting either on the Euclidean space ~" or on digital ones, like 7/". All these situations share a common denominator formed by the two ideas which define the notion of a complete lattice J - [4], namely: 1. There exists a partial ordering > over J . 2. For any (finite or infinite) family (A 3 in J-, there exists: 9 a smallest majorant v A i called the "sup" (for supremum), 9 a largest minorant /x Ai called the "inf" (for infimum). In particular, J - posesses a greatest element, E, and a smallest one, ~ . In a lattice, any logical consequence of a choice of ordering remains true when we commute the symbols v and A, and < and >_. This is called the principle of duality with respect to the order. Here is a review of a few basic lattices:
1.2.1. Boolean lattices. Start from an arbitrary set E. Obviously, the set ~(E) of the subsets of E, which is ordered for the inclusion relationship, is a complete lattice for the operations u (union) and n (intersection). Moreover,
50
SERRA AND VINCENT
with each X r ~(E), there exists a unique x c ~ ~(E), called the complement of X, such that Xc~X c=~ and XuX c=E. (1)
Finally, ~(E) also satisfies the important property of general distributivity under which, for all T E ~(E) and any family (Xi) of elements of ~(E), we have
(U Xi)
~ Y = U ( x i ('~ Y),
(2)
(3)
( ~ Xi) u Y = (-'] (X i u Y).
1.2.2. Topological lattices. When E is a topological space, its open sets generate a complete lattice for the inclusion, where the sup coincides with the union and where inf(Xi) is the interior of ~ X i. This lattice is not complemented. It satisfies the general distributivity of type (2), but finite distributivity only of type (3). Indeed, in the general case of an infinite family (Xi), we have
/U xl)
but only
o
r = U
o
r)
(4)
(s)
Similar structures are derived for the closed sets and the compact sets.
1.2.3. The convex lattice. The class of the convex sets o f the Euclidean space N" generates a complete lattice where the inf coincides with intersection and where the sup is the convex hull. 1.2.4. The partition lattice. In the set of the partitions of an arbitrary set E, we can introduce the following ordering: a partition A is smaller than a partition B when each class of A is included in a class of B. This leads to a lattice which is complete, but neither complemented nor distributive. 1.2.5. Function lattices. Let E be an arbitrary space. The class Y of the "extended" real-valued functions f : E ~ is obviously ordered by the relation f < 9 if, for each x c E, f(x) < g(x) and constitutes a complete lattice. The sup and the inf are given by the relationships f = V fi f' =/~ ~ .~ .~ f ( x ) = s u p fdx), f'(x) = inf f~(x),
Vx~E, Vx ~ E. (6)
The lattice is completely distributive but not complemented. Relationship (6) implies that f(x) may equal + oo. However, if we want to restrict ourselves to bounded functions, it suffices to remark that the previous lattice is
AN OVERVIEW OF MORPHOLOGICAL FILTERING 51
~
Figure 2. Umbra U+(X) of a set X ~ ~ ~. isomorphic (by anamorphosis) to
x)
9 either the class ~ ' of the nonnegative functions f : E ~ [-0, + oo], 9 or the class ~ " of the functions f : E ~ [-0, 1].
Functions and Umbrae. Is it possible to identify the function lattice Y with the set class of the associated subgraphs, or u m b r a e ? R e m e m b e r that with every function f : E --+ ~ (and m o r e generally with every set in E ~, see Figure 2) we can associate the two sets U+(f) and U - ( f ) of E ~ defined by the relations U+(f) = {(x,z)6E ~, f(x) < z}, (7)
U - ( f ) = {(x, z) e E x ~, f(x) < z}.
(8)
Clearly, every u m b r a comprised between U+(f) and U - ( f ) generates the same function f. Moreover, the associated ordering relations are equivalent, since
f <g r
U+(f) c U+(g) ~
U - ( f ) c U-(g).
However, this does not m e a n that u m b r a e and functions are interchangeable. F o r example, consider the threshold mapping defined as follows: {f(~ [~(f)](x) = This operation is shown on Figure 3. when f ( x ) > otherwise. 1, (9)
im
Figure 3. The threshold mapping ~/,.
52
SERRA AND VINCENT
1-2/4 . . . . . . .
1-1/3 ...... 1d/2 . . . . . . .
-1
Figure 4. The family of functions (fl). In set terms, the transformation ~ consists in intersecting the umbra U +(f) by the closed half-space E a ={(x,z),x~E,z> and in taking the upper umbra of the result U+(O(f)) = U+EE1 c~ U+(f)] w E _ , . (10) 1},
If functions and upper umbrae are equivalent, then the two algorithms (9) and (10) must yield the same result. Let us apply both of them to the sup of the following family (see Figure 4):
fi(x) = 1 - 1/i fi(x) - oo
when
Ixl <- 1,
otherwise.
If the sup f of this family is understood in the sense of the function lattice, it is equal to f(x)= 1 f (x) - co when Ixl < 1, otherwise,
and according to (9), ~9f = f. But if the sup is understood in the umbrae lattice, we get the following set:
U
i
the umbra of the function f = supi f/ in this lattice. Then, from (10), we derive U+(~(f))= E _ ~ , i.e., V x e E , ~ f ( x ) = - ~ . In other words, in the Euclidean case, the function lattice and the set-oriented lattice of umbrae are not equivalent at all. Nevertheless, in the discrete case of functions f : 7/"---~Z, the two approaches coincide and we can transpose the way of reasoning from sets to functions.
Functions and Stacks of Sets. Instead of associating with function f : E ~ its set-oriented umbra, we can alternatively consider the stack of its hori-
AN OVERVIEW OF MORPHOLOGICAL FILTERING
53
zontal sections
Xz(f): Xz(f) = {x, f(x) > z}. Xz(f) decreases continuously, i.e.,
As threshold level z increases, Xz(f)= (-]

z'<z
Xz,(f ) or X z ( f ) = J,X~,(f).
Conversely, it is easy to see that every continuously decreasing family { X z } ~ of sets generates a unique function f, by the algorithm
f(x) = sup{z, x e X~}.

Here, the monotonic continuity ~ is essential (see a counterexample on p. 427 of [-24]). It is because of it that we have to find classes of transformations which preserve the S-continuity for some families of stacks of sets. These transformations will be the upper semicontinuous mappings, and these families, the upper semicontinuous functions.
1.2.6. Lattices of semicontinuous functions. A function f is upper semicontinuous when its u m b r a U+(f) is a closed set in E x ~. It is lower semicontinuous when the umbra U-(f) is an open set in E R. Moreover, every continuously decreasing stack of closed sets generates an upper semi-continuous function, and conversely. The use of semicontinuity becomes strictly necessary as soon as extrema are involved, at least in continuous cases. For example, could we extract the maxima of the following function in ~ (see Figure 5):
{~--x2 f(x)=
when O < ' x ' < l ,

when [x[>__l or x=0?
Actually, the m a x i m u m of such a function, although it is bounded, does not exist. Conversely, as soon as we refer to the " m a x i m u m " of a function over a continuous space, we implicitely introduce the requirement that it is upper semicontinuous (or lower semicontinuous when looking for minima). Therefore, in Section 3.6, we assume that the functions under study are lower semicontinuous.
-1
Figure 5. A function without a maximum (it is not upper semicontinuous).
54
SERRAAND VINCENT
In the lattice J~, of the upper semicontinuous functions, we have inffi = { f e ~ , U+(f)
sup fi = { f e "5~, U+(f) a notation which shows that the lattice ~u and that of the closed upper umbrae are isomorphic (in this case, the identification between sets and functions works). The use of upper semicontinuous functions is required not only when maxima are studied, but also when we ,want to use upper semicontinuous transformations, i.e., when we want to make sure that every ~-continuous stack of sets is transformed into a l-continuous stack of sets. In particular, this property is strictly needed in Section 1.5.4 to generate planar increasing mappings.
1.3. Erosion, dilation In the same way that linear image processing puts the emphasis on the transformations that commute with addition, morphology naturally stresses the transformations that commute with the sup or, by duality, with the inf. This results in the following definition: Definition 1.1. Let ~-- be a complete lattice. The mappings from 3- into itself which commute with the sup (resp. the inl) are called dilations 6 (resp. erosions Q: 6(V xi) = V 6(xi), e( A xi) = A e(x~), x~ ~ y , (11)
with in particular 6(~) = ~ and 6(E) = E. The following theorem (see p. 24 of [25]) characterizes these operations: Theorem 1.2. Let 3- be a complete lattice. The classes of the dilations and of the erosions on'~-- are two complete isomorphic lattices, which correspond to one another through the duality relation 6(X) < Y ~=~ X <_ e(r), X, r ~ J - . (12)
A unique erosion ~ corresponds to each dilation 6: e(X) = V {Be J-, 6(B)_< X}, and a unique dilation 6 corresponds to each erosion e: 6(X) -- A {B ~ J-, 8(B) >_ X}. (14) (13)
55
Not only are dilations and erosions themselves increasing mappings, they generate two comprehensive classes of increasing mappings. Indeed, we have the following theorem [25, p. 20]:
Theorem 1.3. Any mapping ~: J" ~ 3- such that ~,(E) = E is increasing if and
only if it can be written as

4, = B (15)
with the erosions e~ given by e~(X)={~B) (dual result for the dilation).
1.4. Increasing mappings on boolean lattices In this section and in the following we would like to compare the two lattices which model the binary and the gray-tone images. The first one is the boolean lattice ~(E), where E is ~" or Z", for example. We can look at mappings from ~(E) into itself as extensions of mappings from E into ~(E). In the following, lowercase letters such as x, y, a, b denote elements of E, or points, and capital letters denote elements of ~(E). A point x ~ E, when considered as an element of ~(E), is written as {x}. The letter 6 denotes the mapping E--+ ~(E), which generates a dilation, as well as the dilation from 9~(E) into itself. We define a structural mapping 3 on ~(E) as any mapping 6: E ~ ~(E). Then we have [25, p. 41]:
Theorem 1.4. Let E be an arbitrary set. The datum of any mapping 6: E -~ ~(E)
if X > B , otherwise
is equivalent to that of a dilation from ~(E) into itself, again symbolized by 6, and defined by the relation = U
x~X
(16)
Conversely, any dilation of ~(E) into itself determines a unique structural mapping 6: E --* ~(E). 1.4.1. The three dualities. In any boolean algebra ~(E), the duality with respect to the complementation associates with each mapping ~p the operation r = | where | designates the complement Operator, as expressed
3 Structural mappings are often referred to in the literature as structuring functions; further to a reviewer's remark, this terminology was abandoned here since it induces confusion with gray-level structuring element. The term "structural mapping" was suggested by P. Maragos.
56 by
SERRA AND VINCENT
v x e ~(e), In the case of the dilation 6, we find

6*(X) =
x
4,*(x) = D ( x c ) ] c.
= ~
xEX C
[6(x)] c.
(17)
6*, which obviously commutes with the inf, is an erosion. 6*(X) consists of the points that are not descendant from any point in the complement of X (that are not included in any 6(x) when x ~ xC), i.e.: 1. Those whose ancestors are all included in X. 2. Those that do not have ancestors (a fixed part S, which remains the same for any set X). We form another duality notion by operating on the structural mapping with the transposition 3 ~ 3, i.e., 3(x) = {y e e, x ~ 6(y)}. The transpose 3 (x) of 6(x) is made of the set of points from which x descends, hence ~ = 6. The structural mapping 3 generates the dilation 3: 3(X) = {y6 E, 6(y)r X r 25}. (18)
From the two relations (17) and (18), we derive the links between these two dualities, and the basic one, namely 6 *-*~ (see (12)): e -- (3)*, ~ = 6", e* = 3. (19)
1.4.2. Translation invariance. We now assume that E is equipped with a translation (e.g., E = 7/" or E = E"), and that the dilation 6 is translation invariant, i.e., is a t-dilation. In other words, the structural mapping 6(h) at point h is deduced from that of the origin o (denoted 6(0) = B) by translation: 6(h) = Bh = {B + h, b ~ B}. The set B is called the structuring element. We see from relation (16) that 6(X) = X @ B = B @ X = ~B,,
xEX
= {b + x, x e X ,
boB}
= [) X b.
b~B
(20)
The t-dilation 6 is classically known as the Minkowski addition between sets X and B. By duality under complementation, it gives
6*(X) = ~(X) = ~ X b = X O B
b~B
(21)
A N OVERVIEW OF MORPHOLOGICAL FILTERING
57
. -~X @B
x-~
= ~
center
Figure 6. Dilation of a set X by a bipoint/3.
and, by lattice duality,
e(X) = n Xb = X @/~ bet?
with
/~ = { - b, b e B}.
(22)
Both operations are Minkowski subtractions of X by B and/~, respectively. According to a classical result due to Matheron, any increasing t-mapping is a union of t-erosions, and also an intersection of t-dilations [15, p. 221]. More precisely, Theorem 1.5. Let ~ be a translation invariant increasin9 mapping. Then, for
any X ~ ~(E), ~(x) = U {x e ~, B ~ ~(E), o ~ ~(B)} = N {x | ~, B ~ ~(E), o ~ r 1.4.3. Examples. We now present some examples of dilations and erosions in the lattice of sets ~(R2). Figure 6 shows the dilation of a set X by a bipoint B, i.e., the set X | Similarly, Figure 7 illustrates the effect of an erosion of X by a segment S. On Figure 8 the same set is dilated and eroded by a disk D (Euclidean dilation and erosion). The dilation by a disk is then compared, on Figure 9, with that by an hexagon H of similar size. We can remark that m a n y parts on the boundary of X @ / { are parallel to the vertices of H.
~X|
v
~/cen,or
Figure 7. Erosion of a set X by a segment S.
58
SERRA AND VINCENT
xofi
X'-'-'-- (
x
D=
Figure 8. Dilation and erosion of X by a disk D =/). Lastly, Figure 10 illustrates the algorithm which is used for performing
a 9eodesic dilation of a set Y inside a set X (see Section 2.4.1).

1.5. Increasin9 mappings on function lattices The lattice ~ ( E , ~) of the functions f : E ~ ~ shares several properties with the previous one, but it differs from ~(E) in two major aspects: 1. It is not complemented. 2. When additions or subtractions are involved, they may lead to indetermination, of the type + oe - o% since the range of variation is ~. We now study ~ ( E , ~) by following the same plan as for ~(E),
1.5.1. Generation of dilations from structural mapping. Call impulse Uh,~ a function whose value is z at point h ~ E, and - oe elsewhere [6]:
Vx6E,
Uhz(X)={Z_._
' oe
w h e n x = h, otherwise.
The class ~r of the impulses is equivalent to that of the points (h, z) ~ E x N. Clearly, any function f e ~-(E, R) is the sup of upper bounded impulses smaller than itself(just as a set is the union of the points it contains): f = sup{Uh,z, h e E, z < f(h)}.
..X(t)[I
...x ~lYl
Figure 9. Comparison between the dilations of X by a disk and by an hexagon. Note that these structuring elements are symmetrical: D = / ) and H =/4.
59
Figure 10. Successive geodesic dilations of set Y inside set X. Now introduce a structural mapping on ~-(E, N) as any upper bounded mapping 8: J ( E ) ~ if(E, N). We then have [25, p. 185]: Theorem 1.6. Any structural mapping is equivalent to a dilation from ~ ( E , ~) into itself, defined by'the relation 8(f) = sup(8(Uh, z), h ~ E, z < f(h)}. (23)
Conversely, any dilation 8: Y ( E , ~ ) ~ ~ ( E , ~) induces a unique structural mapping obtained by restricting 8 to ~r 1.5.2. Dualities. The transposition duality extends immediately to functions, by replacing points by impulses. The duality with respect to the complementation is replaced by all those given by the relation
~*(f) = m - ~(m - f), (24)
as m spans the class of the real numbers. In practice, ~ often commutes with vertical shifts, i.e., O(f + m) = O(f) + m. Then all the relations (24) are equal to ~,*(f) = - O ( - f ) , and the three expressions (19) extend to functions.
1.5.3. Translation &variances. We can consider either a translation operation t~, by vector h e E, or a translation operation th,z by a vector (h, z) E E x ~. The two corresponding formulas are (th,zf)(x) = f ( x -- h) + z, (t'hf)(X) = f ( x -- h).
We focus on the t-invariant mappings, which are the most useful in practice. Saying that dilation 6 is invariant with respect to translations is equivalent to saying that the structural mapping 6 is the same everywhere, i.e., if 9--6Uo.o is the transform of the origin-impulse Uo,o, then, Vx~E, 6Uh,z(X) = g ( X - h)+ z. Then expression (23) of the dilation 6 takes the following simpler form:
(Sf)(x) = sup{g(x - h) + z, z < f(h), h ~ E},
f ~ if(E, ~).
60
SERRA AND VINCENT
Note that the operand g ( x - h)+ z cannot take the undetermined form + oe - oo since, for all h, x, and z, each of the two numbers g(x - h) and z is < + oo. Hence, we have finally
(6f)(x) = sup{g(x - h) + f(h), h ~ E},
f E ~ ( E , ~).
(25)
The two dual erosions s and ~ of 6 are given by the following formulae:
(ef)(x) = inf{f(h) - g(x + h), h ~ E}, (~f)(x) = inf{f(h) - g(x - h), h s E}.
(26)
Similarly to Theorem 1.5, any increasing mapping ~: E ~ ~ which is t-invariant may be decomposed into a sup of erosions as well as into an inf of dilations (same proof as for Theorem 1.5).
1.5.4. Planar increasing mappings. GeneraUy speaking, the representation of a function f : E --* ~ by the stack of its sections allows us to generate a function transformation on f from every set transformation ~: E --* E. It suffices to put Xz(~(f) ) = (~ ~[Xz'(f)].
z'<:z
Then we speak of planar transformation, or again of stack transformation. Obviously, the family Xz(ql(f)) is continuously decreasing with z. Hence, it generates a function. This technique has been used, for example on p. 451 of [24], to derive the thickening for functions from that for sets. However, it is for increasing mappings that the planar increasing transformations have been more extensively studied. A basic reference is pp. 426-434 of [-24]. We can also refer to Chapter 9 of [-25] or to [13]. See also [30], [16-1, [22] and [-37]. Here, in the continuous case, we have to assume that is upper semicontinuous, which implies that it preserves the +-continuity, i.e., ~[X~(f) 3 = ~ ~[X~,(f)].
z'<:z
Then the definition of the planar mapping ~ leads to
Xz[~(f)] = tP[-Xz(f)].
(27)
This relation is illustrated by Figure 11. In other words, the planarity of the mapping t) allows us to process f threshold by threshold. A series of results derive directly from the key relation (27), namely: Theorem 1.7. Let qJ be an upper semicontinuous planar increasing mapping from ~ ( E , ~) into itself. Then:
1. The class of step functions with k levels (k arbitrary) is closed under t~. 2. ~ commutes with anamorphosis.
AN
OVERVIEW
OF MORPHOLOGICAL
FILTERING
61
......
z .....................
Figure 11. An example of a planar increasing mapping.
An anamorphosis is a strictly increasing and continuous point mapping s: [a, b] c ~ -_->[a', b'] ~ ~. For example, if s = exp, then, for any planar increasing mapping Lk, we have Lkexp(f) = exp O(f), f e ~-(E, N),
i.e., "vertical" and "horizontal" dimensions are treated independently. A basic example of a planar increasing mapping consists in the dilation of an upper semicontinuous function f by a function b which is equal .to 0 on its compact support B and to - oe elsewhere. In this case, (25) becomes (Sf)(x) = sup{f(x - h), h ~ B} = ( f 9 B)(x). By duality with respect to lattice, (26) yields (ef)(x) = inf{f(x + h), h E B} = ( f O/~)(x). (29) (28)
These operations are referred to as planar dilations and erosions, or as dilations and erosions with respect to a planar structuring element B. An example of such transformations is shown in Figure 12. Comments. 1. In the example of (28), all the horizontal cross sections are processed by the same operator, namely the dilation by B. Such a vertical invariance is not compulsory, and a stack increasing operator may well depend on the level z. It just has to fulfill the condition that the O~'s associated
~ | N ~ ~ ~ NS . ~ ~I~ ~
I ~i~'~:~i~ . . . . . . . . . . . . . . . . .
(a)
(b)
Figure 12. Dilation (a) and erosion (b) of a function f by a planar structuring element B.
62
SERRAA N D VINCENT
with each level z decrease continuously, i.e., that, for every closed set X, we have O~(X) = (~ Oz,(X) 9
z'<z
As an example, start from a threshold mapping ~'zo similar to that defined in (9) and illustrated in Figure 3: [0z0(f)](x) = {f(~ when f ( x ) > z o, otherwise.
Consider now two circular openings (see Section 2.1) 71 and 72, with 7z >- 71 (72 is less severe than 71). Apply 71 to the initial function f and 72 to the "threshold version" ~z0(f), and take their sup: 7(f) = 71(f) v 72(~z0(f)). The operator 7, which is still an opening, turns out to be a sort of Dolby filter for images: it removes the small details when they are dark enough, but preserves them otherwise. 2. Among all the image transformations presented here, the flat mappings are the only ones to preserve the physical dimensionality of the images under study. Physically speaking, the "vertical" axis is rarely homogeneous with the "horizontal" ones: the latter represent the space [L 2] or [L 3] whereas the former is more like the physical intensity [I] of light, electricity, strength, etc. When we dilate such a function by a cuboctahedron, for example, we loose the physical meaning of the axis, and the z-level of the transform no longer represents an intensity [I]. On the contrary, the fiat mappings preserve this physical meaning. Note that the converse of this last proposition is false: a notion such as the maximum involves a pile of successive sections, since a connected component C, in the cross section X t is a maximum iff, for all t ' > t, X,. c~C, = ~ .
1.6. Digital implementations In this section we concentrate upon implementations of t-dilations (and t-erosions), which are the basic stones for building up more sophisticated algorithms. When the dilation is planar, it is produced for functions in the same way as for sets. We merely have to replace union by sup and intersection by inf (e.g., refer to (28)). When the dilation is not planar, we can scan the successive levels of the structuring function (i.e., the gray-level structuring element) or use Steiner decomposition. In both cases we use the following notation: [ f 0 ( 1 0 ) ] ( x ) = sup{/(x + 1) + 1, f(x)}.
Cube
Octahedron
Rhombododecahedron
Cuboctahedron
Figure 13. Basic spherical shapes in 2~3. The plane z = 0 corresponds to the median horizontal section of the cube. The structuring functions derive from these sets by taking their umbrae. The number associated with each point denotes the altitude of the corresponding structuring function (here a function whose support is reduced to an horizontal doublet)9 When needed, a bold character is used for indicating the location of the origin9 The elementary "spherical"---and centered-structuring functions are: 9 9 9 9 The The The The cube: 9 pixels, on two successive levels9 octahedron: 5 pixels, on two successive levels. rhombododecahedron: 9 pixels, on two successive levels9 cuboctahedron: 9 pixels, on two successive levels9
They are represented in Figure 13. The elementary rhombododecahedron R can be represented (as in Figure 14) by taking the spacing of the horizontal square grid to be xf2. The Steiner
010
1 2 1 9 = (1. 0)~(0 1)@
()()
-1 0 @ 0 -1
0 1 0 .
Figure 14. The elementary rhomhododecahedron R (i.e., Steiner rhomb of ~3) and its decomposition in four dilations by segments. The complex shape of the polyhedron has been decomposed into four simple structuring functions, whose implementation is very simple and extremely efficient.
64
SERRA AND VINCENT
0 010 121 010 R 1 ~ 0 1 0 2 1 1 0 =
010 12221 0 2 3 3 3 1 2 3 4 3 0 2 3 3 3 12221 0 1 R~R*
2 0 2 1 2 0 0
Figure 15. Dilation of the Steiner rhomb R by R* (R* is obtained in the same way as R, by four dilations by doublets). r h o m b k R o f size k is o b t a i n e d by t a k i n g k dilations of R:
k R = R (~ R 0 " "
v
9 R = R ek.
J
ktimes
As k increases, the difference between the Steiner rhomb and the ball becomes more apparent, but it is a simple matter to combine R with other Steiner polyhedra, such as the cuboctahedron, or simply with another Steiner rhomb, R*, that is constructed at 45 ~ to the first one (exactly as we construct octagons in Z2). This possibility is illustrated in Figure 15. The elementary cuboctahedron C does not lead to a sequence of segments. It hag the decomposition shown in Figure 16 (with the horizontal spacing being again equal to x/2). To dilate a f u n c t i o n f b y C, it suffices to perform
f2 = f @
:), (+o+)
0
0
0 0 U 0 0 1 1 1 1 1
a n d then to c o m p u t e the sup between f l a n d f2 + 1: f 9 C = s u p { A , f2 + 1}.

0 1 0 1 1 1 0 1 0 =
co
c1
Figure 16. The elementary cuboctahedron C and its decomposition into two successive horizontal planes.
65
2. Openings and closings

This section is a go-between from dilations to morphological filtering. Here, the two basic references are Chapters 7 and 17 of [25] and Chapters 1 and 5 of [153. We see how, by looking for an inverse to the dilation--i.e., for an impossibility--we find a new operation, the morphological closing, whose three basic properties are extremely useful. We then try and keep these properties as axioms for the general concept of an (algebraic) closing. The notion of an opening is introduced by duality. It satisfies two of the three basic properties of the closing, that will become the two axioms of the morphological filtering in the next section. 2.1. Morphological opening and closing Generally, in a complete lattice J-, the dilation X ~ 6(X) and the erosion X--+ e(X) do not admit inverses and there is no way of determining one element X from the images 6(X) or e(X). However, starting from a dilation and then performing the dual erosion (or the contrary), we always have either an upper, or a lower, bound according to the situation at hand. Indeed, if we take 6(X) for the set Y in (12), the left inclusion is satisfied, so X _< ca(X), and, by duality,
,~ o e ( X )
< x < ~ o a(X),
or, in terms of operators,

& _< I _< ~a.
We say that ea is extensive (larger than the identity mapping) and that & is anti-extensive. Both operations are also increasing as the product of increasing mappings. Now, ea _> I implies, by growth, that && > &, whereas & < I implies the inverse inequality. Hence & = & a e , i.e., is idempotent (as well as ca, by duality). The three properties of ea characterize what is called a closing, in algebra, and those of & an opening. We call these two operators morphological to indicate that they are generated from a dilation and its dual erosion, and we denote
ym = & . ~0= = ca. (30)
Figure 17 shows an example of a morphological opening and of a morphological closing of a set X in the plane. In this two-dimensional case, a morphological opening may remove three types of features: capes, isthma, and islands. By duality, a morphological closing may fill gulfs, channels, and lakes. Let Z = 6(X) be the dilation image of an arbitrary element X e J-. We have {>~:xXI by extensivity of e6, 7re(Z) = &a(X) by anti-extensivity of &.
66
SERRA AND VINCENT
~:..
X
.. - (~(X)
D= Figure 17. Examples of a morphological opening and of a morphological closing of a set X by a disk D.
Hence 7,n(Z) = Z, i.e., Z belongs to the class ~ of the invariant elements of ~3- under 7~- Conversely, if Z e ~ , then Z = 6(~(Z)), i.e., is the dilation of an element of ~-. To summarize, we have the following theorem: Theorem 2.1. Given a dilation 6 on lattice Y and its dual erosion 5, the composition products 7,, = 65 and ~o~ = 56 are respectively an openin9 and a closing on J-, called morphological. The invariance domain of the former is the image of Y under 6 and that of the latter forms the image of 5- under 5. Corollary 2.2. Given X e J-, 7re(X) is the smallest inverse image of X under e, and ~Om(X is the largest one under 6. ) This corollary is illustrated by Figure 18.
P r o o f . Suppose that Y ~ Y is such that e ( Y ) = e(X). Then, a fortiori, Z =
e(X) < 5(Y) and thus, applying (12), 6(Z) < Y or 7=(x) _< Y.
By duality, we also have YYEY, which completes the proof.
~(Y) = ~(x)
Y ___ ~,=(x),
[]
Figure 18. ym(X) is the smallest element Y ~ ~-- such that e(Y) = e(X).
AN O V E R V I E W
OF MORPHOLOGICAL
FILTERING
67
Corollary 2.3. If ~ and ~' stand for the invariance domains of ?m and (o,., respectively, then
~)m(X) V{B, BeM, B < X}, _

:
(31)
(32)
'Pm(X) =
Proof. From (13), we have
A { B , B ~ ~', B >_ X}.
7dx) = ~(~(x)) = ~(V {B ~ 9-, ~(B) _<X}) = V {6(B), B e ~-, 6(B) _<X}, but according to the theorem, ~ = {6(B), B ~ ~--}. Hence, we get (31). Equation (32) has a dual proof. [] Example. We have seen in Section 1.5.4 that planar increasing mappings preserve vertical walls. Figure 19 typically illustrates this point by showing the morphological opening of a one-dimensional function by a horizontal segment. Unlike this kind of opening, circular openings (i.e., openings with disks) do not preserve the vertical parts of the one-dimensional functions on which they act. In this case, Figure 19 clearly indicates changes of slope. The same remarks apply in the two-dimensional case and the experimenter must choose between one approach or the other according to his purpose. It should be noticed that "planar" structuring elements are preferred most of the time, since computation of the corresponding openings and closings can be done more efficiently than with three-dimensional structuring elements.
,".. h", opening by S =
f
m . l'""
opening by
= W_.._~
Figure 19. Comparison between the openings of a one-dimensional function by horizontal segment S and by a disk D.
68
SERRAA DVN E T N I CN
2.2. Algebraic openings and closings The important Corollary 2.3 directly associates an opening 7m with its invariant elements, without referring to the intermediary erosion and dilation. Should it also be true for any algebraic opening 7, i.e., for any operation on ~'- which is increasing, anti-extensive, and idempotent? Let 8 be the invariance domain of such a 7, and let B be an invariant element, B _< X. Then (by growth) B = 7(B) < 7(X), hence 7(X) >_ V {B, B ~ 8, B < X}. But 7(X) e 2 (by idempotence) and 7(X) < X (by anti-extensivity), therefore 7(X) is one of the B of the right member. Thus, (31) is valid for any opening. Conversely, start from an arbitrary part 20 of lattice 9-" and let 2 be the class closed under union generated by 20. The operation defined by 7(X) = V {B, B e 2o, B < X} (33)
is increasing and anti-extensive. Moreover, 7(X) = X iff X E 8. The product 7 ~7 is smaller than 7 (growth and anti-extensivity), but also
77(X) >- V {7(B), B e 20, B < X} _ = V { B , B e 2 o , B _ X}

= ~(x).
We may therefore state the following: Theorem 2.4. An operation 7 (resp. r on Y is an opening (resp. a closing) if and only if there exists a class 2 ~_ ~--, closed under union (resp. intersection) such that 7(X) = V {B, B e 8 , B _< X}, q0(X) = / ~ {B, B ~ 8 , B > X}. 2 is the invariance domain of 7 (resp. rp). In other words, we can approach openings and closings either directly or via their invariance domains. NOW, what about the composition, the sup, or the infof openings. Are they still operations of the same type? As far as sups are concerned, the answer is yes. Indeed: Theorem 2.5. The sup of a family (7i) of openings is again an opening, whose domain of invariance is the class closed under union generated by the union of the 8 i (invariance domains of the 7i's). Proof. Clearly, V 7, is increasing and anti-extensive. Furthermore, for all i, we have 7,~ (V 7,) > 7,. Therefore, (V 7,) ~(V 7,) > (V 7i), and also the inverse inclusion, since (V 7,) < I. This gives us the idempotence. The domain of invariance is determined as before. []
69
Unfortunately, the class of the openings is neither closed under /x nor under composition. Consider for instance in Z the following set: X = ..1111..111111..1111... and the two structuring elements A=.I ..... 1. and B=.lllll.
Denoting the associated morphological openings by 7A and 7B we have yA(X)=X and and ?B(X)=.llllll.
Hence,
and
(vB ^ ~A)(?B /' 7A)(X) = s # (TB ^ ?,0(X) = ~B(X).
Let us quote a last result which clarifies the links between morphological and algebraic openings: Theorem 2.6. A mapping 7: Y- ~ Y is an opening if and only if it is the sup of a family (71) of morphological openings. Moreover, if a translation is defined over ~--, 7 is translation invariant if and only if the 7~ are translation invariant (dual statement for the closings). Proof. Easy, refer to p. 190 of [15], p. 161 of [24], and p. 22 of [25]. []
2.3. (Nonexhaustive) catalog of openings and closings Although Theorem 2.6 is heuristically deep, we may have difficulty in applying it directly, as the number of terms 7i necessary for generating a given 7 becomes prohibitive. Actually, there are four starting points for creating openings, namely: 9 9 9 9 the the the the morphological openings; trivial openings, connected openings, envelope openings,
plus any derivation obtained by cross-union of these various types. The first mode has already been developed. We now present the other three.
70
SERRAAND VINCENT
2.3,1. Trivial openings. A criterion T is said to be increasing when, for all

X~J-, X satisfies T a n d Y > X X does not satisfy T a n d Y _< X ~ ~ Ysatisfies T, Ydoes not satisfy T.
F o r example, in ~", for X to hit a given set A o as well as having a Lebesgue measure larger than a given value 2 o are b o t h increasing criteria. Proposition 2.7. Given an increasing criterion T over lattice Y , the operation 71(X) =
{XwhenXsatisfiesT, otherwise
(with 7 1 ( ~ ) = ~ ) is an opening called the trivial opening associated with criterion T. 2.3.2. Connected opening 7~- We consider a boolean lattice ~(E) and an arbitrary point x ~ E. A part cg of ~(E) is called a eonneetefl class on ~ ( E ) when it satisfies:
(i) ~ ~ cg and, Vx ~ E, {x} ~ ~. (ii) F o r every family (Ci) in cg, ("] Ci :A ~ ~ U Ci ~ ~g. We prove then [-25, p. 523 that the d a t u m of a connected class ~ on ~ ( E ) is equivalent to the family of openings 7x such that: (iii) Vx ~ E, 7x({x}) = {x}. (iv) VA ___E, x, y e E, 7x(A) and 7y(A) are equal or disjoint, i.e., 7~(A) C~Tr(A) r ~ ~ 7x(A) = 7y(A).
(v) VA g E, Vx e E, x q~A ~ 7x(A) = ~ . At first sight, this theorem just indicates that the operation shown in Figure 20 is an opening called the connected component of A that contains
Figure 20. The opening called the connected component of A that contains point x.
AN OVERVIEWOF MORPHOLOGICAL FILTERING
71
Figure 21. A less obvious connectivity notion, associated with the opening v~.
point x, which is somewhat obvious. But it also tells for instance that the operator vx(A),={(7~ofi(A))c~A
when x ~ A , otherwise
(where 6 is a dilation by a disk) is again a connected opening associated with the connectivity shown in Figure 21, which is less obvious [25, p. 55].
2.3.3. Envelope openings. Consider a finite lattice Y and an increasing mapping ~k: J - ~ ~--. Then, for any X e J , the sequence IX c~ ~9(X)]" decreases with n, and finally stops for a certain no, since Y" is finite. The operator
= (I
A
~b) "~
(34)
is therefore an opening. Moreover, if h is an opening smaller than @, then h < I/~ ~,. Hence h = h" < (I A @)" for every n and thus h < @. In other words: Theorem 2.8. Let J- be a finite lattice. Then, for every increasing mapping
@: J---+ 9--, there exists an upper envelope @ of the openings which minorate ~9. It is itself an opening and is given by the relation ~/ = (I
A
@),o
for a finite n o
(dual result with @ = (I v @)"~

Some interesting remarks and counterexamples concerning the case of nonfinite lattices can be found in [5]. Note. (i) The iterations may well stop at the first step. In Section 3.5.3 the example of the rank-operators illustrates this point. (ii) Under conditions which are always fulfilled in practice, Theorem 2.8 extends to the lattice of the functions f : Z" ~ Z [26].
72
SERRA AND V I N C E N T
2.4. Cross-unions of basic openings 2.4.1. Reconstruction algorithms (combination of types 2 and 3). A series of algorithms are based on the same approach and resort to the so-called 9eodesic methods [10]. For example, for two-dimensional binary images: keep the connected components of X whose circular openin O of size k is not empty and filter out the others. The algorithm is extended to numerical functions via their horizontal sections, but, as we will see, it can be directly implemented in terms of gray-scale operators. In this section we only consider the discrete cases of the lattice ~(772) and of the lattice ~ ( Z 2, Z +) (functions from 7/2 into Z+). In practice, they correspond respectively to binary and gray-scale morphology. For all k ~ Z+/{0}, denote by ~k and &k the isotropic erosion and dilation of size k (square, hexagonal, octogonal . . . . ) in ~(Z2), as well as in ~(Z2, 77+). We first focus on the set case: let X be a finite set of ~(7?2). We call geodesic dilation of size one in X the mapping & defined on ~(X) as follows: x 6x: ~(X) ~ ~(X), Y ~-, 61(Y ) c~ X. (35)
The geodesic dilation of size n > 0 inside X, denoted &~), is then simply defined by iterating fix n times: &~) = &x o 6x o"" ~6x.
j
(36)
n times
Note that this operation has already been illustrated by Figure 10. Now, for a given set r ~_ X, 6~)(Y), 6~)(Y),..., &~)(Y). . . . constitutes an increasing sequence of sets, which tends toward an idempotent limit. This limit, denoted px(Y), is called the reconstruction of set X from the "marker" Y: px(Y) = lim 6~)(Y).
k-~ + oo
(37)
In practice, this transformation allows us to extract--or to "reconstruct"-those connected components of set X which are said to be "marked" by IT, i.e., whose intersection with Y is not empty. Let us now take the erosion of size k of X--i.e., ek(X)--as the marker set. The reconstruction operation allows us to extract the connectedcomponents of X that do not completely disappear after an erosion (or after an opening) of size k, i.e., that can hold the isotropic element of size k. Denote by #k the transformation thus brought to the fore: VX ~ 7/2, #k(X) = px(ek(X)) = Px(Tk(X)), (38)
where 7k stands for the opening with respect to the isotropic element of size k. It is very easy to show that /~k is an increasing, anti-extensive, and idempotent operation. It is therefore an algebraic opening, often referred to
Original image
Opening by a disc
Reconstruction
Figure 22. Algebraic opening which is defined as a morphological opening by a disk followed by a reconstruction. as opening by reconstruction. Contrary to the morphological opening 7k, #k allows us to preserve the shape of the components that are not filtered out. It is illustrated by Figure 22. Note that the same technique still yields an opening when replacing ek by any increasing mapping in (38). As already mentioned, these notions extend directly to the gray-scale case, i.e., to lattice ~(77 2, 77+). Here, ek, 6k, 7k, and q5k denote respectively the erosion, dilation, opening, and closing with respect to the fiat isotropic element of size k. Given two functions f and g such that g < f, the geodesic dilation of size one of g with respect to f is denoted by as(g) and given by 6s(g) =
61(9) A f.
~ 1 7 6
J
(39)
Similarly, 6ts")(g)denotes the geodesic dilation of size n of g with respect to f :

= Os ' (40)
.,i;.es
whereas the reconstruction operation Ps is given by the following equation: Ps(g) = lim
k~+oo
6~)(g).
(41)
Similarly as in the binary case, this gray-scale reconstruction preserves the peaks or crest-lines of f that are initially marked by g. It is illustrated in Figure 23. Moreover, the gray-scale opening by reconstruction #k is defined exactly as for sets and allows us to preserve the peaks, domes, or crest-lines of f that do not totally disappear after an erosion of size k. This kind of opening is extremely useful in practice, since it enables very clean and efficient filtering of gray-scale images. Some "structures" are filtered out whereas the remaining ones are left unchanged.
74
SERRA AND VINCENT
""
,-
Figure 23. Gray-scale reconstruction of function f from the "marker-function" g: the right peak of f is marked by 9 and thus preserved in the reconstruction process.
By duality, we can also define gray-scale geodesic erosions, as well as the reconstruction p~ and the closing by reconstruction #*. Suppose that g >_ f. The geodesic erosion of size one of g with respect to f is denoted by ~s(g) and given by ez(g) = ea(g) v f, whereas that of a given size n, denoted by e(z")(g), has the following definition:
e~)(g) = (~so ~- o"" o es)(g ).
ntimes
(42)
Now, the "dual" reconstruction o f f from the marker-function 9 is defined by
p~(g) = lim s~)(g).

k~+~
(43)
This operation preserves the basins and valleys o f f that are initially marked by g. It is at the basis of the definition of the closing by reconstruction t~ of a function f :
I~(f) = P~((~k(f)).
(44)
Moreover, as we shall see in Section 3.6, the dual reconstruction p~ enables a very efficient filtering of the minima of a gray-scale image. This turns out to be extremely interesting for the segmentation of decimal images by means of watershed techniques [1], [-3].
2.4.2. Annular opening (unions of type 1). Consider the pair of points
B = {o, b}, made of the origin o and a point b in direction ~ in ~2 or in 772. Clearly, the morphological opening 7b with respect to B is equivalent to
7b = I A 6B, ,
where aa' is the t-dilation by the bipoint B' = { - b ; +b}. Now, vary b in a certain domain D which does not contain the origin (e.g., three consecutive vertices of an hexagon centered on o, half a circle. . . . ) and take the sup 7i
7 = V{7b, b e D } = 1 /', {VaB,, b e D }
Figure 24. Annular opening 7 versus a classical opening by a disk and a union of openings by segments.
i.e., since the dilation commutes with v : 7 = I / x 6z~u6, (45)
where 6D~6 is the dilation by D w / ) = Ub~D{--b; +b}. The effect of this annular openin9 7 is shown in Figure 24. 7 eliminates the components of a given set X as a function of their environment more than of their size or shape. On the example presented here, D w / ) is taken to be a circle and 7 eliminates the central particle without touching the others. To illustrate the specific action of 7, we can compare it with the morphological opening 7' by a disk and with the union 7" of the morphological openings by segments in various directions (see Figure 24).
76
SERRA AND VINCENT
3. Morphological filters
3.1. The lattice of the increasin9 mappings This section constitutes an overview of the theory of morphological filtering, due to Matheron [25, Chapter 63. The lattice examples introduced in Section 1 concerned the scenes under study. We now consider classes of operations working on these objects. Let r be such an operator, i.e., be a mapping from a complete lattice ~-- into itself. We assume that r is increasing, i.e., that it preserves the ordering relation of 3-: VA, A'~ 3-, A >_ A' ~ ~(A) > ~(A'). (46)
The set Y ' of the increasing mappings on the complete lattice J - satisfies the following properties: 1. Y-' is a semigroup for the composition product o, with a unit element, namely the identity mapping I (VA ~ ~--, I(A) = A). 2. Y ' is a complete lattice for the ordering relation, f > g .~ VA e g , f(A) >_ g(A),
since the following identities, ( V f0(A)=V(f~(A))

, j
and
(j~, f~)(A)=A(f~(A),,
, y
generate a supremum and an infimum in the set ~-'. The two basic structures of the semigroup and of the lattice interact with each other, and we have, for all f, 9, h, and (fi) in Y-',
( V ~)o g = V(~o g), ( A ~)o 9 = A(fio g), and
g.o ( V ~) > V(g ~f), go ( A ~) < A(g ~~),

~f[ h>-g~ (h f > - h ~
(4"7) (48)
f >g
-
In the following, the two classes of the overfilters (i.e., the mappings f ~ J ' such that f o f > f ) and of the underfilters play a major role. Indeed: Theorem 3.1. The class of the underfilters (resp. overfilters) is closed under A (resp. v ) and under self-composition. For example, let (fj);E J be a family of underfilters. From (46) and (48), we get
is J \
jsd
ieJ
i~J
77
so that /~s~s fs is an underfilter. Moreover, given an underfilter f, f f _< f implies, by growth, that f f o f f < if, so that the self-composition f f is an underfilter. 3.2. Morphological filters." definition Following Matheron and Serra [25, Chapters 5 and 6], we define the notion of a morphological filter as follows: Definition 3.2. The elements of J-' which are both underfilters and overfilters are called (morphological) filters. Note that in literature, the term "filter" may also be associated with growth only [12], [13], and can even be synonymous with mapping [37]. Here, however, the morphological filters are the transformations acting on the scenes under study (i.e., the lattice 5-) which are increasing and idempotent. We denote by "f" the class of the filters, with ~ _~ 3-'. Remark that the class "U is not closed either under v, or A, or under composition (a counterexample, based on openings, has been brought to the fore in Section 2.2.). This apparent drawback suggests we should investigate more accurately the possible connections of class ~/- with the composition product and with extrema. Can we find, for example, pairs (f, 9) of filters such that f o g, g ~f, f ~g ~f, etc., are surely filters (composition problem)? Can we keep the usual ordering relation in U and equip ~ with new sup and inf, such that it turns out to become a complete lattice (extrema problem)? These two sorts of questions constitute the subject of Sections 3.3 and 3.4. 3.3. Composition of morphologicalfilters With any increasing mapping if: 5" --* 5-, associate: 1. The image domain ~(~-'), i.e., the set of the transforms by if: r = {~p(A),A e J } .
2. The invarianee domain No, i.e., the class of those B e : - which are left unchanged under ~k: ~q, = {B e J , O(B) = B}. When 0 is a filter, N~, is often called the root of ~ in the literature. We always have ~q, ___~(~--), an inclusion which becomes an equality
~o = O(f)
if and only if ~ is idempotent. This preliminary remark leads to the following two criteria:
78
SERRA AND VINCENT
Criterion 3.3. For any mappings f, g from 3- into itself,

fg=g ~. g(~-)=_M e.
In particular, when g is idempotent: fg=g <=> M g ~ My.
(49)
Criterion 3.4. Two mappings f and g from ~- into itself are idempotent and admit the same invariance domain M I = M o if and only if
fg=g and gf=f.
(50)
Proof. Criterion 3.3 is obvious. Now, if (50) is satisfied, then f f = f o gf = gf = f, i.e., f and, similarly, g are idempotent. Hence, from (49), Mz _~ Mg and Mo _~ MI. Conversely, when f and g, idempotent, have the same invariance domain, (50) is nothing but (3.4). [] In these two criteria, the ordering _< does not intervene. From n o w on, we only consider the increasing mappings 0, i.e., the mappings 0 e F ' . For any filter ~, the class of the filters ~,' that have the same invariance domain M0 as ~ is denoted Jd(O). The following theorem is the key result concerning the composition of filters: Theorem 3.5 (Structural Theorem). Let f and g be two filters on 3- such that
f >_ g. Then:
(i)
f >_ f g f >_ g f v f g >_ g f A fg >_ gfg >_ g.
(ii) gf, fg, fgf, and gfg are filters, and f g f E Jd(fg), gfg ~ Jd(gf). (iii) f g f is the smallest filter greater than gf v fg and gfg is the greatest filter smaller than gf /x fg. (iv) The following equivalences hold:
Mfo=Mof <:~ M f g = M f c ~ M o <:*. M o y = M f n M ~*, f g f = gf ~=~ gfg = fg <:,. gf>__fg. o
Proof. The inequalities (i) are obvious. From the relationships

fg = fffg >_ fgfg >_ fggg = fg,
we conclude that fg is a filter. By the dual inequalities, gf is also a filter. Now, we have
f g f o fg = f g ( f f ) g = M g = fg,
fgo f g f = fgfgo f = (fg o f g ) f = fgf,
79
and thus, f g f ~ J d ( f g ) , by Criterion 3.4. In the same way, we find that f o r ~ J d ( g f ) , so that (ii) is proved. Now, f g f is a filter (by (ii)) and f g f > gf v f9 (by (i)). Let r be a filter such that ~ > f9 and r _> gf. It follows that r = 0~ > f g g f = fgf. Thus, f g f is the smallest filtering upper bound of f9 and 9f. Hence (iii) is proved. By Criterion 3.4 we have Nzo = ~gz if and only if
f g o g f = f g f = gf
and
gf o fg = gfg = fg.
These relations actually imply one another. For instance, f g f = 9f implies f g f o g = 9fgf , i.e., f9 = 9f9. By (iii), these relations are equivalent to 9f >- f9. The inclusions ~ f N ,~g CZL~So CZ"~f always hold, so that ~sg = ~'s n ~g if and only if ~sg -~ ~ , i.e., by Criterion 3.3, if and only if 9f9 = fg. This completes the proof. [] Examples. 1. Start from an arbitrary opening 7 and an arbitrary closing 4. Since
~_<I_<4,
by Theorem 3.5, 74, 4~, 747, and 474 are filters. The composition products of 4 by 7, then by 4, etc., generates the oscillating sequence
4 ~ 74 ~ 474 ~ 74 ~
Remark that when 74 > 47 (which is generally not the case), we have 74 = 474 and the oscillations are stopped after the first step. 2. There is a more particular example, which illustrates point (iv) of the theorem. In ~(R") (or ~(7/"), or ~(R", ~), or y(7/", Z)) consider the morphological opening 7, by a segment of length 1 in the horizontal direction. For a given X, 7~(X) is made of horizontal segments of length > L Moreover, closing this set' by the dual closing 41, i.e., determining 4wt(X) may only suppress intervals between two such segments, hence increase the length of the horizontal intercepts; therefore, 7~q~tvz(X) = 4~,(X), and, by the theorem, 4~h < 7141.
3.4. The lattice of the filters

We now go back to the structure of the set ~t of the filters acting on lattice ~ J . Clearly, if (qh) is a family of elements of ~/~, then V ~i is an overfilter,
and similarly, A r is an underfilter. Since V r is an overfilter, the class ~g closed under v and self-composition generated by V ~'~ only comprises
80
SERRAAND VINCENT
overfilters (Theorem 3.1). It admits a largest element f, for J-' is a complete lattice. However, f of also belongs to cg (closure under self-composition), hence f o f < f, i.e., f is a filter. Consider now a filter ~' larger than V Ip~. The class ~g' of the overfilters smaller than ~' is, in turn, closed under v and self-composition, and contains the previous class cg. Hence, f < ~', i.e., f turns out to be the smallest filter which majorates the ~,'s. By duality, we have a similar result for the inf, and we may state: Theorem 3.6. The set ~U of the filters on i f is a complete lattice. For any family OPi) of filters on J , the smallest filter greater than V tPi is the largest element f -of the class closed under v and self-composition generated by (~). Dual result for the largest filter smaller than A tPl. In particular, when Y- is finite, we always have, for a large enough n, f = (V Oi)", (51)
a result which provides the algorithm for computing f in practice. Refer to [5] for some counterexamples in the case where Y is not a finite lattice. Examples. 1. Lattice of the openings: take the class ~//~' ~ Y/- of the openings on J . We have seen that, for every family (7~) in ~ ' , V 7~ is still an opening (Theorem 2.5). Moreover, from Theorem 3.6, there exists a largest filter g which is smaller than all the 7~'s. With g being obviously anti-extensive, ~ ' is a complete lattice. 2. Start from an arbitrary increasing mapping ~. Then the extensive mapping I v ~ is an overfilter and the proof of the theorem shows that there exists a smaller filter ~ hence a smaller e l o s i n ~ t h a t majorates ~,. In the finite case we find again Theorem 2.8.
3.5. v - and A-filters, 7(~ and ~bT, strong filters
3.5.1. Introduction. We say that a mapping f : J" ~ ~-- is a v -mapping when

f = f o(I v f ) and a /x-mapping when f = f o(I A f). (53) (52)
Basically, this property is something new and independent from the two axioms which build the definition of the morphological filters. If, now, f is increasing and satisfies (52), we call it a v-underfilter. Indeed, any v underfilter is an underfilter and, similarly, any A-overfilter is an overfilter.
81
'r'(A)=~'(B) ~
~I'(A)=~I'(B) ~
(a)
(b)
Figure 25. An example of a v -filter (a) and of a strong filter (b). If f is, for instance, a v-underfilter, then we have f=/o(Ivf)>_fvff_>f. Thus, f = f v f f is an underfilter. A filter which satisfies (52) (resp. (53)) is called a v -filter (resp. a A -filter). W h e n it satisfies both (52) and (53) it is said to be a strong filter. The geometrical interpretation of v-filtering and of /x-filtering are very easy. Indeed, ff is a v -filter if and only if, for any A ~ J - , every B between A and A v ~k(A) has the same transform as A itself (see Figure 25), i.e., v -filter: Similarly, we have ~k /x -filter: VA ~ J , (A /x ~(A) < B < A ~ ~,(B) = if(A)). VA ~ 9--, (A < B _< A v ~k(A) ~ ~(B) = ~b(A)).
The following result corresponds to T h e o r e m 3.1 of the general c a s e - - a n d is proved in the same way: Theorem 3.7. The class of the v-underfilters (resp. /x-overfilters) is closed under /x (resp. under v ) and self-composition. (Note the chiasma; it is the sup and not the inf o f / x -overfilters which is still an ^ -overfilter.) This t h e o r e m suggests to us to a p p r o a c h first the properties of self-composition and t h e n t h a t of sup and inf, just as we did in the general case.
3.5.2. Composition o f v - and /x-filters.

Theorem 3.8. Let f and g be two filters on J - and f >_ g. Then: (i) (ii) (iii) (iv)
I f f is a v -filter, gf and f g f are v -filters. I f f is a /x -filter, fg and gfg are /x -filters. I f gf is a /x -filter, f g f is a /x -filter. I f fg is a v-filter, gfg is a v-filter.
82
SERRAAND VINCENT
Proof. Easy [25, p. 119]. Example. If 7 is an opening and ~b a closing, we have 7 < I < q~, and then 9 7 and ~b are strong filters, 9 y~b and ~by~bare v-filters, 9 q57 and Yq~7 are A-filters.
[]
Moreover, if 7~b is a A-filter, and thus a strong filter, ~byq5is a strong filter. In the same way, if q57 is a strong filter, then 7qSy is a strong filter.
3.5.3. The two envelopes 6'~ and 6'~. In Section 3.4 we have associated with each 6' e J-' the largest opening ~ that minorates 6' and the smallest closing which majorates 6'. These two primitives play a central role in the v - and A-characterizations, as is shown by the following theorem:
Theorem 3.9. An increasing mapping 6': J - ~ J- is a A-overfilter) if and only if6' = @(~ (resp. 6' : ~(~). Proof. If 6' = 6'(1 v 6'), then
(I v O)(I v 6')
: I
v-underfilter(resp. a
v 6' v 6'(I v 6')
v ~, v 6'
v 6'.
The mapping I v 6', which is idempotent and which majorates I, is nothing but 6', and 6'(1 v 6') = 6'~. Conversely, start from
_<
v 6')_< v
which implies ~ : @(I v 6'). Now, if 6' = 6'~, then

6' : O ~ = 6'~(I v 6') : 0(I v 6'). []
Corollary 3.10. If 6' e J " is an overfilter (resp. an underfilter), then 6'~ is the smallest v -filter which majorates 6' (resp. the largest /x -filter which minorates
4').
Proof. If 6' is an overfilter, then
and 6'~ is a filter. It is also a v-filter, since
t)~ <_ 6'~(1 v 6'@) <_ 6'@1 v @~) : 6'~,

which implies 6'6'(1 v @ ) = ~ 6 ' . Finally, if 6"^is a v-filter, with 6" _> 6', then 0' = 0'~' > 6' implies 6" > 0, hence 6" = 0'6" -> 0~[] Corollary 3.11. When ~-- is a finite lattice, then, for n large enough, 6'~ = 6'(1 v 6')". (54)
Indeed, Corollary 3.10 associates two envelopes with each filter 0, so that is finally surrounded by four extremum filters as follows:
We have seen that the product 7q~ of any closing followed by any opening ~as a v-filter. We now prove that the converse is true, so that the v -property characterizes the class of the filters of the type 7~b. Theorem 3.12. A mapping 0 ~ J-' is a v-filter (resp. a /x-filter) if and only if there exist an opening 7 and a closing ~ such that 0 = 7d? (resp. 0 = qbT). Proofi Assume that 0 is a v-filter and consider its invariance domain ~. Denote by ~ the class closed for the sup which is generated by B, and by . / t h e associated opening. Clearly, we have 0 -> _/. Moreover, according to Criterion 3.4, ~3 _ ~ implies 0 = ! 0 and (Theorem 3.9) 0 = !0~- Thus, we may write 0=!0=10
-
~>I~
for
for
0->!,
0-<6.
Hence, 0 = !~, i.e., the composition product of a closing by an opening. [] Remark. The above decomposition is not unique. We also have, for a v -filter
Example: the Rank Operators. We illustrate the important Theorem 3.9 by considering some properties of the rank operators [19], [8], [20]. Let f be a function from Z" into Z and let B c Z" be a finite and symmetrical set of p points implanted at the origin. B is the (moving) window in which the ranking operation will be implemented. The transform R k of rank k o f f at point x EZ ~ is obtained by ordering the family (f(Y))y~x with decreasing values (for example), and by replacing f ( x ) by the kth value of the sequence that is thus constructed (1 < k < p). For k = p and k = 1, this leads to Minkowski addition and subtraction. When k = 89 + 1) and p is an odd number, the resulting operation is sometimes called median filtering [37] in the literature. The rank operator R k of rank k is increasing, since it can be decomposed into the sup of the t-erosions el by all the Bi _%B which possess k points: __
Rk(f) = V {e~(f), B i ~_ B, Card(B~) = k}.
(56)
Now, 0i = 6Bei is a /x-overfilter. Indeed:

0i7i = bB~ibiei _ 6 ~ e i 01
for
t~ie i <_ I,
84
SERRA AND VINCENT
and 0i71 = ~Oi. On the other hand, 71 < ~tl implies ~i < I / x Og < I, hence 7i = 71(1/x ~91) and, finally, 01 = fit(I/x ~93. Equation (56) shows that 61~Rk = V ~ is still a /x-overfilter (Theorem 3.7), and by application of Theorem 3.9, I / x 6BRk is an opening (called the Ronse opening). In particular, for k = 1, we find the morphological opening by B, and for k = p, the identity I. The other openings increase with k. As another application of the results presented in this section, we can refer to an interesting study of Meyer El8], where the author directly transcribes into practice and into algorithms the above theorems. Some of the filters which are thus brought to the fore stem directly from classical median filters [37]. Besides, in two papers by Maragos [12], [13] which are among the most interesting ones in the recent literature on morphological filters, we can find a thorough study on the relations between morphological filters and nonmorphological ones, namely median filters, rank filters, and stack filters.
3.5.4. The lattice o f the strong filters. Starting from a /x -overfilter f', the first corollary of Theorem 3.9 ensures that f = f f ' is a v-filter. It would be excellent if it could also keep the /~ -overfiltering property of its primitive f ' . In this case, we would have found the key for producing strong filters. The answer will actually be positive in the case of modular lattices ~--, i.e., such that VA, B , C ~ J - , B>A ~ (AvC) AB=Av(BAC).
(Except for the partition lattice, all the lattices used as models in morphology are modular.) Then we have the following lemma: Lemma 3.13. When ~-" is modular, then
1. f o(I /x f ) is a v-filter for any v-filter f, 2. g o ( l v g) is a A-filter for any A-filter g.
Proof. Easy. Refer to p. 124 of [25].
[]
Theorem 3.14. When the lattice Y is modular, if f ' is a /x-overfilter (resp. a A-underfilter), then f = f ' f ' (resp. g = g'~') is a strong filter. Proof. Let f ' be a /x -overfilter and let f = f f ' .
We have
f ' = f ' ( I /x f ' ) <_ f ( I /x f ) < f.
Now, from the lemma, f ( I A f ) is a v -flter. Since it also majorates f', it is larger than the smallest v-filter which majorates f ' , namely f. Hence, f = f ( I /x f ) is strong. []
85
Corollary 3.15. When Y is modular, the class of the strong filters on Y is a complete lattice based on the usual ordering. The supremum of a family (~Pi)of strong filters is f f ' , with f ' V @i, and the infimum is given by g'(t', with
g'=
Examples. 1. T h e o r e m 3.14 opens the way for the construction of as m a n y

strong filters as we wish, by iterations. It suffices to start from an arbitrary opening 7 and an arbitrary closing ~b: when lattice ~- is finite, there exist two integers n and p such that b o t h m a p p i n g s ~ b = ~ b 7 ( I v ~b~)n and ~'=7~b(IA7~b) n
are strong filters. 2. In some cases, it is not necessary to perform iterations. It is the case, for instance, of the morphological opening 7l and the closing ~bz with respect to a segment l (see Section 3.3). The points which change from 0 to 1 in I v ~bt7z are uniquely those which were initially modified from 1 to 0 by 7z, and preserved as O's by ~bI (see the example of Figure 26). When 7t acts on I v (b171, we then recover ~bl7I. Hence, ~bl71 is strong. However, it is not self-dual, since, by T h e o r e m 3.5(iv), we have 71(bl > qgz7I. As a conclusion, we can say that in the f r a m e w o r k of morphological filtering, the v - and A -properties are weaker substitutes for extensivity and anti-extensivity (closings and openings are strong filters, but the converse is false). Such a weaker version allows us to combine b o t h properties in filters
<
d'
>
I=I Vr 1
(with d < 1< d')
TI= ~ITI
Figure26. A case where it is not necessary to perform iterations: the opening 7z and the closing q~z with respect to a segment/.
86
SERRAAND VINCENT
that are not trivial, whereas the only strong filter to be extensive and anti-extensive at the same time is the identity mapping I. We see in Section 4 how this advantage can be used to produce self-dual filters. 3.6. Filtering as a tool for segmeniation In this section we deal with discrete gray-scale images, i.e., we address the lattice ~(7/2, Z+). One of the main problems with which image analysts are concerned is that of the segmentation of these gray-scale images. By segmenting an image, we mean dividing it into regions, one of which standing for the background and each of the remaining ones representing one of the objects to be extracted. The boundaries of these regions must be as close as possible to the "true" contours of these objects. Therefore, the segmentation task we deal with is nothing but a contour detection problem. The major difficulty is to define the contours of the image I under study at best. The objects to extract are generally light on a dark background, or are dark on a light background. Therefore, we can imagine defining their contours as the regions of I where the gray values are varying very fast, i.e., as crest-lines of the gradient of this image. Depending on the problem, i.e., on the contours that have to be detected, many different gradients may be used. In the field of mathematical morphology, the most commonly used one is often referred to as Beucher's gradient [24]: denote by 6 and e respectively the dilation and the erosion with respect to the flat elementary isotropic element (square, hexagon . . . . ). The gradient of ! is given by grad(I) = ~(I) - e(I), (57)
where the minus refers to the algebraic difference of two functions. In some particular cases, we can also use directional gradients, disymmetric ones like 6(1) - I or I - e(I), etc., and various combinations of these different gradients. Now, to extract the crest-lines of a gray-scale image (namely a gradient image), one of the most appropriate tools is the watershed transformation [2]. Let us briefly give here an intuitive introduction to watersheds (for more in-depth presentation refer to [1], [3], [33], or 1-34]). Consider an arbitrary gray-scale image J, and regard it as a relief or a topographic surface (the gray level of a pixel stands for the elevation at this point). A water drop falling at a given pixel p of J flows down along the relief, following a certain descending path called the downstream of p, until it finally reaches a minimum M of J, i.e., a connected plateau of pixels at a given level h such that every surrounding pixel has an altitude strictly higher than h. The pixel p is said to belong to the catchment basin C(M) associated with minimum M: Definition 3.16. The catchment basin associated with a minimum M of a gray-scale image J is the locus of the pixels p such that a drop falling on p eventually reaches M.
87
Figure 27. Minima, catchment basins, and watersheds.
The crest-lines which separate different catchment basins build the watersheds--or dividin 9 lines for some authors--of image J. These notions are illustrated by Figure 27. In fact, the brutal computation of the watersheds of the gradient image grad(I) does not provide a good segmentation of I. Indeed, whatever gradient is used, the simple computation of its watersheds most of the time results in an oversegmentation: the correct contours are lost in a mass of irrelevant ones, i.e., I is divided into too many regions. This is true even if we take the precaution of using some simple filters on the original image I or on its gradient before computing the watershed transformation. This oversegmentation is simply due to the fact that the gradient image exhibits too many minima. To eliminate this problem we may try to remove the irrelevant contour arcs of the watershed image, but this is usually a difficult task. For some very complex segmentation problems, it may be interesting to work directly on the tessellation of I provided by the above catchment basins: we can consider their associated adjacency graph and resort to region-growing techniques, which can themselves be efficiently approached via watersheds on graphs [31]-[33]. However, in most practical cases, there is a much better solution which consists in modifying the gradient function before computing its watersheds. The idea is to make use of grad(I) for the construction of a function O(I) whose watersheds provide the desired segmentation. To achieve this goal, a very powerful method was introduced by Meyer, which is detailed in [3]. Rather than filtering out the irrelevant minima of the gradient blindly, this technique assumes that markers of the objects of I are available. By marker we mean a connected component of pixels which is included in the object which is marked. The development o f a robust automatic marking procedure may well be an extremely complex task, and has to be adapted to each case. Usually, some external knowledge on the collection of images under study has to be used. From now on, we assume that a set of markers is available. We also suppose that a marker of the background has been brought to the fore.
88
SERRA AND VINCENT
Let S be the set of pixels of I belonging to one of the above markers. Starting from grad(I), the construction of 0(I) is done in two steps: 1. Impose as minima the previously extracted markers. 2. Suppress the unwanted minima. In step 1 we simply construct the function f defined by Vp, fc f(P) = ~grad(I)(p) when p ~ S, otherwise,
with c being an arbitrary constant, strictly minorating grad(I). In step 2 we have to filter out the unwanted minima of f, without forgetting to fill up their associated watersheds! To do so, we first construct a function, say 9, as follows: {A
Vp,
9(P) ---
when P ~S, otherwise,
with A being an arbitrary constant majorating grad(I). Then, using the dual gray-scale reconstruction operation p~ presented in Section 2.4.1, we build 0(I) as follows:
0(i) =
These two operations are illustrated in Figure 28. We can see that the resulting function 0(I) is such that its dividing lines correspond exactly to the desired contours. Indeed, the highest crest-lines of the gradient separating the markers have been preserved. According to the set of markers available and to the gradient which is being used, we therefore have extracted the best possible contours! Note that, given a set of markers, the transformation (grad(I)~-~ 0(I)) is nothing but a strong filter: it is indeed the composition product of a closing by an opening, and can also be defined as the composition of an opening by a closing. We have thus illustrated how we can develop customized filters adapted to a specific problem--namely the filtering of unwanted minima. The present method turns out to be
P~esultinR A
lllnCtlOn0(I) "',, ~
--
o
........ "Iml~osed minima
Figure 28. Construction of the function 0(I) whose watersheds correspond to the desired contours.
AN
OVERVIEW
OF MORPHOLOGICAL
FILTERING
89
extremely powerful in a number of complex segmentation cases, since the only problem (which can itself be very complicated !) comes down to detecting the markers of the objects to extract. 4. Granulometries, alternating sequential filters 4.1. Size distributions Size distributions (also called granulometries) deal with families of openings or closings that are parametrized by a positive number (the size) [14]. More precisely, we have the following: Definition 4.1. A family (74) of mappings from Y into Y-, depending on a positive parameter 2 is a size distribution when (i) V2 > 0, 74 is an opening, and when one of the three equivalent conditions (ii)-(iv) is fulfilled: (ii) 4, # > 0 ~ 7~7u = 7~74 = 7sup~,u), (iii) 2 _> # > 0 ~ 74 < 7u, (iv) 2 > # > 0 ~ B 4 _ B u. Similarly, we introduce also antisize distributions as the families of closings (qh), whose dual openings build size distributions. These conditions are called Matheron's axioms for sized distributions [15, p. 192]. It is easy to verify that these conditions are satisfied by every process that common sense would designate as a size distribution. For example, consider the following structuring elements:
9 9 9 9 I 9 9149 9149 9 9149 9 e6 Q9 000 9 000 9 Q9 e7 000 00 9 cos e8 OOO 00001 9149 eeoc 9 e9
00
e0
el
e~
e3
e4
e5
If we denote by 7i the morphological opening with respect to structuring element e i, then a m o n g the various sequences 7o 71V 72 75
77
7o 71V 72 V 73 V 74 76
77
70 71V Y2 75
77
70 73 V 74 75
~8
7o Yl V 72 75
76
7o 75 77
78
79
79
79
79
78
79
the four former ones lead to size distributions, but not the last two. Note. Remark that the convexity of the structuring elements is not a
90
SERRA
AND
VINCENT
necessity. For instance, the sequence

| . . . . el 9 O9 00 O9 9 O9 OIQ OIO 9 OOO COO 9 9 0OO
e2
e3
e~c.
induces a size distribution. However, in the Euclidean space ~", a family (B~)a_>o of structuring elements generates a size distribution (Ta)~>_owhich is compatible with the magnification, i.e.,
V)~ > O, VX c ~", 2z(X) = & i ( X / & (58) if and only if the B,'s are the homotethics of a compact convex set B. The significance of (58) is clear: it just means that 7z acts on 2X just as 21 does on X. Such a property, which is always satisfied for convolution products, may not exist for morphological filters. However, in the two important cases of the size distributions and of the alternating sequentiat filters (see Section 4.2), we easily obtain it.
4.2. Alternating sequential filters This section is devoted to the presentation of a particular class of filters, namely the alternating sequentialfilters (ASF). It provides a simplified version of the results proved in Chapter 10 of [25]. The theory developed in [25] requires monotonous sequential continuity for covering both continuous and discrete cases. For the sake of simplicity, we restrict ourselves here to the latter case (i.e., ~(7/"), ~(Z", 7/), or planar graphs), which allows us to avoid monotonous topology. Furthermore, so as not to bore the reader, most of the proofs are omitted. They are indeed all very similar to one another and the missing ones can be found in Chapter 5 of [28].
4.2.1. Introduction. The appearance of some alternating sequential filters (of Ni type, see below) in the world of mathematical morphology is due to an experimental work of Sternberg [30] (the corresponding theoretical developments are due to Serra [25, Chapter 10]). His study consisted in taking a polyhedric form (namely a cuboctahedron, see Section 1.6), altering it by the addition of a largely varying white noise, and then trying to clean the resulting image X (see p. 204 of [25]). To attain this goal, X was first filtered by a small dosing ~b1, followed by a small opening 21, then by a slightly larger closing ~b2, followed by. a slightly larger opening 22, etc. The final operator produced by this succession of openings and closings was
M = (2i~bi). . . . o (22(J~2)o (21(~1). The family (~bi) that was used in this example consisted in morphological closings by homothetic structuring elements, wheras (V~)was the dual family of openings. After this experiment, a certain number of questions arose: is the operator M a filter? To what extent does it depend on the totfility of the sequence of parameters 1, 2. . . . . i? Is it essential to use a size distribution (21) and its dual (~b,)? Is the product of these operators an operator of the same type?
In this section a more formal approach of the above operators is proposed. The ASFs of type M are defined, as well as other classes of ASFs, namely that of type N, R, and S. A number of properties of these operators, dealing with composition and order relation, are then given, and the symmetrical sequential filters are also presented. The end of the section is concerned with more concrete problems, such as computation time and practical use of these filters.
4.2.2. Definitions. In the following, ~-- denotes a discrete lattice such as ~(Z") or ~(7-", 7J). We also consider two families (?i)i_>a and (q)i)i>_1--indexed on 2~+/{0}--of operators on Y , which are a size distribution and an antisize distribution, respectively, i.e., Vi c Z,
i > 1,
Yi is a n opening and q~i is a closing, l_i_<j, Yj-<Ti-<I-<q5 i<~bj.
(59) (60)
Vi, j~7/,
Moreover, we have seen in Definition 4.1 that the inequalities (60) are equivalent to the following property:
Vi, j c 7-, ?iYj = ~)j~i -~ ~max(i,j) and ~ i ~ j = ffij(fii = ~)max(i,j)
(61)
Note that these two families are chosen independently from one another (although they are often taken as dual of each other in practice). Now, we know (refer to the structural theorem (Theorem 3.5)) that there are four different ways for composing two filters f and g such that f < g in order to get new filters. Therefore, we define, for all i ~ 7-, i > 1, the following operators:
mi = ~)ir ri = (fiiTi~)i,
(62)
nl = ckiTi,
si = yic~i~i,
which are filters (see Theorem 3.5). From now on, we give the proofs only for the case of the m~, the other cases being left to the reader. Proposition 4.2. For i, j E 7_ such that 1 < i <_j, we have
~rjr~
m j m i <_ m j <_ m i m j , <_ rj,
(rirj) ~sjsi'~ (s~ s i .)
(63) (64)
ninj < nj < n-ini,
> sj.
Proof. From y~ _< I, we obtain y~bi < q~i < ~b-i.Hence, ~bjyi~bi < ~b-iand, finally, mjmi = ?j~b~yi~bi< yj~-i = m~, which is the first inequality. Similarly, I < ~b i implies Yi < Y i ( P i 9 Thus ?j = ?i'gj < Yi~iYj and mj = yjqbj <_ ?ifbiyjq5 j = mlm-i. This completes the proof. [] Proposition 4.3. Let (ik)l zk<_v be p numbers such that
Vk, ik e Z, l < ik <__il = i v ,
92
then
SERRA AND VINCENT
mipmip 1''" mi2mil nipnip 1 " " ni2 nil
mip = rail, nip ~- nil"
Proof. Easy by using (63). Refer to p. 47 of [28].
[]
We can now give the definition of the ASFs and prove that they are effectively filters:
Definition 4.4. For all i~ 7/, i _> 1, the following operators,

M i = mimi_ 1 ""m2ml, X i = n i n i _ 1 "'"/'/2hi,
R i ~ riri_l..
"r2rl~
S i : sisi_ 1 ""$2s1,
are called ASFs of order i.
Proposition 4.5. V i e 7/, i >_ 1, the operators
M i , Ni, Ri, and S i are fihers.
Proof. These operators are increasing as compositions of increasing mappings. Moreover, we have
M i M i = (mim i _ l . . . m 2 m l m i ) ( m i = mi(mi_l'"mEml) = M i. l'"mEml)
(by Proposition 4.3)
The idempotence of M~ is thus proved. We can also easily show that N~, R , and Si are filters. []
4.2.3. Properties. We have already said that there is no need for duality between the 7i's and the Oi's. Anyway, an ASF cannot be self-dual, since
N~ ~ M~. We present below some properties of the ASFs with respect to the composition product and the order relationship between operators: the "absorption taws." We then deal with new filters derived from the previous ones and called symmetrical alternating filters.
Proposition 4.6 (Absorption Laws). For i, j e 7/ such that 1 < i <_j, we have
the following relations: M j M i = M j <_ M I M j , N I N j <_ N j = N j N i , R i R j <_ R j = R j R i , SjSi : Sj <_ SiS j.
(65) (66)
[]
Proof. Similar to that of Proposition 4.5.
Definition 4.7. With, for all i ~ 7/, i >_ 1,

M~ ~ m l m 2 " " m i _ l m i , N~ = n l n 2 '' " n i_ lni,
R~ ~ rlr 2 9 9 9 rl
- lri,
(67)
S~ = sls a " " " s i_ tsl,
93
the following operators,
Mi = m~mi, Ni = N~Ni,
Ri = R~Ri, Si = S~SI,
(68)
are called alternating symmetrical filters of order i. We can easily prove that these operators are indeed filters. They also satisfy absorption laws and other properties, which are listed and proved in [28]. In addition we have the following theorem: Theorem 4.8. The set (f/l i, i 6 Z , i>_ 1} (resp. {fVi, i ~ Z , i>_ 1}, {R,, i ~ Z , i _> 1}, {~i, i 6 7/, i _> 1}) constitutes a commutative semigroup offilters satisfying the following internal law of composition: Vi ~> 1, j ~> 1, 2~rii~r = 2~r~up(i,j). i
Note that, apart from the above ones, the only known semigroups of this type are the granulometries. Within each of the four types of operations that we have described in this section (types M, N, R, and S), we can find interesting equalities and inequalities. The most important ones are detailed in [28], together with the corresponding proofs. These formulas lead to a better understanding of the ASFs and tell us how they should be composed to get new filters. Moreover, they help simplifying the expression of a given filter until the most "compact" expression is reached. This allows us to optimize the computation times.
Compatibility Under Magnification. When the two primitive families (?~)and (~bi) are compatible under magnification, i.e.,
VX, Vk > O, ~ f c~k(kX) = kq~l(X)' ( 7,(kX) = k7l(X),
(69)
this property is transmitted to the corresponding ASFs. The physical interpretation of this is clear: it means that the ASF of order k works on the k times magnified image exactly as does the ASF of order 1 on the initial image. In fact, it is probably this property which makes the ASFs of such common use.
4.2.4. Applications, computation time. In practice the ASFs are among the most efficient morphological filters and can be finely adjusted to each case. Indeed, one can operate on:
9 the families (7i) and (~bi) (most of the time, this comes down to choosing families of structuring elements),
94
SERRA AND VINCENT
Table 1. Computation time Of some sequential filters, provided that the time required for computing 7e or ~b equals i At (for i a fixed image size). Filter M i, N i R~, S e Mi, Ni R~, S, Computation time i(i i(i 2i(i (2i2 + 2i + + + + 1)At 2)At 1)At 1)At Computation time for i = 5 30 35 60 65 x At At At At
9 the type of filter (alternating sequential filter, alternating symmetrical filter), 9 the "size" of the filter. The only problem with these filters is that of the computation time. Indeed, although the formulas for computing them can be "compacted," a certain number of elementary operations has yet to be performed in each case. As an example, suppose that the computation times of q~e and 7e are equal to i At, for a fixed image size. Then the computation time of M e, N~, etc., is proportional to iz, as shown by Table 1, where we suppose that the most efficient formulas are used.
5. Activity lattice, toggle mappings

5.1. Toggling, optimization, and self-duality In Section 1 we have seen that, given a set E, the two sets ~(E) = {X, X _~ E} and Y(E, ~) = {f, f : E ~ ~} are complete and distributive lattices. The first one is even complemented. Later, in Section 3, we have associated with any complete lattice ~- the set J ' of the increasing mappings applying Y on itself, and shown that ~--' is in turn a complete lattice. When starting from ~(E) or ~ ( E , ~), we denote this second lattice ~ ' or Y ' , respectively. More generally, the set of all the (increasing or not) mappings ~O: 3- ~ Y is still a complete lattice g-" (same approach as for ~--' in Section 3), denoted ~'" and ~-" in the two cases of sets and of functions. The idea developed in this section consists in associating with a function f (or with a set X): 1. A series of possible transforms ffif. 2. A toggling criterion, i.e., a decision rule which determines at each point x ~ E the "best" value among the "candidates" (~Oef)(x). Such an approach allows us to introduce optimality conditions. The "best" choice may be understood in the sense of noise reduction, of contrast
95
enhancement, or of both. It may also be understood, for multispectral images, in the sense of edge matching between channels, etc. The toggling approach corresponds to the various minimization techniques involved in linear processings (e.g., minus squares). But there is no quadratic form here. The candidate mappings r play the role of the Lagrange coefficients. Moreover, for minimizing the criterion, the inf is used rather than derivatives. Toggle mappings are not necessarily self-dual for the complement (set case) or for the negation (function case). However, they can always admit a self-dual version. It is obtained by taking the self-dual family (~, r generated by any family (Oi) of primitives, and by symmetrizing the toggling criteria with respect to duality. We give an explicit example of such a procedure in Section 5.3.1, where morphological centers are concerned. In the following we take it for granted that all the algorithms presented in this section possess the property of having self-dual versions, and we do not repeat this point after each result. 5.2. Activity lattice and semilattice
5.2.1. Boolean case. Let ~(E) be a boolean lattice, and let N', N" be the two
associated operator lattices. Equip ~" with the following relationships:
,/,d, Q /
~In~l
~Ic~2,
[ I w O l c_IuO2.
(Note that the second inequality may be rewritten O n 4 h -= | n q~2,
(70)
with | standing for the complement operator, i.e., | X~--~XC.) Obviously, relation ~ is an ordering relationship. Moreover, it makes ~" a complete lattice. Indeed, any minorant m of a family (~'i) in N" satisfies the following system:
{llnm~-Ic~(' wm~_Iutl,
with (=Ur
i
(71)
and
r/= ~ r
i
(72)
Therefore, if there exists a mapping fl verifying I c~ fl = I c~ (,
I w fl
I w ~l,
(73)
then fl is larger, for order relation ~ , than any minorant m. Note that we can modify the second equality of (73) by using again the complement
96
SERRAAND VINCENT
operator |
Iwfl=Iw~l
,~
|174
Now, system (73) admits one and one only solution, since every element e ~ " may be decomposed into = (I w 0) c~ ~ = (I (q ~) w (0 ~ ~). By applying this remarkable identity to 13, we find
13 = (I n 0 u (O n ~)
= [ ( I n ~') u rt] n [(/,'-, 0 u O ]
(74)
/~i
= 13 =
(I w q) n (
(I ~ () w q,
(75)
a quantity which satisfies system (73) (Note that in this section ;x denotes the inf associated with the order relation -Z_ of lattices ~ " or ~ " . It is called the activity inf. In the following, ;/denotes the activity sup.) Similarly, if there exists a smaller majorant 6 to the family (0~), it must satisfy the system I n 6 = I n r/, Iw6 Iw( r162 O n S = O n ( . By using again identity (74), this time we find x(,/0i =
6 = ( O w r/) n ~ = ( O n () u r/. (77)
(76)
Therefore, we have a lattice, which turns out to be complete and distributive (easy proof). We go from the inf ;x to the sup ;/by replacing the identity I by the complementation | As a matter of fact, the two operators I and | are just the null element and the universal element of the lattice generated by order ~ . Applied to set A _ E, the first one does not modify any point of A or of Ac, whereas the second one modifies all the points of A and of Ac. Given two mappings 0~ and 02 in ~", we say that 02 is more active than ~ when 02 ~_ 01. By acting on a set A __ E, 02 removes more points from A than 0t does, and also 02 adds more points to A than 0t does (which is nothing but the geometrical meaning of system (70)). This gives the following: Theorem 5.1. Let E be an arbitrary set, and let ~" be the set of the mappings from ~(E) into itself. The two inclusions
V01, 02e~",
{Iin~1~1~02, u01 ~ / n ~ 2 ,
generate an ordering relation 01 ~ ~b2 on ~", to which is associated a complete and distributive lattice. It is called the activity lattice d . The null element is the identity mapping I and the universal element is the complementation mapping O. If ~l = ~ 0i and ( -- U 01 denote respectively the intersection and the union of a family (01) of elements of ~", then the inf fl and the sup 5 of
97
~f
~~.. ,
~l~ ~::~
(a)
(b)
Figure 29. (a) The anticenter + of ~ and ~/ may no longer be a function. (b) The conditions of Theorem 5.2 are satisfied, so that, in this case, 6f is a function.
family (~91)with respect to the activity are called center and anticenter of the ~i's and are given by the expressions A ~i = fl = (I u q) n ~ = (I n ~) u q,
~/~,
= ,s = ( o u
~) :~ ~
= (o n 0 ~
~.
5.2.2. Function case. The activity ordering (70) applies obviously to the lattice ~ " of the mappings acting on ~ ( E , ~), by changing the symbols c , ~_, u, and n into <, >, v, and A, respectively. The existence and the unicity of the activity inf fl derive from the fact that ~ " is completely distributive [25, p. 164]. Thus, ~ " turns out to be an inf semilattice for the activity order, but no more. As a set, the activity sup of subgraphs of functions exists, but it is not itself a subgraph (see Figure 29). The question which arises then is to know which conditions two mappings ~/and ~ in ~ " , q < ~, must satisfy to define one and only one 6 such that {~
A
6 = I A ~/, v6 Iv~
(78)
(i.e. the equivalent of (76)). For the sake of clarity, we introduce the following simplification in the notation: when a condition applies to all the elements f of ~-(E, ~), function f is removed from the notation. (For example, if at a given point x E E, q f must always be larger than or equal to f we write qx > Ix instead of Vf e 2 ( E , ~), (r/f)(x) >_ f(x).) Using this notation, we may state the following: Theorem 5.2. Let J~(E, ~) be the class of the functions f : E--* ~, and let Y " be the family of the mappings from ~ ( E , ~) into itself. The elements
98
SERRAAND VINCENT
of ~ " constitute an inf semilattice for the activity ordering ~ . Moreover, the sup 6 = ( 9 r/of an ordered pair ((, q) of ~ " exists if and only if {x ~ E, qx < Ix} n {x ~ E, (x > Ix} = 5Z5. The activity sup a is then given by Vx ~ E, ax = (79)
~lx (x Ix
when when when
qx < Ix, (~ > I x, ~lx=(x=Ix.
(80)
Proof. Easy, refer to [17]. In particular, we have: Corollary 5.3. Let z and 0 be respectively an extensive and an anti-extensive mapping from ~ ( E , ~) into itself. Their activity sup c5 = z 9 0 also maps ~ ( E , ~) into itself if and only if, for every f ~ o~(E, ~), zhe two domains X ( f ) = {x ~ E, (Of)(x) ~ f(x)} are disjoined. Finally, we draw two lessons from this section: 1. Any toggle mapping on J~(E, ~) using the activity ordering will also apply on ~a(E), since the former case is more restrictive. Therefore, in the following, we concentrate exclusively on Y(E, ~), ~,~', and ~,~". 2. The dissymmetry between the activity inf and sup suggests treating as a priority the simpler extremum, namely the inf, also called the center. 5.3. Morphological center 5.3.1. Geometrical interpretation. The center fl of a family (~i) in ~-" is itself a mapping on ~ ( E , ~), i.e., an element of Y", whose definition algorithm derives from (75): /~@i=fl=(Ivq)/x~=(I^ 0vq, (81) and Y(f) = {x ~ E, (zf)(x) ~ f(x)}
with */= V ~Oi and ( = A @i. Clearly, algorithm (81) is equivalent to the following system:
Iv~_<fl_<~ r
(82)
When applied, for example, to images ~ ( f ) of a real-valued function f on E, this system states that if, as point x E E, all the ( ~ f ) ( x ) are above f(x), then we take the lowest value. If, on the other hand, they are all below f(x),
99
Figure 30. Morphological center fl between two increasing mappings ~1 and I//2, when applied to the real-valued function f. We see the two properties that are common to all "central value mappings" (average, median, etc.) of an image: they wander between ~Ol(f) and r and are closer to the initial function f. then we take the highest value. In all other cases we leave f(x) as it is (see Figure 30). In particular, when i = 2, i.e., when ~1 and ~2 are the only primitives we have, then the morphological center is nothing but the median of I, ~ , and 02. This bridges the gap between the approach developed by Yli-Harja et al. in [37] and the present one. We are able to produce as many self-dual centers as we may desire. It is sufficient to start with a self-dual family (~). In particular, to each element e ~ " , there corresponds a self-dual element f l ( O ) e ~ ' that is the inf h ~*, where ~*(f) = - ~ ( - f ) . In fact, fl* = [I v (~ v ~*)3" /x (~/x ~*)* = [I v (~/x ~*)] /x (0 v ~p*) = ft.
(83)
We now carry on with increasing centers, indeed, when the ~ ' s are increasing, we see from (81) that they transmit to fl their growth property (as well as their possible planarity). Then the center mapping has the advantage of not damaging high frequencies--as do convolutions--of commuting with anamorphosis, etc. Now, the rank operators, and in particular the median filtering (see Section 3.5.3), are also increasing and planar. However, they may oscillate under iteration (see Figure 31) and even become periodic for some starting functions [25, p. 160]. We now try to avoid this drawback by building up increasing centers which do not oscillate under iteration and tend toward morphological filters.
5.3.2. Iterations ofincreas&g centers. A center fl, of primitives (~), does not
oscillate if and only if, for any function f ~ ~ ( E , ~) and for all points x ~ E, we have either
f (x) <_ (flf)(x) <_ (flo flf)(x) <_ (fl3f)(x) <_...

or
f(x) > (flf)(x) > (flo flf)(x) > (fl3f)(x) > " ' .
100
SERRAAND VINCENT
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ~ 0 0 ~ 0 0 0 0 0 0 ~ 0 0 0 ~ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Figure 31. Oscillations of a median filter. The neighborhood Bp of pixel p consists of the seven points of the elementary hexagon H(p) centered at point p. The transformation replaces f(p) by the median of the histogram of values of H(p). When acting on parallel lines, it oscillates. In terms of operators, this point monotonicity means exactly that the activity of the successive powers fl" of fl increases with n, i.e., Vn > 0, fl" + 1 _~/~,. We now present two classes of fl, where such an activity growth is satisfied. In practice, these two classes cover almost all situations (for a general theory, see p. 166 of [-25]).
5.3.2.1. ( A-overfilters, ~ v-underfilters. We always have

/3ofl = (fl A (fl) v r/fl = (I A ~ A (fl) v (r/A (fl) v ~/fl. (84)
Moreover, when ~ is a A -overfilter and r/is a v -underfilter, inequalities (82) imply that
and, according to (84), tiff = ft. The center/3 is thus a morphological filter. Such a case occurs, among others, when the ~9~'s are strong filters. In such a case indeed, ~ is the sup of A -filters and r/the inf of v -filters. Moreover, when the r are strong, fl admits the following double decomposition:
fl-----(I v q)(I A ( ) = ( I A ()(I v q ) = O ( = ~ O ,
(85)
i.e., by Theorem 3.12, it is itself a strong filter, called the middle element between q and (. Examples. 1. We have seen in Section 3.5.4 that the composition product 01,~Yz,~ of the morphological opening Yl,~ and closing ~b~,~with respect to a segment of length l and direction e is a strong filter (in ~" or in 7/"). Put ~b~ = 4t,~7~,,- Then, in ~2 for instance, the family (O~)~to,~ admits a strong,
isotropic, and self-dual filter fl as center. N o t e that, by starting from 7z,,~z,,, we obtain another center fl', strong itself, isotropic and self-dual, but such that fl' ___ft. 2. We have also seen iri Section 3.5.3 that the median transformation m in a given neighborhood B was associated with the two envelopes
r m
and
~B m,
where 6B and sB stand for the Minkowski addition and subtraction respectively. Now, since 6Bm is a A -overfilter and eBm is a v -underfilter such that 6Bm > sBm, then the center fi = (I A 6~m) v e~m is idempotent. Therefore, it is a morphological filter. 5.3.2.2. ~ v-underfilters, q A-overfilters. Under these new assumptions, we have~<~(Ivl/)<~(Iv 0=~and fl=(Ivl/)A(=(Ivr/)A~(Ivr/)=(IA as well as, by duality, fl = (I v q)(I A O. This gives us fl" = (I A ~)"(I v q)" = (I v q)"(I A ~)". We know, from Theorem 2.8, that when the lattice ~ ( E , ~) is finite, then, for a certain no < + o% (1 v 0 "~ = ~ and (I v q),o = ~. Hence,
8.0 = =
0(Ivq),
The noth iteration of fl yields the middle element between ~ and ~/, in the sense of (85). It is still a strong filter, since it is written as a commutative product of an opening and a closing (Theorem 3.12). Moreover, we have, for all n, IAS"=(IA ~)" and IvS"=(Ivq)",
so that the successive powers of fi are more and more active. When applied to a function f, the successive transforms fl"f coincide with ( f A ~f)" in the zones where (flf)" decreases, and with ( f v q f ) " in the zones where (flf)" increases. Example. It suffices to start from an arbitrary opening y and an arbitrary closing 0, and to take ~ = ~by~band q = y~by. F o r small structuring elements (size one or two), the range of no goes from two to five. For larger ones (e.g., size five), this range goes up to ten.
102
SERRA AND VINCENT
5.4. Toggle mappings A toggle mapping co is defined, on the one hand, by a family (~b~)of reference mappings called primitives, and, on the other hand, by a decision rule which makes, at each point x, cox equal to one of the primitives O~,x. The first example of a toggle mapping which comes in mind is that of the thresholding operation. The primitives are the white and the black, and the decision rule involves, at point x, the value f ( x ) and that of a constant, namely the threshold level. Note that thresholding is an idempotent operator. In this simple example, the two primitives are independent from the function f under study. But it is not always the case, and we have just seen toggle mappings such as the center, where the primitives are themselves transformations acting on the initial image. This leads us to the following formal definition: Definition 5.4. Let ~ ( E , ~) be the class of the functions f : E ~ ~, and let ~ " be that of the mappings from ~ ( E , ~) into itself. Given a family (qJ~) of elements of Y", we call toggle mapping of primitives (~bi) any mapping co of Y " such that: (i) At each point x, cox equals one of the 01,x or I x. (ii) The criterion which affects one of the ~i's, say ~i0, to co at a given point x depends only on the various primitives ~ , on the numerical value I x, and on possible constants. (iii) If at point x, at least one of the ~'s, say ~0, coincides with the identity mapping I, then cox = Ix = ~io,x. (86)
Toggle mappings generate jumps, and the first way of keeping down this effect is to look for idempotent toggles. The following theorem, which involves the class ~ of the continuous functions f : ~ " ~ ~, provides a comprehensive class of such operators [27]: Theorem 5.5. Let, in ~", co: cg __+c~ be a toggle mapping which admits as primitives the family (Oj)j~j of strong filters, and let p and t7 be the two mappings defined by Px = V{~tj, x, J ~ J, $j,x < Ix}, ax = A { $ j , x , J e J , ~kj.x > Ix}. If, for all points x, cox ~ {P~, Ix, crx} and if the family ($j)j~s is ordered for the order >_, then the toggle mapping co is idempotent.
Comments.
9 When dealing with the functions f : Z" --, Z, the distinction between and ~(Z", Z) is cumbersome. 9 The middle element of (85) turns out to be a particular toggle, admitting the strong filters ~b plus the identity, as primitives, and the activity inf i, as a criterion. 9 Associate with co the two following mappings:
{ p~
0~= Ix
when cox otherwise,
__Px,
and
z~--
{.
Ix
w en Ox=.
otherwise.
Then the conditions of Corollary 5.3 are fulfilled, and the toggle co is nothing but the activity sup z + 0 (whereas the Oi themselves do not have any activity sup).
5.4.1. First example." contrast mapping. Take for primitives an opening 7 and
a closing ~b, and the criterion according to which, when 7x ve I~ and Ix ~ ~b~, the value at point x must move down (to 7x) or up (to qSx).This generates a two states contrast which is idempotent (Theorem 5.5). In this simple case, the idempotence may be proved directly, since
Hence, 7x = ~:x ~ (7~)x = ~:x- Now (86), when applied to ~c itself, implies
i.e., by combining these two implications as well as their dual versions,

(~c~ = 7x or ~:x = q~x) ~
(K)~= (~c~:)x,
so that ~: is idempotent. Figure 32 presents an example of such a contrast algorithm (for the sake of clarity, the closing and the opening have been replaced in this example by a dilation and an erosion o f f ) .
5.4.2. Second example: combined to991e. In this second exampl e we combine the antinomic properties of noise reduction of the centers and of contrast
Figure 32. Example of contrast mapping.
104
SERRA AND VINCENT
Figure 33. Function f, four primitives (~f, (f, qf, and 7f. In superimposition, the toggle transform ~of given by rules 1-3 detailed in the text.
enhancement. Let ~ and ~/be two strong filters, and let (7, q~) be a pair made
of an opening and a closing such that
~_<~ <~___q~.
Figure 33 illustrates the arrangement of the four transforms ?f, r/f, (f, and ~bf around the initial function f. According to Theorem 5.5, any toggle mapping oJ is allowed to jump up (or down) at each point x, from f(x) to the nearest transform above (or below)f. However, this flexibility is amended by one constraint: if at point x, f equals one of the four transforms under study, we cannot modify it. As an efficient definition rule for co, we may proceed as follows: 1. When (rff)(x) < f(x) <_((f)(x), apply a contrast , of primitives ~/and (. 2. When (Tf)(x) < f(x) < (r/f)(x), go down to (vf)(x), except if (~f - r/f)(x) is smaller than a fixed value d. In the latter case, go up to (rf)(x). 3. When ((f)(x) < f(x) < (4Jf)(x), apply a rule similar to rule 2, possibly with a scalar d' different from d. The reason for this rule is the following. Usually, we take ~ = ~'7'~' and ~/= ?'~b'7', with an opening 7 ' > y and a closing ~b'_< ~b. We can easily ascertain that ~ and r/ are close to one another in the narrow dark (resp. clear) features when both of them are also above (resp. below) the initial function. In the technique adopted here, such features are considered as nonsignificant, and are reduced, if their sizes and their depths (their heights) are small enough with respect to the large opening 7 and closing q~. 5.5. Weakening of idempotence: the amplifier toggle
mappings
We have used the idempotence assumption because it ensures that the toggling process is well controlled. If not, we would run the two risks of generating, by iterations, oscillations and/or parasitic halos (especially around the peaks and slope changes, see [-17]). However, some behaviors under iteration, such as the fixed zones growth, overcome the parasitic effects.
105
Figure 34. Example of an amplifier co = co"co'based on a strong filter ~9 = 7~b(I/x 7~b)" (with 7, ~b morphological opening and closing). The activity of a m a p p i n g q/is said to exhibit a f i x e d zone 9rowth when q/oq/_~q/ and ~x=Ix ~ (~o~)x=-Ix. (87)
In other words, the successive iterations q/x, ~ . . . . . ~ m a y only strictly increase (or decrease) at point x, or stop. In the latter case, the stop becomes p e r m a n e n t through further iterations. Here is n o w an example of such a behavior, which also shows a toggle m a p p i n g based on one primitive, and called amplifier (see Figure 34). Starting from a strong filter ~k, define the toggle m a p p i n g co' by
Vx ~ E,
~2'(I - q/)x + Ix co'x = ( i x
when when
Ix > q/x Ix < q/x
(~co~ > Ix > q/x), (~co'x = Ix < q/x)
and, similarly, the toggle m a p p i n g co" by Vx ~ E, ~'2"(I -- q/)~ + I x co~ = / . I x when when lx < q/x, lx -> q/x-
Consider the composition product co = co"co'. W h e n cox < (q/co')x = q/x, cox is equal to
(co"co')x = ,V'(co' -
' q/co')x + cox =
2"(co' - q/)x + cox.
However, cox < q/x implies that co~, -- I x, hence (co"co')x = 2"(1 - q/)x + Ix = co~. When co~, > (q/co')x, then co" is the identity o p e r a t o r and cox = (co"co')x = cox. Finally, we obtain co"co' = co" + co' and, by duality, we can state co = co"og' = co'co" = co" + co'. (88)
!
U n d e r iteration, the toggle co exhibits a fixed zone growth behavior, since coco = co"co'co'co" = co"[2"(2" + 2)]co'[2'(2' + 2)].
106
(.0 2
SERRAAND VINCENT
is of the same type as o~, with changes in the intensity factors 2' and 2" only. Therefore, the ~o(2', 2")'s satisfy a semigroup relationship when ~ is fixed, namely
~(2', 2")o(~', ~") = o(,~' + ~' + 2'~', 2" + ~" + ,~"~"). 6. Conclusion
The theory of morphological filtering is recent and first appeared totally formalized in 1988 in E25], where it occupies half the book. As a result, practitioners of image analysis are still surprised by it. We have written this paper to overcome this effect and also to bring to the fore a comprehensive series of practical implementations, from " D o l b y " openings to middle elements and toggle contrasts. Which domains of morphological filtering still lie fallow? In which directions may the method evolve? One of them is certainly that of geodesic filters: up to now, the emphasis was put on the translation invariant operators. A suitable extension should consist of the geodesic approach. Moreover, the functions from E (i.e., ~d or Z ~) onto a vectorial space instead of ~ or Z should open the way to color and vector fields filterings. Last, but surely not least, it should be fruitful to question oneself about the use of morphological filters for interpolating. Typically, linear filtering has progressed in both directions of image cleaning and predictor. Will we witness a similar evolution with morphological filters?
Acknowledgments
This paper was initially written for the morphological part of the EURASIP short course Median and Morphological Filtering in Image and Signal Processing, given at the University of Tampere, in June 1989. We gratefully acknowledge Professors Y. Neuvo and J. Astola for having invited us to participate in this seminar and in this issue. We would also like to thank M. Schmitt for many useful remarks.
References
[1] S. Beucher, Segmentationd'images et morphologiemath6matique, Ph.D. Thesis, School of Mines, Paris, June 1990. [2] S. Beucherand Ch. Lantu6joul, Use of Watersheds in Contour Detection,Proceedings of
the International Workshop on Image Processing, Real-Time Edge, and Motion Detection~Estimation, Rennes, September 1979.
[-3] S: Beucher and L. Vincent, Introduction aux outils morphologiquesde segmentation, in

Traitement d'images en microscopic fi balayage et en microanalyse par sonde dlectronique,
ANRT ed., Paris, pp. F1 F43, March 1990. 1-4] G. Birkhoff,Lattice Theory, 3rd edition, AMS Colloquium Publications, Vol. 25, AMS, Providence, RI, 1983.
AN OVERVIEW OF MORPHOLOGICALFILTERING 107 [5] H. J. A. M. Heijmans, Morphological Filtering and Iteration, Proceedings." SPIE Visual Communications and Image Processing "90, Lausanne, October 1990. [6] H. J. A. M. Heijmans and C. Ronse, The Algebraic Part of Mathematical Morphology, Part One: Erosions and Dilations, Computer Vision, Graphics, and Image Processing, Vol. 50, pp. 245-295, 1990. [7] H.J.A.M. Heijmans and J. Serra, Convergence, Continuity, and Iteration in Mathematical Morphology, Internal Report, CWI, Amsterdam, 1990. [8] G. Heygster, Rank Filters in Digital Image Processing, Computer Graphics and Image Processing, Vol. 19, No. 2, pp. 148-164, June 1982. [9] M. B. Kurdy and D. Jeulin, Directional Mathematical Morphology Operations, Acta Stereologica, Vol. 8/2, Proceedings of the 5th European Congress for Stereology, Freiburg, September 1989, pp. 473-480. [10] Ch. Lantu6joul and S. Beucher, On the Use of Geodesic Metric in Image Analysis, Journal of Microscopy, Vol. 121, pp. 39-49, 1981. [11] R. Lougheed, Lecture Notes for Summer Course on Mathematical Morphology, University of Michigan, 1983. [12] P. Maragos and R. W. Schafer, Morphological Filters--Part I: Their Set-Theoretic Analysis and Relations to Linear Shift-Invariant Filters", IEEE Transactions on Acoustics, Speech, and Signal Processing, Vol. 35, No. 8, pp. 1153-1169, August 1987. [13] P. Maragos and R. W. Schafer, Morphological Filters--Part II: Their Relations to Median, Order-Statistics, and Stack Filters, IEEE Transactions on Acoustics, Speech, and Signal Processing, Vol. 35, No. 8, pp. 1170-1184, August 1987. [14] G. Matheron, Elkments pour une thkorie des milieux poreux, Masson, Paris, 1967. [15] G. Matheron, Random Sets and Integral Geometry, Wiley, New York, 1975. [16] F. Meyer, Contrast Feature Extraction, in Quantitative Analysis of Microstructures in Material Sciences, Biology and Medicine, J.-L. Chermant, ed. (special issue of Practical Metallography), Riederer Verlag, Stuttgart, 1978. [17] F. Meyer and J. Serra, Contrasts and Activity Lattice, Signal Processing, Vol. 16, No. 4 (special issue on Mathematical Morphology), pp. 303-317, April 1989. [18] F. Meyer and J. Serra, Filters: From Theory to Pratice, Acta Stereologica, Vol. 8/2, Proceedings of the 5th European Congress for Stereology, Freiburg, September 1989, pp. 503-508. [19] K. Preston and M. Duff, Modern Cellular Automata, Plenum, New York, 1985. [20] C. Ronse, Erosion of Narrow Image Features by Combining Local Rank and Max Filters, Proceedings of the 2rid International Conference on Image Processing and Its Applications, London, 1986, pp. 77-81. [21] C. Ronse and H. J. A. M. Heijmans, The Algebraic Basis of Mathematical Morphology. Part II: Openings and Closings, Internal Report, CWI, Amsterdam, 1988, to appear in Computer Vision, Graphics, and Image Processing: Image Understanding. [22] A. Rosenfeld and A. Kak, Digital Image Processing, Academic Press, NeWYork, 1982. [23] J. Serra, Morphologie pour los fonctions "fi peu pr6s en tout ou rien", Internal Report, School of Mines, Paris, 9 pp., April 1975. [24] J. Serra, Image Analysis and Mathematical Morphology, Academic Press, London, 1982. [25] J. Serra (ed.), Image Analysis and Mathematical Morphology, Part H: Theoretical Advances, Academic Press, London, 1988. [26] J. Serra, Iterations et convergence, Internal Report CMM, School of Mines, Paris, 11 pp., February 1989. [27] J. Serra, Toggle Mappings, in From Pixels to Features, J. C. Simon, ed., North-Holland, Amsterdam, 1989, pp. 61-72. [28] J. Serra and L. Vincent, Lecture Notes on Morphological Filtering, Cahiers du Centre de Morphologie Math6matique, No. 8, School of Mines, Paris, 1989. [29] S. R. Sternberg, Parallel Architecture for Image Processing, Proceedings of the 3rd International IEEE Compsac, Chicago, 1979.
108
SERRAAND VINCENT
[30] S. R. Sternberg, Grayscale Morphology, Computer Vision, Graphics, andlmage Processing, Vol. 35, pp. 333-355, 1986. [31] L. Vincent, Mathematical Morphology for Graphs Applied to Image Description and Segmentation, Proceedings: Electronic Imaging West, Pasedena, CA, April 1989, pp. 313-318. [32] L. Vincent, Graphs and Mathematical Morphology, Signal Processing, Vol. 16, No. 4 (special issue on Mathematical Morphology), pp. 365-388, April 1989. [33] L. Vincent, Algorithmes morphologiques it base de files d'attente et de lacets. Extension aux graphes, Ph.D. Thesis and Cahiers du Centre de Morphologie Mathbmatique, No. 9, School of Mines, Paris, 301 pp., May 1990. [34] L. Vincent and P. Soille, Watersheds in Digital Spaces: An Efficient Algorithm Based on Immersion Simulations, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 13, No. 6, pp. 583-598, June 1991. [35] J. Vitria, Estudi Sobre L'Automatitzacio de l'Analisi Visual de Circuits Integrats Mitjan~ant Morfologia Matematica, Ph.D. Thesis, Universitat Autonoma de Barcelona, Barcelona, October 1990. [36] P. D. Wendt, E. J. Coyle, and N. C. Gallagher Jr., Stack Filters, IEEE Transactions on Acoustics, Speech, and Signal Processing, Vol. 34, No. 4, pp. 898-911, August 1986. [37] O. Yli-Harja, J. Astola, and Y. Neuvo, Analysis of the Properties of Median and Weighted Median Filters Using Threshold Logic and Stack Filter Representation, submitted to IEEE Transactions on Acoustics, Speech, and Signal Processing.

Morphological Filtering - An Overview

Hochgeladen von

Dokumentinformationen

Originalbeschreibung:

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Morphological Filtering - An Overview

Hochgeladen von

Copyright:

Verfügbare Formate

CIRCUITS SYSTEMS SIGNAL PROCESS VOL. 11, NO.

AN OVERVIEW OF MORPHOLOGICAL FILTERING*

Abstract. This paper consists of a tutorial overview of morphological filtering, a

1. Mathematical morphology for complete lattices

SERRA AND VINCENT

AN OVERVIEWOF MORPHOLOGICAL FILTERING 49

SERRA AND VINCENT

( ~ Xi) u Y = (-'] (X i u Y).

AN OVERVIEW OF MORPHOLOGICAL FILTERING 51

U - ( f ) = {(x, z) e E x ~, f(x) < z}.

SERRA AND VINCENT

1-1/3 ...... 1d/2 . . . . . . .

fi(x) = 1 - 1/i fi(x) - oo

AN OVERVIEW OF MORPHOLOGICAL FILTERING

As threshold level z increases, Xz(f)= (-]

f(x) = sup{z, x e X~}.

when O < ' x ' < l ,

Figure 5. A function without a maximum (it is not upper semicontinuous).

AN OVERVIEW OF MORPHOLOGICAL FILTERING

only if it can be written as

SERRA AND VINCENT

v x e ~(e), In the case of the dilation 6, we find

A N OVERVIEW OF MORPHOLOGICAL FILTERING

Figure 6. Dilation of a set X by a bipoint/3.

and, by lattice duality,

e(X) = n Xb = X @/~ bet?

SERRA AND VINCENT

a 9eodesic dilation of a set Y inside a set X (see Section 2.4.1).

AN OVERVIEW OF MORPHOLOGICAL FILTERING

(Sf)(x) = sup{g(x - h) + z, z < f(h), h ~ E},

SERRA AND VINCENT

(6f)(x) = sup{g(x - h) + f(h), h ~ E},

Then the definition of the planar mapping ~ leads to

Figure 11. An example of a planar increasing mapping.

AN OVERVIEW OF MORPHOLOGICAL FILTERING 63

SERRA AND VINCENT

0 010 121 010 R 1 ~ 0 1 0 2 1 1 0 =

010 12221 0 2 3 3 3 1 2 3 4 3 0 2 3 3 3 12221 0 1 R~R*

a n d then to c o m p u t e the sup between f l a n d f2 + 1: f 9 C = s u p { A , f2 + 1}.

AN OVERVIEW OF MORPHOLOGICAL FILTERING

2. Openings and closings

or, in terms of operators,

SERRA AND VINCENT

~)m(X) V{B, BeM, B < X}, _

A { B , B ~ ~', B >_ X}.

,".. h", opening by S =

77(X) >- V {7(B), B e 20, B < X} _ = V { B , B e 2 o , B _ X}

AN OVERVIEW OF MORPHOLOGICAL FILTERING

2.3,1. Trivial openings. A criterion T is said to be increasing when, for all

AN OVERVIEWOF MORPHOLOGICAL FILTERING

(dual result with @ = (I v @)"~

AN OVERVIEW OF MORPHOLOGICAL FILTERING 73

Similarly, 6ts")(g)denotes the geodesic dilation of size n of g with respect to f :

SERRA AND VINCENT

Now, the "dual" reconstruction o f f from the marker-function 9 is defined by

p~(g) = lim s~)(g).

7 = V{7b, b e D } = 1 /', {VaB,, b e D }

AN OVERVIEW OF MORPHOLOGICAL FILTERING 75

i.e., since the dilation commutes with v : 7 = I / x 6z~u6, (45)

SERRA AND VINCENT

since the following identities, ( V f0(A)=V(f~(A))

( V ~)o g = V(~o g), ( A ~)o 9 = A(fio g), and