Journal of Neuroscience Methods 177 (2009) 94107

Contents lists available at ScienceDirect

Journal of Neuroscience Methods
j our nal homepage: www. el sevi er . com/ l ocat e/ j neumet h
Integration of confocal and atomic force microscopy images
Shripad Kondra
a
, Jummi Laishram
a
, Jelena Ban
b
, Elisa Migliorini
c
, Valentina Di Foggia
a
,
Marco Lazzarino
c,d
, Vincent Torre
a
, Maria Elisabetta Ruaro
a,
a
International School for Advanced Studies, Area Science Park, Basovizza 34012 Trieste, Italy
b
Glance Vision Technologies, Area Science Park, Basovizza 34012 Trieste, Italy
c
CBM, Area Science Park, Basovizza 34012 Trieste, Italy
d
CNR-INFM Laboratorio TASC, Area Science Park, Basovizza 34012 Trieste, Italy
a r t i c l e i n f o
Article history:
Received 5 August 2008
Received in revised form
15 September 2008
Accepted 26 September 2008
Keywords:
Image registration
Growth cone
Atomic force microscopy
Confocal laser scanning microscopy
a b s t r a c t
Atomic force microscopy (AFM) provides the possibility to map the 3D structure of viewed objects with
a nanometric resolution, which cannot be achieved with other imaging methods such as conventional
video imaging and confocal uorescent microscopy. Video imaging with CCD cameras can provide an
analysis of biological events with a temporal and spatial resolution not possible with AFM, while confocal
imaging allows the simultaneous acquisition of immunouorescence images. In this communication we
present a simple method to combine AFM and confocal images to study differentiating embryonic stem
(ES) cells-derived and dorsal root ganglia (DRG) neurons in culture. Neurons were grown on coverslips
with micrometric markers that allow nding and imaging the same neuron with different microscopes.
AFM and confocal images were registered using conventional methods used in Computer Science. The
combination of these two techniques allows relating functional properties to morphological features of
imaged neurons.
2008 Elsevier B.V. All rights reserved.
1. Introduction
Atomic force microscopy (AFM) provides anunprecedentedway
to image the morphological structure of cells and other biological
samples (Fotiadis et al., 2002; Engel et al., 1999; Engel and Mller,
2000; Kasas et al., 1993). AFMmicroscopy is a scanning methodand
therefore cannot be used to investigate the motion of large biologi-
cal samples at a high temporal resolution, which are better viewed
with conventional video imaging with CCD cameras (Cojoc et al.,
2007; Niell and Smith, 2004; Schaefer et al., 2002) or time-lapse
confocal microscope (Dailey et al., 1999). It is possible to follow
moving biological structures with AFM only when small details
such as lopodia are imaged (McNally et al., 2005) or when high
temporal resolution is not required (Yunxu et al., 2006).
AFM provides a direct measure of the height of samples under
investigation with a resolution of 1 nanometer (nm) (Binnig et al.,
1986) and even less in some special conditions (Lal and John, 1994).
Lateral resolution better than 1nm has been achieved also on bio-
logical samples operating in liquid (Mller et al., 1999; Scheuring et

Corresponding author at: International School for Advanced Studies

(SISSA/ISAS), Ed. Q1AREASCIENCEPARK, S.S.14Km163,5, 34012Basovizza (TS), Italy.
Tel.: +39 040 3756532 fax: +39 040 3756502.
E-mail address: eruaro@sissa.it (M.E. Ruaro).
al., 2003); however, due to enormous complexity of the tip-sample
interaction on biological samples, the minimumresolution achiev-
able can vary signicantly from case to case.
Early investigations of the morphology of neurons with AFM
(Parpura et al., 1993; Bonglio et al., 1995) have allowed a pre-
cise measurement of lopodia height but they have also shown
that AFMimaging is invasive and that the tip of the used cantilever
can modify and distort the membrane of imaged neurons. Indeed
when a high force, in the order of 530nN is used the tip of the
cantilever can damage neurons. This side effect has been utilized
to perform nanosurgery, i.e., controlled lesions with a nanoscale
dimension. By functionalizing the cantilever tip with appropriate
molecules, suchas Nerve GrowthFactor (Reddy et al., 2004) or anti-
bodies (Li et al., 2006), it is possible to localize specic molecules
on the surface of neuron and to obtain a high-resolution map of
their localization.
Confocal laser scanning microscopy (CLSM) utilizes the optical
pathway of a regular optical microscope and is capable of collect-
ing three-dimensional (3D) images. CLSMusing uorescence mode
can collect emissions originating from the interior of a biological
sample and therefore CLSM has been widely used to map intra-
cellular mechanisms. When antibodies are used to stain specic
biological structures, uorescent confocal microscopy provides a
powerful way to simultaneously map the distribution of different
cellular components (Dailey et al., 1999; Rajwa et al., 2004). The
0165-0270/$ see front matter 2008 Elsevier B.V. All rights reserved.
doi:10.1016/j.jneumeth.2008.09.034
S. Kondra et al. / Journal of Neuroscience Methods 177 (2009) 94107 95
200nm resolution limit of the CLSM is restricted by the diffrac-
tion limit of the microscope objective and does not permit accurate
evaluation of the sample height. The z-resolution of CLSM for any
given wavelength is always at least 2 times less than the corre-
sponding xy-resolution (Rajwa et al., 2004). Rajwa et al. (2004) and
McNally et al. (2005) have emphasized that difference in image
collection modes between AFM and CLSM make any image com-
parison a very difcult task and have pointed out the need for an
effective integration of these techniques.
In the present work we provide a simple and efcient way to
integrateconfocal andAFMimagingof xedneurons. Duetothedif-
ferent format of the images acquired by different imaging devices,
it is not obvious how to superimpose them. This problem has been
extensively addressed in Computer Vision particularly in medical
imaging where it is referred to as Registration, and solved at
some extent (Hill et al., 2001; Lester and Arridge, 1999; Maintz and
Viergever, 1998; Van den Elsen et al., 1993). Here we show how to
register AFM and confocal images in a coherent and friendly way.
2. Materials and methods
2.1. Coverslips and printing
Coverslips with printed markers were fabricated, starting from
24mm diameter glass coverslips (Menzel-Glser, Germany), by
optical lithography and metal evaporation. Apolymeric photoresist
was depositedonthe backside of the cover slips byspincoating. The
indexed pattern was produced by UVexposure through a patterned
optical mask. To minimize the toxicity to the cell culture, a 20-nm
titanium layer was nally deposited and stripped by lift off tech-
niques. The glass coverslips were cleaned in acetone and methanol
and then autoclaved in order to remove contamination produced
during the lithographic process. Cells were grown on the opposite
glass side.
2.2. Embryonic stem cell culture
ES cells were induced to differentiate into neurons using the
protocol described in Ban et al. (2007). Undifferentiated mouse
BF1/lacZ ES cells were plated on Mitomycin C (Sigma)-treated
MS5 cells as single-cell suspension at a density of 250cells/cm
2
in Knockout SerumReplacement (KSR, Invitrogen) and cultured for
6 days. The KSR medium was changed every day. The ES-derived
epithelia structures were then mechanically separated from MS5
cells monolayer, by incubating the cells with Hanks balanced salt
solution (HBSS, Sigma) supplemented with 0.15M Hepes (Sigma)
for 5min and ushing through the pipette tip near the colony with-
out removing MS5 feeder. The detached ES cells colonies were
resuspended with KSR medium and plated on 15g/ml polyor-
nithine (Sigma) 1g/ml bronectin (Invitrogen) coated dishes.
After 23h the medium was changed with N2 medium (Invit-
rogen) containing 10ng/ml basic broblast growth factor (bFGF,
R&DSystems) and 1g/ml bronectin (amplication medium) and
cells were induced to proliferate in presence of bFGF for 4 days.
During the last 2 days 200ng/ml of Sonic Hedgehog (SHH, R&DSys-
tems) and100ng/ml broblast growthfactor (FGF) 8(R&DSystems)
were added as patterning factors. Cells were then trypsinized and
plated on 15g/ml polyornithine1g/ml laminin (Sigma)10%
fetal bovineserum(FBS, Sigma) inN2medium-coated24mmdiam-
eter glass coverslips with a printed grid in N2 medium containing
10% FBS at a density of 310
4
cells/cm
2
. 24h after differentiation
induction cells were xed in 4% paraformaldehyde (Sigma) con-
taining 0.15% picric acid (Sigma) in phosphate-buffered saline (PBS,
Invitrogen).
2.3. Dorsal root ganglia isolation and culture
Dorsal root ganglia (DRG) were prepared from P1012 Wis-
tar rats as previously described (Cojoc et al., 2007). Wistar rats
(P1012) were anesthetized with CO
2
and sacriced by decap-
itation in accordance with the European Communities Council
Directive of 24November 1986(86/609/EEC) andthe ItalianAnimal
WelfareAct andapprovedbytheLocal AuthorityVeterinaryService.
DRGs were incubated with 0.5mg/ml trypsin, 1mg/ml collagenase,
and 0.1mg/ml DNase (all from Sigma) in 5ml Neurobasal medium
(Invitrogen) in a shaking bath at 37

C for 3540min. They were

mechanically dissociated, centrifuged at 300rpm, resuspended in
culture medium, and plated on0.5g/ml poly-l-lysine (Sigma) and
Matrigel (BD Biosciences, Bedford, MA, USA)-coated 24mm cover-
slips in Neurobasal medium containing 10% fetal FBS (Sigma) and
100ng/ml nerve growth factor (NGF, Alomone, Israel) at a density
of 2.510
5
cells/cm
2
. 2048h after differentiation induction cells
were xed in 4% paraformaldehyde containing 0.15% picric acid in
phosphate-buffered saline (PBS).
2.4. Confocal imaging
After xation cells were saturated with 0.1M glycine, perme-
abilized with 0.1% triton X-100, saturated with 0.5% BSA (all from
Sigma) in PBS and then incubated for 1h at room temperature
(2022

C) with primary antibodies: rabbit polyclonal antibody

against neural cell adhesion molecule (NCAM, Sigma) and mouse
monoclonal antibody-TUJ1 (Covance) against neuronal class III -
tubulin. The secondary antibodies were goat anti-mouse-594 Alexa
and anti-rabbit-594 Alexa (Invitrogen). Alexa Fluor 488 phalloidin,
used to mark F-actin, was fromInvitrogen. The incubation time was
30min at room temperature (2022

C). Total nuclei were stained

with 2g/ml in PBS Hoechst 33342 (Sigma). The cells were exam-
ined using a confocal (Leica DMIRE2) microscope equipped with
differential interference contrast (DIC) and uorescence optics.
The uorescence images were collected with a 63 magnication
and 1.4NA oil-immersion objective (ES-derived neurons) or with a
40 magnication and 1.25NA objective (DRG neurons). Confocal
images had 10241024 pixels with an intensity varying from 0 to
255 coded in 8 bits.
2.5. AFM imaging
Atomic force microscopy was performed using the Nanowiz-
ard II instrument (JPK, Berlin) combined with an inverted optical
microscope (Zeiss Axiovert 200), and a uorescence set-up (Zeiss
X-cite). AFM was operated in contact mode in liquid, adjusting
the contact force during imaging to compensate the thermal and
chemical bending of the cantilever and thus to minimize the force
exerted by the tip on the sample during scanning. Gold coated,
soft V-shaped SiliconNitrite cantilever (VEECO) withnominal force
constant of 0.01N/m and a resonance frequency in the range of
410kHz were used and forces were kept between 100pN and
1nN during scanning. To perform AFM analysis, coverslips were
mounted into the AFM liquid cell, immersed in 2ml PBS con-
taining 2g/ml gentamycin to avoid bacterial contamination, and
positioned the sample xy stage. 40 magnication with 0.6NA
objective was used to nd specic neurons/growth cones using the
markers as a reference of position. After laser alignment and tip
calibration, the system was left to settle for a half an hour with
laser and microscope condenser on, until thermal and force drift
reached a steady state before starting image acquisition. The scan
speed of the tip was maintained below 5m/s to minimize the
tip-induced damage and keep the gain as low as possible. Digi-
tal AFM images were composed of 512512 pixels or 10241024
96 S. Kondra et al. / Journal of Neuroscience Methods 177 (2009) 94107
pixels. Post processing was performed using JPK Image Processing
(Berlin) or, to obtain detailed data analysis, using SPIP Scanning
Probe Imaging Processor Image post processing (Image Metrol-
ogy A/S Lyngs Alle 3A DK-2970 Hrsholm, Denmark); in any case
the post processing was minimized so to reduce image processing
artifacts.
3. Results
Neurons were grown on coverslips with printed markers, such
as those shown in Fig. 1A. The free space between printed mark-
ers is between 150 and 400m and in this space neurons and
their growth cones could be clearly seen to grow and differenti-
ate (Fig. 1B). Printed numbers allowed assigning coordinates to a
chosen neuron and to relocate it after xation. In this way, it was
possible to image the same growth cone both with Confocal Laser
Scanning Microscope (performing animmunouorescence assayas
shown in Fig. 1C) and Atomic Force Microscope (Fig. 1D) separately
and at different times.
3.1. Integration of imaging modalities
AFM and CLSM images contain different information: every
pixel at location (x,y)
AFM
in the AFMimages provides a direct mea-
surement in nm of the sample height while uorescence images
acquired with a confocal laser scanning microscope (CLSM) char-
acterize the emitted uorescence at location (x,y)
CLSM
. In order
to integrate these information it is important to properly align
or superimpose these different images. This problem has been
extensivelystudiedinComputer Vision, whereit is referredas Reg-
istration of different images, and it will be addressed in the next
section.
3.2. Registration of confocal and AFM image
Image registration is the process of overlaying two or more
images of the same scene taken at different times, different view-
points, and/or bydifferent imagingmodalities. Differences between
images are introduced because of different imaging conditions,
suchas different viewingpoints or light conditions or becauseof the
useof twodifferent microscopes, as inthecaseunder consideration.
Registration transforms one image usually the sensed image so
that it becomes aligned to the reference image. Image registration
is a crucial step in image analysis and can be solved using meth-
ods used in computer vision (Trucco and Verri, 1998), in which the
nal information is obtained by integrating various data sources
like in image fusion (Zitova and Flusser, 2003). Images taken from
different modalities may undergo a linear deformation in scale,
translation, or rotation, and non-linear deformation, i.e., shearing.
As linear deformation is a special case of afne, the transformation
is in general non-linear.
Fig. 1. (A) An example of a coverslip with printed markers and numbers with a spacing varying from150 to 400m(see Section 2). Scale bar =100m. (B) ES-derived growth
cone (inset) imaged with DIC using a confocal microscope. Scale bar =25m. (C) Confocal image (10241024 pixels) of a growth cone stained with an antibody against
NCAM. Scale bar =10m. (D) AFM image (10241024 pixels) of the same growth cone. Scale bar =5m. Color bar from 0 to 350nm.
S. Kondra et al. / Journal of Neuroscience Methods 177 (2009) 94107 97
The word registration is used with two different meanings (Hill
et al., 2001). The rst meaning is to determine a transformation
of one image so that features in the sensed image can be put in
a one-to-one correspondence to features in the reference image.
The symbol T is used to represent this type of transformation.
The second meaning of registration enables also the comparison
of the intensity at corresponding positions. The symbol T
i
is used
to describe this second meaning of registration, whichincorporates
the concepts of re-sampling and interpolation. Using the language
of geometry, the registration transformation is referred to as the
mapping T, transforming a position (x,y)
A
in the system of refer-
ence of image A to the position (x,y)
B
in the system of reference of
image B.
T : (x, y)
A
(x, y)
B
T(x, y)
A
= (x, y)
B
(1)
Using this notation, T is a spatial mapping. The more complete
mapping T
i
maps both position and associated intensity value from
image A to image B. Therefore T
i
maps an image to an image,
whereas T maps between coordinates of image A into coordinates
of image B. To overlay two images that have been registered, or
to subtract one from another, is necessary to know T
i
, not just T.
T
i
is only dened in the region of overlap of the image elds of
view, and has to take into account of image sampling and spatial
resolution.
Before doing the registration, it is necessary to select the correct
type of transformation required for the images under considera-
tion. The most widely used transformations are Linear, Afne and
Projective (for a more detailed mathematical description of the dif-
ferent transformations see Appendix A). The linear transformation
is used when shapes in the sensed image are unchanged, but are
distorted by some combination of translation, rotation, and scal-
ing (Fig. 2). Rectangles remain rectangles. This is the case of the
same biological sample viewed by two microscopes using differ-
ent objectives and imaging systems, such as an AFM (Fig. 1D) and
a CLSM (Fig. 1C). Registration using afne transformation is neces-
sarywhenshapes inthe sensedimage are distortedalsobyshearing
or a linear deformation (Fig. 2). In this case, straight lines remain
straight, and parallel lines remain parallel, but rectangles become
Fig. 2. Examples of 2D transformations.
parallelograms. Projective transformation is used when the scene
appears tilted (Fig. 2). Straight lines remain straight, but parallel
lines converge toward vanishing points that might or might not fall
within the image.
Let us nowconsider two images of the same biological sample S,
the rst one acquired using AFM (I
AFM
(x

,y

)) and the second using

a CLSM (I
CLSM
(x,y)). The two imaging systems differ because of dif-
ferent scale factors, by a rigid translation of their origin (t
x
,t
y
) and
a possible rotation of their axis. It is assumed that the relation
between the images is afne so that the transformation between
the two systems of coordinates (x,y) and (x

,y

) can be solved using

Eq. (10) in Appendix A. Registration of the two I
AFM
(x

,y

) and
I
CLSM
(x,y) consists inthe determinationof the transformationT (see
Eq. (1)) by aligning properly two images such that I
AFM
(T(xi

,yi

))
and I
CLSM
(xi,yi) is the AFM and CLSM of the same physical
point.
The unknown parameters in the transformation matrix T can
be estimated either by matching a selected number of points or
landmarks in the two images (Section 3.2.1) or by matching entire
contours in the two images (Section 3.2.2). The rst method is used
when it is possible to identify in the two images I
AFM
(x

,y

) and
I
CLSM
(x,y) enough points corresponding to the same physical struc-
ture, such as the tip of a dendrite, or a small vesicle. The second
method is used in presence of rounded biological structures, with
no obvious marks and it is more convenient to put in correspon-
dence two contours instead of isolated points.
To apply these methods to the images derived from different
microscopes we have implemented our Registration programusing
Matlab7.1 (The MathWorks Inc. http://www.mathworks.com) a
standardtool commoninthe computer visionandbiomedical com-
munity for statistical analysis and production of gures and images
of analyzed data and results.
A detailed description of the programis shown in Appendices B
and C. Corresponding points or contours in I
AFM
(x

,y

) and I
CLSM
(x,y)
were marked by hand using a friendly graphical user interface
developed for this particular purpose (Fig. 3).
3.2.1. Registration by points selection method
This method is illustrated in Fig. 4. The operator identies N
(minimum 3) points in the I
AFM
(x

,y

) which are put in correspon-

dence with N points in the I
CLSM
(x,y). Corresponding points in
Fig. 4A and B are indicated by the same number. By using these cor-
respondences the parameters determining the transformationT are
obtained by solving a system of linear equation and the I
CLSM
(x,y)
can be registered. In this way same physical points in I
AFM
(x

,y

) and
in the new registered image I*
CLSM
(x,y), shown in Fig. 4C, have the
same location (x,y).
To make the selection of points more user friendly, I
CLSM
(x,y)
was rotated in a sequence of 90

and/or ipped to make an approx-

imate alignment with I
AFM
(x

,y

) (compare Fig. 1C with Fig. 4B).

Control Point Selection Tool in Matlab was used to mark the con-
trol point pairs in the image to be registered, the input image
(Fig. 4B), and the image to which you are comparing it, the
base image (Fig. 4A) corresponding points were initially speci-
ed by pointing and clicking in the input and base images so that
each point specied in the input image had a match in the base
image.
The advantage of this tool is that, after the rst three pairs are
selected, it is sufcient to select a point in the input image and the
Control Point Selection Tool estimates its match point in the base
image or vice versa automatically, based on the geometric relation-
ship of the previously selected control points. Another advantage of
this tool is that if the image is not registered properly, it is possible
to change the position of control points to get the exact superim-
position.
98 S. Kondra et al. / Journal of Neuroscience Methods 177 (2009) 94107
Fig. 3. Snapshot of the graphical user interface.
3.2.2. Registration by aligning contours with Procrustes analysis
Procrustes was a robber in Greek mythology. He would offer
travelers hospitality in his road-side house, and the opportunity to
stay the night in his bed that would perfectly t each visitor. As
the visitors discovered to their cost, however, it was the guest who
was altered to t the bed, rather than the bed that was altered to
t the guest (Hill et al., 2001). To make the idea more clear let us
consider that the AFMimage, I
AFM
(x

,y

) is like the bedandI

CLSM
(x,y)
is the guest. So using the Procrustes method means that I
CLSM
(x,y)
is altered by scaling and/or rotating to approximately t I
AFM
(x

,y

).
The number of landmarks available depends on the structure of
the image, and there may be differences in opinion between scien-
tists as to which landmarks are consistent and locatable. Marking
of the corresponding points thus becomes difcult when there are
few corners in the structure, and thus exact location of the points
is impossible and can differ from observer to observer. This prob-
lem can be solved by marking many points along the contour of
the structure in both images. The contours are then interpolated
so that both of them contain the same number of equally spaced
points.
Fig. 4. Example of registering images using points selection method. (A) AFM image of ES-derived growth cone. Color bar from 0 to 350nm. (B) Confocal image of the same
ES-derived growth cone. Scale bar =10m. (C) Registered confocal image.
S. Kondra et al. / Journal of Neuroscience Methods 177 (2009) 94107 99
Fig. 5. Example of registering images using Procrustes analysis. (A) AFMimage of ES-derived growth cone. Color bar from0 to 2200nm. Scale bar =10m. (B) Confocal image
of the same ES-derived growth cone. Scale bar =10m. (C) Registered confocal image of membrane protein NCAM.
Given two contours S
1
and S
2
with the same number of points
N, the Procrustes analysis nds the best transformation T of one
contour, i.e., S

2
= T(S
2
) in such a way that the distance between the
two contours S
1
and S

2
is minimal (Cootes et al., 1995). The pro-
cess involves least square matching of two congurations using the
similarity transformations. The parameters are estimated using the
squared Euclidean distance. The least square distance or L2 norm
is represented using double vertical bars.
_
_
S

2
S
1
_
_
2
(2)
By restricting transformation to the similarity case, we
have:
T
_
x
y
1
_
=
_
a b tx
b a ty
0 0 1
__
x
y
1
_
(3)
If the two shapes have the same center of gravity located in the
origin, the optimal transformation is:
((t
x
, t
y
) = (0, 0)),
a =
_
n
i=1
(x
i
x
t
i
+ y
i
y
t
i
)
_
_
_
S
1
_
_
2
,
b =
_
n
i=1
(x
i
y
t
i
y
i
x
t
i
)
_
_
_
S
1
_
_
2
(4)
Fig. 6. Registered Confocal images overlayed with the corresponding AFM image. Points selection method (indicated with white dots) was used in rows A and B while
Procrustes analysis was applied for images in rows C and D (indicated with white lines). (Column 1) AFMimages of ES-derived (A, C, D) and DRG neuron (B). Scale bar =5m.
(Column 2) Confocal Images of the same neurons in column 1. (Column 3) 3D view of the overlayed images. Color bar represents values in nm. (Column 4) Confocal images
texture mapped on the 3D AFM images.
100 S. Kondra et al. / Journal of Neuroscience Methods 177 (2009) 94107
Fig. 7. Confocal images edges superimposed with AFM images. (Row 1) ES-derived neuron. (Row 2) DRG neuron. A1A2: AFM images. Scale bar =5m. Color bar represents
values in nm. B1B2: Confocal images. Scale bar =5m. C1C2: Confocal images registered with respect to AFM images. D1D2: Edge detected on C1 and C2 images. E1E2:
Superimposition of A and D images.
where S
1
={(x
i
,y
i
); i =1. . .n}, S
2
= {(x
t
i
, y
t
i
); i = 1...n} and ||S
1
|| is the
Euclidean norm.
Scale =
_
a
2
+ b
2
(5)
(t
x
, t
y
) = (xi, yi) (6)
where xi, yi are mean of the x and y coordinates of the points of
contour S
1
.
Here the translation was removed only for the second contour.
The rst contour S
1
was xed like the bed of Procrustes. Since it
was not possible to obtain the precise correspondence for the two
point-sets, it was necessary to consider N possible cyclic renum-
bering iteratively for each shape, so that the optimal alignment was
obtained when the full Procrustes distance (Eq. (2)) was minimal.
We use Procrustes analysis only with rotation and translation and
not Afne. The scale can be recovered using Eq. (5).
An example of this method is illustrated in Fig. 5. A contour was
markedby continuously clicking onpoints following the borderline
of the structure of interest in I
AFM
(Fig. 5A) and in I
CLSM
as well
(Fig. 5B). As a consequence of Procrustes analysis, I
CLSM
(x,y) was
properly transformed and put in correspondence with I
AFM
(x

,y

),
as shown in Fig. 5C.
3.3. Examples of combined images
When I
AFM
(x

,y

) and I
*
CLSM
(x,y) have been appropriately regis-
tered, they can be superimposed to localize the labeled proteins
with respect to cell topography. In the following examples when
many prominent corners were present in the growth cones, the
CLSM images were registered by points selection method (Fig. 6A
and B). In such cases Procrustes analysis was not used because
marking a contour was very laborious. This method was applied
in cases where the growth cone had a smoother contour (Fig. 6C
and D). Superimposition can be achieved in different ways. Reg-
istered Confocal images can be overlayed with the corresponding
AFM image as shown in Fig. 6.
Image overlay is a promising data visualization technique with
a wide variety of medical applications. The 3D image of AFM was
rst plotted and then the confocal image was texture mapped on
a at surface below it as shown in Fig. 6 column 3. An alternative
way to visualize the information from these two types of images is
to texture map the confocal image on the 3D AFM image as shown
in Fig. 6 column 4. In both cases the 3Dimage can be rotated and/or
zoomedtoobtaina better viewof the structure (See supplementary
materials).
The 3D overlay gives an indicative correspondence of the pres-
ence of sub-cellular components marked with uorescence with
topographical information fromAFM. This way one can get a quick,
accurate and detailed understanding of the structure before doing
any further analysis such as calculating average height of a partic-
ularly stained protein.
Another way to combine AFM and CLSM images is to super-
impose edges detected from CLSM images on AFM images. Fig. 7
shows images registered using the points selection method. In
the CLSM images, growth cones from ES-derived neuron (row
1) were stained for neural cell adhesion molecule (NCAM) while
DRG (row 2) neurons were double-stained for F-actin (green)
and -tubulin III (red). The edges were then detected using the
method described in Pellegrino et al. (2004). In the case of DRG,
the edges were found separately from the two channels (green
and red), and an OR operation was performed on the resultant
edges. The result was then superimposed onto the AFMtopography
image.
In this way, one can obtain the precise correspondence between
cellular components and topography of the biological sample being
studied, as shown in row 2 of Fig. 7D, where tubulin localizes in
correspondence with high surface structures measured by AFM.
4. Discussion
Previous reports using AFM and confocal microscopy (McNally
et al., 2005; Rajwa et al., 2004) have emphasized the need for an
effective integration of the information provided by these tech-
niques. In the present manuscript we provided a suitable method
for solving this requirement. Taking advantage of tools used in
Computer Vision we were able to superimpose images with dif-
ferent format and resolution. To our knowledge this is the rst
report where the images of the same cell obtained with different
microscopes separately could be directly compared and superim-
posed. By using home-made coverslips with printed markers we
were able to observe the growth cones of ES-derived and DRG
neurons after xation with confocal laser scanning microscopy
and to retrieve them for further imaging with AFM. The con-
focal images with uorescent staining of specic markers (such
as neuronal class III -tubulin, neural cell adhesion molecule
(NCAM) or F-actin, a principal component of neuronal growth
cones) and the AFM images of the 3D structure (topography) of
the same growth cone were obtained separately by the use of
different instruments. The information obtained by the two tech-
niques could be then fused together by the use of the method here
described.
Point based registration is a suitable method for accurate
matching of corresponding structures especially when the sample
S. Kondra et al. / Journal of Neuroscience Methods 177 (2009) 94107 101
contains many corners and projections. However, further improve-
ment can still be done to automatically detect corners, to make
it easier for the user to select the candidates for correspondence.
Matching with the contour is the selected method when there are
no prominent corners. However it is not possible to use this method
when the objects have steep walls or undercut regions. AFM scan-
ning of these objects appear to have ared sides in the images
(compare structures indicated by arrows in Fig. 5A and C) due to
the fact that AFM images are always a convolution of the original
structure and the AFM tip shape.
Given the higher vertical resolution of AFM, novel struc-
tural components were discovered (McNally et al., 2005) while
confocal microscopy was widely used in neuroscience (for a
review see Niell and Smith, 2004). The possibility to com-
bine the images from these two different analyses will increase
the possibility to associate structure with the presence of sub-
cellular components. Moreover the possibility to monitor cell
behavior by time-lapse confocal laser scanning microscopy will
allow to link movement with structural changes (AFM) and
the expression of specic sub-cellular components (confocal
microscope).
The registration method here described is versatile and not
limited to just AFM and CLSM images. Images taken from other
modalities like CCDor SEMcould be registered in a similar manner
to get more understanding of the specimen under consideration.
In future we plan to register the confocal layers taken at different
depth with the 3D AFM topography to do in-depth analysis of the
structure.
Acknowledgements
This work was supported by the EU projects: NEURO Contract
n. 012788 (FP6-STREP, NEST) and NanoScale Contract n.214566
(FP7-NMP-2007-SMALL-1). In addition we need to acknowledge
the nancial support of a FIRB grant D.M.31/03/05 from the Ital-
ian Government, Contr. RICNno.011936 BINASP fromthe European
Community, funds fromthe IstitutoItalianodi Tecnologia (Research
Unit IIT) and of the GRAND Grant from CIPE/FVG by the Friuli
Venezia Giulia region. SKs work was supported by EU Marie
Curie Training Network: Vision train Contract n. MRTN-CT-2004-
005439).
Appendix A. Transformation matrices of the different
methods
If (x,y) is the systemof reference of image Aand(x

,y

) the system
of reference of image B. The transformation T is dened as:
_
a1 a2 t
x
a3 a4 t
y
c1 c2 1
_
(7)
where

_
a1 a2
a3 a4
_
is a rotation matrix. This matrix denes the kind of the
transformation that will be performed: scaling, rotation, and so
on.

_
t
x
t
y
_
is the translation vector. It simply moves the points.

_
c1 c2

is the projection vector.

If x and y are the coordinates of a point, the transformation can
be done by the simple multiplication:
_
a1 a2 t
x
a3 a4 t
y
c1 c2 1
_

_
x
y
1
_
=
_
x

1
_
(8)
Here, x

and y

are the coordinates of the transformed points.

For Linear transformation

c1 and c2 are equal to 0.

_
a1 a2
a3 a4
_
can be written as
_
sc ss
ss sc
_
where sc =scale cos(),
ss =scale sin(), and is the rotation.
Thus the transformation matrix becomes
T =
_
sc ss t
x
ss sc t
y
0 0 1
_
(9)
At least two pairs of corresponding points are needed to solve
for the four unknown coefcients (sc, ss, t
x
, t
y
).
For Afne transformations

c1 and c2 are equal to 0.

Thus the transformation matrix becomes
T =
_
a1 a2 t
x
a3 a4 t
y
0 0 1
_
(10)
At least three pairs of corresponding points are needed to solve
for the six unknown coefcients.
For projective transformations
The transformation matrix is
T =
_
a1 a2 t
x
a3 a4 t
y
c1 c2 1
_
(11)
At least four pairs of corresponding points are needed to solve
for the eight unknown coefcients.
102 S. Kondra et al. / Journal of Neuroscience Methods 177 (2009) 94107
Appendix B. Main GUI for performing Registration
S. Kondra et al. / Journal of Neuroscience Methods 177 (2009) 94107 103
Appendix B CONTINUED
104 S. Kondra et al. / Journal of Neuroscience Methods 177 (2009) 94107
S. Kondra et al. / Journal of Neuroscience Methods 177 (2009) 94107 105
Appendix B CONTINUED
106 S. Kondra et al. / Journal of Neuroscience Methods 177 (2009) 94107
Appendix C. Matlab Script for Procrustes Analysis
S. Kondra et al. / Journal of Neuroscience Methods 177 (2009) 94107 107
Appendix C CONTINUED
Appendix D. Supplementary data
Supplementary data associated with this article can be found,
in the online version, at doi:10.1016/j.jneumeth.2008.09.034.
References
Ban J, Bonifazi P, Pinato G, Broccard FD, Studer L, Torre V, et al. Embryonic stemcell-
derived neurons form functional networks in vitro. Stem Cells 2007;25(March
(3)):73849.
Binnig G, Quate CF, Gerber C. Atomic force microscope. Phys Rev Lett 1986;56(March
(9)):9303.
Bonglio A, Parodi MT, Tonini GP. Subcellular details of early events of differentiation
induced by retinoic acid in human neuroblastoma cells detected by atomic force
microscope. Exp Cell Res 1995;216(January (1)):739.
Cojoc D, Difato F, Ferrari E, Shahapure RB, LaishramJ, Righi M, et al. Properties of the
force exerted by lopodia and lamellipodia and the involvement of cytoskeletal
components. PLoS ONE 2007;2(October (10)):e1072.
Cootes TF, Taylor CJ, Cooper DH, Graham J. Active shape modelstheir training and
application. Comput Vis Imag Underst 1995;61(1):3859.
Dailey M, Marrs G, Satz J, Waite M. Concepts in imaging and microscopy. Explor-
ing biological structure and function with confocal microscopy. Biol Bull
1999;197(October (2)):11522.
Engel A, LyubchenkoY, Mller D. Atomic forcemicroscopy: apowerful tool toobserve
biomolecules at work. Trends Cell Biol 1999;9(February (2)):7780.
Engel A, Mller DJ. Observing single biomolecules at work with the atomic force
microscope. Nat Struct Biol 2000;7(September (9)):7158.
Fotiadis D, Scheuring S, Mller SA, Engel A, Mller DJ. Imaging and manipulation of
biological structures with the AFM. Micron 2002;33(4):38597.
Hill DL, Batchelor PG, Holden M, Hawkes DJ. Medical image registration. Phys Med
Biol 2001;46(March (3)):R145.
Kasas S, Gotzos V, Celio MR. Observation of living cells using the atomic force micro-
scope. Biophys J 1993;64(February (2)):53944.
Lal R, John SA. Biological applications of atomic force microscopy. Am J Physiol
1994;266(January (1 Pt 1)):C121.
Lester H, Arridge SR. A survey of hierarchical non-linear medical image registration.
Pattern Recog 1999;32:12949.
Li G, Xi N, Wang DH. Probing membrane proteins using atomic force microscopy. J
Cell Biochem 2006;97(April (6)):11917.
Maintz JB, Viergever MA. A survey of medical image registration. Med Image Anal
1998;2(March (1)):136.
McNally HA, Rajwa B, Sturgis J, RobinsonJP. Comparative three-dimensional imaging
of living neurons withconfocal andatomic force microscopy. J Neurosci Methods
2005;142(March (2)):17784.
Mller DJ, Fotiadis D, Scheuring S, Mller SA, Engel A. Electrostatically balanced sub-
nanometer imagingof biological specimens byatomic forcemicroscope. Biophys
J 1999;76(February (2)):110111.
Niell CM, Smith SJ. Live optical imaging of nervous system development. Annu Rev
Physiol 2004;66:77198.
Parpura V, Haydon PG, Henderson E. Three-dimensional imaging of living neurons
and glia with the atomic force microscope. J Cell Sci 1993;104(February (Pt
2)):42732.
Pellegrino FA, Vanzella W, Torre V. Edge detection revisited. IEEE Trans Syst Man
Cybern B Cybern 2004;34(June (3)):150018.
Rajwa B, McNally HA, Varadharajan P, Sturgis J, Robinson JP. AFM/CLSM data
visualization and comparison using an open-source toolkit. Microsc Res Tech
2004;64(June (2)):17684.
Reddy CV, Malinowska K, Menhart N, Wang R. Identication of TrkA on living PC12
cells by atomic force microscopy. Biochim Biophys Acta 2004;1667(November
(1)):1525.
Schaefer AW, Kabir N, Forscher P. Filopodia actin arcs guide the assembly and trans-
port of two populations of microtubules with unique dynamic parameters in
neuronal growth cones. J Cell Biol 2002;158(July (1)):13952.
Scheuring S, Seguin J, Marco S, Lvy D, Robert B, Rigaud JL. Nanodissection and
high-resolution imaging of the Rhodopseudomonas viridis photosynthetic core
complex in native membranes by AFM. Proc Natl Acad Sci USA 2003;100(Febru-
ary (4)):16903.
Trucco E, Verri A. Introductory techniques for 3-d computer vision. Prentice Hall;
1998.
Van den Elsen PA, Pol EJD, Viergever MA. Medical image matchinga review with
classication. IEEE Eng Med Biol 1993;12:2639.
Yunxu S, Danying L, Yanfang R, Dong H, Wanyun M. Three-dimensional structural
changes in living hippocampal neurons imaged using magnetic AC mode atomic
force microscopy. J Electron Microsc (Tokyo) 2006;55(June (3)):16572.
Zitova B, Flusser J. Image registration methods: a survey. Image Vision Comput
2003;21(October (11)):9771000.