Sie sind auf Seite 1von 9

Epipolar Re

ti ation
Andrea Fusiello
Dipartimento S ienti o e Te nologi o, Universita di Verona
Ca' Vignal 2, Strada Le Grazie, I-37134 Verona, IT
Tel: +39 045 802 7088 Fax: +39 045 802 7928
E-mail: fusiellos i.univr.it
http://www.s i.univr.it/~fusiello
Mar h 17, 2000

1 Introdu tion
Given a pair of stereo images, epipolar re ti ation (or simply re ti ation) determines a transformation of
ea h image plane su h that pairs of onjugate epipolar lines be ome ollinear and parallel to one of the image
axes (usually the horizontal one). The re ti ed images an be thought of as a quired by a new stereo rig,
obtained by rotating the original ameras. The important advantage of re ti ation is that omputing stereo
orresponden es [3 is made simpler, be ause sear h is done along the horizontal lines of the re ti ed images.
We assume that the stereo rig is alibrated, i.e., the ameras' internal parameters, mutual position and
orientation are known. This assumption is not stri tly ne essary, but leads to a simpler te hnique. On the
other hand, when re onstru ting 3-D shape of obje ts from stereo, alibration is mandatory in pra ti e, and
an be easily a hieved [2, 13.
Re ti ation is a lassi al problem of stereo vision. Aya he [1 introdu ed a re ti ation algorithm, in
whi h a matrix satisfying a number of onstraints is hand- rafted. The distin tion between ne essary and
arbitrary onstraints is un lear. Some authors report re ti ation under restri tive assumptions; for instan e,
[11 assumes a very restri tive geometry (parallel verti al axes of the amera referen e frames). Re ently,
[6, 14, 8 have introdu ed algorithms whi h perform re ti ation given a weakly alibrated stereo rig, i.e., a rig
for whi h only points orresponden es between images are given (or, equivalently, for whi h the fundamental
matrix ould be omputed).
Latest works in ludes [10, 9, 12. Some of them on entrates on the issue of minimizing the re ti ed
image distortion. We will not address this problem, also be ause distortion is less severe than in the weakly
alibrated ase.
This do ument have been adapted from [5.

2 Camera model and epipolar geometry


This se tion re alls brie y the mathemati al ba kground on perspe tive proje tions ne essary for our purposes. For more details see [4, 15.
2.1

Camera model

A pinhole amera is modeled by its opti al enter C and its retinal plane (or image plane) R. A 3-D point
W is proje ted into an image point M given by the interse tion of R with the line ontaining C and W. The
line ontaining C and orthogonal to R is alled the opti al axis and its interse tion with R is the prin ipal
point. The distan e between C and R is the fo al length.
Let w = [x y z > be the oordinates of W in the world referen e frame ( xed arbitrarily) and m = [u v >
the oordinates of M in the image plane (pixels). The mapping from 3-D oordinates to 2-D oordinates
1

is the perspe tive proje tion, whi h is represented by a linear transformation in homogeneous oordinates.
~ = [u v 1> and w
~ = [x y z 1> be the homogeneous oordinates of M and W respe tively; then the
Let m
~:
perspe tive transformation is given by the matrix P

m~ ' P~ w~ ;

(1)

where ' means equal up to an arbitrary s ale fa tor. The amera is therefore modeled by its perspe tive
~ , whi h an be de omposed, using the QR fa torization, into the
proje tion matrix (hen eforth PPM) P
produ t

P~ = A[R j t:

(2)

2
3
u u0
A = 4 0 v v0 5 ;

(3)

The matrix A depends on the intrinsi parameters only, and has the following form:
0

where u = fku , v = fkv are the fo al lengths in horizontal and verti al pixels, respe tively (f is the
fo al length in millimeters, ku and kv are the e e tive number of pixels per millimeter along the u and v
axes), (u0 ; v0 ) are the oordinates of the prin ipal point, given by the interse tion of the opti al axis with
the retinal plane, and is the skew fa tor that models non-orthogonal u v axes..
The amera position and orientation (extrinsi parameters), are en oded by the 3  3 rotation matrix R
and the translation ve tor t, representing the rigid transformation that brings the amera referen e frame
onto the world referen e frame.
Let us write the PPM as

2
P~ = 4

q>1>
q2>
q3

q14
q24 5 = [Qjq~ :
q34

(4)

In Cartesian oordinates, the proje tion (1) writes

8
q>1 w + q14
>
>
>
u
=
< q>3 w + q34
>
>
>
>
: v = q2 w + q24 :

(5)

q>3 w + q34

The fo al plane is the plane parallel to the retinal plane that ontains the opti al enter C. The oordinates

of C are given by

~ an be written:
Therefore P

= Q 1q~:

(6)

P~ = [Qj Q :

(7)

~w
~ 'P
~ g.
The opti al ray asso iated to an image point M is the line M C, i.e. the set of 3-D points fw : m
Its parametri equation in Cartesian oordinates writes:

w = + Q 1m~ ;
2.2

 2 R:

(8)

Epipolar geometry

Let us onsider a stereo rig omposed by two pinhole ameras (Fig. 1). Let C1 and C2 be the opti al enters
of the left and right ameras respe tively. A 3-D point W is proje ted onto both image planes, to points M1
2

C1

M1
W

E1

M2

R1
E2
C2

R2

Figure 1: Epipolar geometry.


and M2 , whi h onstitute a onjugate pair. Given a point M1 in the left image plane, its onjugate point in
the right image is onstrained to lie on a line alled the epipolar line (of M1 ). Sin e M1 may be the proje tion
of an arbitrary point on its opti al ray, the epipolar line is the proje tion through C2 of the opti al ray of
M1 . All the epipolar lines in one image plane pass through a ommon point (E1 and E2 respe tively) alled
the epipole, whi h is the proje tion of the opti al enter of the other amera.
When C1 is in the fo al plane of the right amera, the right epipole is at in nity, and the epipolar lines
form a bundle of parallel lines in the right image. A very spe ial ase is when both epipoles are at in nity,
that happens when the line C1 C2 (the baseline) is ontained in both fo al planes, i.e., the retinal planes are
parallel to the baseline. Epipolar lines, then, form a bundle of parallel lines in both images. Any pair of
images an be transformed so that epipolar lines are parallel and horizontal in ea h image. This pro edure
is alled re ti ation.

3 Re ti ation of amera matri es

~ o1 and P
~ o2 are known. The idea behind
We assume that the stereo rig is alibrated, i.e., the PPMs P
~ n1 and P
~ n2 obtained by rotating the old ones around their opti al
re ti ation is to de ne two new PPMs P
enters until fo al planes be omes oplanar, thereby ontaining the baseline. This ensures that epipoles are
at in nity, hen e epipolar lines are parallel. To have horizontal epipolar lines, the baseline must be parallel
to the new X axis of both ameras. In addition, to have a proper re ti ation, onjugate points must have
the same verti al oordinate. This is obtained by requiring that the new ameras have the same intrinsi
parameters. Note that, being the fo al length the same, retinal planes are oplanar too, as in Figure 2.
In summary: positions (i.e, opti al enters) of the new PPMs are the same as the old ameras, whereas
the new orientation (the same for both ameras) di ers from the old ones by suitable rotations; intrinsi
parameters are the same for both ameras. Therefore, the two resulting PPMs will di er only in their opti al
enters, and they an be thought as a single amera translated along the X axis of its referen e system.

M1
C1

R1

M2

C2

R2

Figure 2: Re ti ed ameras. Retinal planes are oplanar and parallel to the baseline.
Let us write the new PPMs in terms of their fa torization. From (2) and (7):

P~ n1 = A[R j R 1 ; P~ n2 = A[R j R 2:
(9)
The intrinsi parameters matrix A is the same for both PPMs, and an be hosen arbitrarily (see matlab
ode). The opti al enters 1 and 2 are given by the old opti al enters, omputed with (6). The matrix R,
whi h gives the amera's pose, is the same for both PPMs. It will be spe i ed by means of its row ve tors

2 >3
r1
R = 4r>2 5

r>3

(10)

that are the X, Y, and Z axes, respe tively, of the amera referen e frame, expressed in world oordinates.
A ording to the previous omments, we take:

r1 = ( 1 2)=jj 1 2 jj
2. The new Y axis orthogonal to X (mandatory) and to k: r2 = k ^ r1
3. The new Z axis orthogonal to XY (mandatory) : r3 = r1 ^ r2
In point 2, k is an arbitrary unit ve tor, that xes the position of the new Y axis in the plane orthogonal
1. The new X axis parallel to the baseline:

to X. We take it equal to the Z unit ve tor of the old left matrix, thereby onstraining the new Y axis to be
orthogonal to both the new X and the old left Z.
This algorithm fails when the opti al axis is parallel to the baseline, i.e., when there is a pure forward
motion.

4 The re tifying transformation


In order to re tify { let's say { the left image, we need to ompute the transformation mapping the image
~ o1 = [Qo1 jq~ o1 onto the image plane of P
~ n1 = [Qn1 jq~ n1 . It is useful to think of an image as the
plane of P
interse tion of the image plane with the one of rays between points in 3D spa e and the opti al entre. We
are moving the image plane while leaving xed the one of rays. We will see that the sought transformation
is the ollinearity (linear transformatiopn of the proje tive plane) given by the 3  3 matrix T1 = Qn1 Qo11 .
The same result applies to the right image.
For any 3-D point w we an write

m~ o1 ' P~ o1w~
m~ n1 ' P~ n1 w~ :

(11)

A ording to (8), the equations of the opti al rays are the following (sin e re ti ation does not move the
opti al enter):

w = 1 + oQo111m~ o1
w = 1 + n Qn1 m~ n1

o 2 R
n 2 R;

(12)

m~ n1 = Qn1Qo11m~ o1

2R

(13)

hen e
(note that  absorbes the arbitrary s ale fa tor, hen e we use =).
The transformation T1 is then applied to the original left image to produ e the re ti ed image, as in
Figure 3. Note that the pixels (integer- oordinate positions) of the re ti ed image orrespond, in general,
to non-integer positions on the original image plane. Therefore, the gray levels of the re ti ed image are
omputed by bilinear interpolation.
Re onstru tion of 3-D points by triangulation an be performed from the re ti ed images dire tly, using
Pn1,Pn2.

5 Summary of the re ti ation algorithm


Given the high di usion of stereo in resear h and appli ations, we have endeavored to make our algorithm
as easily reprodu ible and usable as possible. To this purpose, we give the working matlab ode of the
algorithm; the ode is simple and ompa t (22 lines), and the omments en losed make it understandable
without knowledge of matlab. The usage of the re tify fun tion (see matlab ode) is the following:





Given a stereo pair of images I1,I2 and PPMs Po1,Po2 (obtained by alibration);
ompute [T1,T2,Pn1,Pn2 = re tify(Po1,Po2);
re tify images by applying T1 and T2.

fun tion [A,R,t = art(P)


% ART: fa torize a PPM as P=A*[R;t
Q = inv(P(1:3, 1:3));
[U,B = qr(Q);
R
t
A
A

=
=
=
=

inv(U);
B*P(1:3,4);
inv(B);
A ./A(3,3);

fun tion [T1,T2,Pn1,Pn2 = re tify(Po1,Po2)


% RECTIFY: ompute re tifi ation matri es
% fa torize old PPMs
[A1,R1,t1 = art(Po1);
[A2,R2,t2 = art(Po2);
% opti al enters (un hanged)
1 = - inv(Po1(:,1:3))*Po1(:,4);
2 = - inv(Po2(:,1:3))*Po2(:,4);
% new x axis (= dire tion of the baseline)
v1 = ( 1- 2);
% new y axes (orthogonal to new x and old z)
v2 = ross(R1(3,:)',v1);
% new z axes (orthogonal to baseline and y)
v3 = ross(v1,v2);
% new extrinsi parameters
R = [v1'/norm(v1)
v2'/norm(v2)
v3'/norm(v3);
% translation is left un hanged
% new intrinsi parameters (arbitrary)
A = (A1 + A2)./2;
A(1,2)=0; % no skew
% new proje tion matri es
Pn1 = A * [R -R* 1 ;
Pn2 = A * [R -R* 2 ;
% re tifying image transformation
T1 = Pn1(1:3,1:3)* inv(Po1(1:3,1:3));
T2 = Pn2(1:3,1:3)* inv(Po2(1:3,1:3));

The MATLAB an C implementation of the algorithm an be found on line.

6 Experiments
For this experiments we used alibrated stereo pairs from INRIA-Syntim. We show the results obtained
with a nearly re ti ed stereo rig (Figure 3) and with a more general stereo geometry (Figure 4). The pixel
oordinates of the re ti ed images are not onstrained to lie in any spe ial part of the image plane, and
an arbitrary translation were applied to both images to bring them in a suitable region of the plane, by
hanging the image enter. Then the output images were ropped to the size of the input images.
For example, In the ase of the \Sport" stereo pair (image size 768  576), we started from the following
amera matri es:

2
9:765102
Po1 = 4 9:8491011
5:79010

5:382101
2:398102 3:875105
2
9:33310
1:574102 2:428105 5
1
1:10810
8:07710 1 1:118103

2
9:767102
Po2 = 4 9:8681011
5:76610

5:376101
2:400102 4:003104
2
9:31010
1:567102 2:517105 5
1
1:14110
8:08910 1 1:174103

After adding the statement A(1,3) = A(1,3) + 160


to the re tify program, to keep the re ti ed image in the enter of the 768  576 window, we obtained the
following re ti ed amera matri es:

2
1:043103
Pn1 = 4 1:1651021
6:85510
2
1:043103
4
Pn2 = 1:1651021
6:85510

7:452101
2:585102 4:124105
2
9:33810
1:410102 2:388105 5
1
1:13910
7:19010 1 1:102103

7:452101
2:585102 4:069104
2
9:33810
1:410102 2:388105 5
1
1:13910
7:19010 1 1:102103

Left image

Right image

Rectified left image

Rectified right image

Figure 3: \Sport" stereo pair (top) and re ti ed pair (bottom). The right pi tures plot the epipolar lines
orresponding to the points marked in the left pi tures.

Left image

Right image

Rectified left image

Rectified right image

Figure 4: \Color" stereo pair (top) and re ti ed pair (bottom). The right pi tures plot the epipolar lines
orresponding to the points marked in the left pi tures.

Referen es
[1 N. Aya he and F. Lustman. Trino ular stereo vision for roboti s. IEEE Transa tions on Pattern
Analysis and Ma hine Intelligen e, 13:73{85, January 1991.
[2 B. Caprile and V. Torre. Using vanishing points for amera alibration. International Journal of
Computer Vision, 4:127{140, 1990.
[3 U. R. Dhond and J. K. Aggarwal. Stru ture from stereo { a review. IEEE Transa tions on Systems,
Man and Cyberneti s, 19(6):1489{1510, November/De ember 1989.
[4 O. Faugeras. Three-Dimensional Computer Vision: A Geometri Viewpoint. The MIT Press, Cambridge, MA, 1993.
[5 A. Fusiello, E. Tru o, and A. Verri. A ompa t algorithm for re ti ation of stereo pairs. Ma hine
Vision and Appli ations, 2000. To appear.
[6 R. Hartley and R. Gupta. Computing mat hed-epipolar proje tions. In Pro eedings of the IEEE Conferen e on Computer Vision and Pattern Re ognition, pages 549{555, 1993.
[7 R. I. Hartley and P. Sturm. Triangulation. Computer Vision and Image Understanding, 68(2):146{157,
November 1997.
[8 R.I. Hartley. Theory and pra ti e of proje tive re ti ation. International Journal of Computer Vision,
35(2):1{16, November 1999.
[9 F. Isgro and E. Tru o. Proje tive re ti ation without epipolar geometry. In CVPR99, pages I:94{99,
1999.
[10 C. Loop and Z. Zhang. Computing re tifying homographies for stero vision. In Pro eedings of the IEEE
Conferen e on Computer Vision and Pattern Re ognition, pages I:125{131, 1999.
[11 D. V. Papadimitriou and T. J. Dennis. Epipolar line estimation and re ti ation for stereo images pairs.
IEEE Transa tions on Image Pro essing, 3(4):672{676, April 1996.
[12 M. Pollefeys, R. Ko h, and L. VanGool. A simple and e ient re ti ation method for general motion.
In ICCV99, pages 496{501, 1999.
[13 L Robert. Camera alibration without feature extra tion. Computer Vision, Graphi s, and Image
Pro essing, 63(2):314{325, Mar h 1996.
[14 L. Robert, C. Zeller, O. Faugeras, and M. Hebert. Appli ations of non-metri vision to some visuallyguided roboti s tasks. In Y. Aloimonos, editor, Visual Navigation: From Biologi al Systems to Unmanned Ground Vehi les, hapter 5, pages 89{134. Lawren e Erlbaum Asso iates, 1997.
[15 E. Tru o and A. Verri. Introdu tory Te hniques for 3-D Computer Vision. Prenti e-Hall, 1998.