Beruflich Dokumente
Kultur Dokumente
1 Introduction
In this paper we investigate the colorimetric simulation of a scene as viewed under different illuminants. Applied to ne arts paintings, museological objects, jewelry, textiles, etc., such simulations could be of particular interest as a multimedia application. It is well known that the appearance of an object or a scene may change considerably when the illuminant changes, due to physical and psychophysical effects. These effects are taken into account in most colour appearance models in a somewhat heuristic manner. However, such models can not predict correctly changes for arbitrary illuminants, one important reason for this being metamerism. To be able to predict quantitatively the physical phenomena involved in a change between any illuminants, a more complete spectral description of the illuminants and the scene reectances is needed. The acquisition of multispectral images is thus required. A multispectral image is an image where each pixel contains information about the spectral reectance of the imaged scene. Multispectral images carry information about a number of spectral bands: from three components per pixel for colour images to several hundreds of bands for hyperspectral images. Multispectral imaging is relevant to several domains of application, such as remote sensing [1], physics, analysis of museological objects [2], cosmetics, medicine [3], high-accuracy colour printing, or computer graphics [4]. Hyperspectral image acquisition systems are complex and expensive, limiting their current use mainly to remote sensing applications. Multispectral scanners are mostly based on a point-scan scheme [5], and are thus too slow for our applications. We use a low-cost approach based on digital imaging techniques in which a set of chromatic lters are used with a CCD camera. It is well known that with 3 well-chosen lters, it is possible to obtain a fairly good reconstruction of the colour tristimulus values of the reference human observer as dened in colorimetry. Our aim is to reconstruct the spectral reectance curve using more than three lters. We propose a solution where the lters to be used are chosen from a set of readily available lters [2]. This choice is optimized, taking into account the statistical spectral properties of the objects that are to be imaged, the spectral characteristics [6] of the camera, and the spectral radiance of the lighting used for the acquisition. In the next section we present the principles of a system for multispectral image acquisition, including the necessary transformations to obtain spectral reectance data from the camera responses and a discussion on the choice of the lters. Then we proceed, in Section 3, to the simulation of coloured objects viewed under various illuminants. We compare the results obtained with the multispectral imaging approach to the CIELAB space used as a colour appearance model.
()
()
()
()
()
Figure 1: Schematic view of the image acquisition process. The camera response depends on the spectral sensivity of the sensor, the spectral transmittance of the optical lter and optical path, the spectral reectance of the objects in the scene, and the spectral radiance of the illumination. Note that only one optical colour lter is represented in the gure. In a multichannel system, a set of lters are used. Supposing a linear optoelectronic transfer function of the acquisition system [2, 10], the camera response c to an image pixel is then equal to
c
max min
lR
max min
( )!( ) d
(1)
lR o a denotes the system unknowns. By uniformly sampling the spectra at N where ! wavelength intervals, we can rewrite Equation (1) as follows c
( )= ( ) ( ) ( )
(2)
where ! is the vector containing the joint characteristics of the system unknowns, and spectral reectance.
r is the sampled
Let us now consider the vector ! describing the system unknowns. To perform an estimation of ! , we measure the responses cp, p : : : P , corresponding to a selection of P colour patches, P N, with known reectances p. Denoting the sampled spectral reectances of all the patches as the matrix c1c2 : : : cP t to these P samples is then given simply by 1 2 : : : P , the camera response P
R= r r
r]
=1
c =
cP = Rt!:
(3)
If
rank(R)
where t? denotes the Moore-Penrose pseudo-inverse of t which minimises the root mean square spectral error [11]. Note that in practice, with noisy data, this system inversion could be not straightforward [6].
(4)
()
=1 ()
ck
()
max min
( ) k( )!( )d
(5)
c =
c1c2 : : : cK t representing the response to all K lters may be described using matrix
cK = tr;
= ] = ( ) ( )]
(6)
where is the known N -line, K -column matrix of lter transmittances multiplied by the camera characteristics, that is kn k n ! n . We now adress the problem of how to use these camera responses. A rst approach is to dene a direct colorimetric transformation from the camera responses K into for example the CIELAB space [12] under a given illuminant, minimising typically the root mean square error, in a way similar to what is often done for conventional three-channel image acquisition devices [13]. Given an appropriate regression model, this is found to give quite satisfactory results in terms of colorimetric errors [14]. However, for our applications we are concerned not only with the colorimetry of the imaged scene, but also with the inherent surface spectral reectance of the viewed objects.
We will formulate the problem of the estimation of a spectral reectance ~ from the camera responses r cK as nding a matrix Q that reconstructs the spectrum from the K measurements as follows ~ = QcK : r (7) r Our goal will thus be to determine a matrix Q that minimises a distance d(r; ~), given an appropriate error measure d. Some solutions to this problems are presented and discussed in the following sections.
In existing multispectral acquisition systems, the lters often have similar and rather narrow bandpass shape and are located at approximatively equal wavelength intervals. For the reconstruction of the spectral reectance, it has been proposed to apply interpolation methods such as spline interpolation or Modied Discrete Sine Transformation (MDST) [15]. Such methods are not well adapted to lters having more complex wide-band responses, and suffer from quite severe aliasing errors [14].
2.2.1
Pseudoinverse solution
An immediate solution for estimating the spectral reectance consists in simply inverting Equation (6) by using a pseudoinverse approach, which provides us with the following minimum norm solution [11]
Q0 is thus given by Q0 =
~= ( r
)?1 cK =
t? :
t?
cK :
(8)
(9)
is of full rank N , and if we assume noiseless recordings, this reconstruction would be If the matrix perfect. However in practical situations this method of reconstruction is not very well adapted. Firstly, to achieve a rank of that equals N would require the number of colour lters K to be greater or equal to the number of sampling points N of the spectrum. Furthermore this representation is very sensitive to signal noise. In fact, by this solution we minimise the Euclidian distance dE t ; K in the camera response domain. A small distance does not guarantee that the spectra and are close. Nevertheless, this approach is used by Tominaga [9] to recover the spectral distribution of the illuminant from a sixchannel acquisition. However, he applies a nested regression analysis to choose the proper number of components in order to better describe the spectrum and to increase the spectral-t quality.
~ r
( rc )
2.2.2
We now dene another reconstruction operator 1 that minimises the Euclidian distance dE ; between the original spectrum and the reconstructed spectrum 1 K . To achieve this minimisation we take advantage of apriori knowledges on the spectral reectances that are to be imaged. We know that the spectral reectances of typical objects are smooth. We might present this by assuming that the reectance in each pixel is a linear combination of a set of smooth reectance functions known apriori. Denoting these reference spectral reectances as 1 2 : : : P , our assumption implies that, for any observed reectance , it exists a vector of coefcients such that may be expressed as
~=Q c r r] r
(r ~) r
~ = Q1cK = Q1 tr = Q1 tRa: r (11) r Inserting Equations (10) and (11) into the expression r = ~ gives Q1 tRa = Ra: (12) Assuming that R is a good representation of the reectances that will be encountered, Equation (12) should be true for any a, and hence Q1 tR = R: (13)
This gives then the reconstruction operator minimising the RMS spectral error by a pseudoinverse approach as
~ r
R= r r a r = Ra:
(10)
encountered in the applications. In our experiments on paintings we used a set of 64 spectral reectances
Q1 = R( tR)?: (14) The choice of the spectral reectances chosen in R must be well representative of the spectral reectances
of pure pigments used in oil painting [2]. For other applications, sets that are supposed to be representative of general reectances could be used, such as the object colours of Vrhel et al. [16] or the natural colours of Jaaskelainen et al. [17]. Slightly different methods exist, such as the Wiener estimation method [18] based on the autocorrelation matrix of , and a principal component analysis method where the principal components of the spectral reectance are estimated by a least mean square approach from the camera responses [14].
2.2.3
When designing or evaluating the quality of an acquisition system, it is of great importance to dene appropriate error or eventually quality measures. Depending on the intent, these may be based on colorimetric or spectral properties, on mean or maximal errors in a data set, or eventually on critical samples for which the reconstruction quality is particularly important to guarantee for a specic application. The colorimetric and spectral sampling methods being not adapted to our applications, we have performed a rapid evaluation of the reconstruction methods presented in Sections 2.2.1 and 2.2.2. These experimental results shown in Figure 2 indicate clearly that the reconstruction operator 1 is superiour to 0 , as expected.
1.4 1.2 1
0.45
Reflectance
Reflectance 400 450 500 550 600 650 Wavelength, , [nm] 700 750
0.2 0
0.05 0 400 450 500 550 600 650 Wavelength, , [nm] 700 750
Figure 2: Two spectral reectance reconstructions from the response of a camera using seven lters. We note the clear superiority of method 1 compared to method 0 . Method 1 takes into account apriori knowledge on the spectral reectances encountered in a given application.
of the objects that are to be imaged, as well as the spectral transmittances of the lters and the spectral characteristics of the camera [6]. The main idea of this method is to choose the lters so that, when multiplied with the illuminant and camera characteristics, they span the same vector space as the reectances that are to be acquired. The lters are then supposed to be optimal for this particular application and for a given scanning illuminant and camera spectral characteristics. In our simulation, we used D65 as the scanning illuminant, the Eikonix camera spectral characteristics, lters chosen from a set of 37 Wratten, Hoffman, and Schott lters, and the spectral reectances of a colour chart of 64 pure pigments used in oil painting and provided to us by the National Gallery in London. The resulting spectral sensitivities of the camera channels are shown in Figure 3 for the case of seven lters. We resume in Figure 4 the complete chain of a multi-channel image acquisition system with the nal spectral reconstruction step.
Spectral transmittance of the 7 filters 1 0.9 0.8 0.6 Transmittance, () 0.7 0.6 0.5 0.4 0.3 0.2 0.2 0.1 0 400 450 500 550 600 650 Wavelength, , [nm] 700 750 0.1 0 Sensitivity, () 0.5 0.4 0.3 0.8 0.7 Spectral sensitivity of the 7 channels
1 2 3 4 5 6 7
400
450
500
700
750
Figure 3: The spectral transmittance of the seven lters chosen according to the method described in [2] (left). The corresponding spectral sensitivitiy of the seven camera channels including the spectral characteristics of the Eikonix camera, the spectral radiance of the illuminant D65, and the spectral transmittance of the lters (right). The numbers denote the sequence in which the lters are chosen.
0.9 Emerald green Ultramarine Red ochre Mercuric Iodide 0.8 Emerald green Ultramarine Red ochre Mercuric Iodide
0.5 0.4 0.3 0.2 0.1 0 400 450 500 550 600 650 Wavelength, [nm] 700 750
Reflectance r()
0.6
450
500
Multi-channel image
700
750
Figure 4: The complete chain of multi-channel image acquisition system with the nal spectral reconstruction step.
computer monitor could be of particular interest as i) a high-end multimedia application for the open market, the user himself choosing his preferred light source, or ii) a computer aided tool for specialists, for example a curator having to decide the appropriate light sources for an art exhibition. It is well known that the appearance of an object or a scene may change considerably when the illuminant changes, due to physical and psychophysical effects. These effects are taken into account in most colour appearance models in a somewhat heuristic manner. However, such models can not predict correctly changes for arbitrary illuminants, one important reason for this being metamerism. To be able to predict precisely the physical phenomena involved in a change between any illuminants, a complete spectral description of the illuminants and the scene reectances is needed. We will here present two methods for the simulation of objects viewed under different illuminants. First, a classical method based on the CIELAB space is described in Section 3.1. Then, we describe in Section 3.2 the method applying multispectral imaging techniques. In Section 3.3 we compare the two methods using E94 under the simulated illuminant as an error measure.
where represents the colour matching functions, and the D65 spectral radiance.
A = xyz]
] = AtLD65 r;
(15)
The key point in the way CIELAB treats the illuminant is that when converting from XYZ to CIELAB, the XYZ values are taken relative to the XYZ values of the illuminant. Thus,
tristimulus values of a perfect diffuser under D65 lighting. Since we assume an ideal image capture device, these CIELAB values are colorimetrically exact for illuminant D65.
( ) being dened by well-known functions given in [12], and [XW;D65; YW;D65; ZW;D65] being the
(16)
^ ^ ^ ] = LD65 aD65 bD65]: By applying the inverse transformation g ?1 ( ) we obtain the following relation
Lsim asim bsim
When using CIELAB as a colour appearance model, we assume that the CIELAB values of a given surface colour are constant and independent of illuminant changes. The estimation of the CIELAB values of this colour under a simulated illuminant Lsim are thus given by (17)
"
= ]
"
(18)
where Xsim ; Ysim ; Zsim and XW;sim ; YW;sim ; ZW;sim are the tristimulus values of the surface and of the perfect diffuser, respectively, under the illuminant Lsim . We see from Equation (18) that the CIELAB space takes into account the effects of chromatic adaptation by applying a Von Kries [30] transform in the XYZ space.
~ r ~ = Qc r
where sim is the diagonal matrix corresponding to the spectral radiance of the simulated illuminant. These values are then used to estimate the CIELAB values under this particular illuminant,
Lsim asim bsim
~ ~ ~ ] = AtLsim ~; r
(19)
(20)
] = g(Xsim=XW;sim; Ysim =YW;sim; Zsim =ZW;sim): (22) ^ ^ b These values are then compared to the estimated values by the CIELAB model, Lsim asim ^sim] (cf. Equa~ sim asim ~sim] (cf. Equation (20)). tion (17)), and to those estimated by the multispectral image approach L ~ b
Lsim asim bsim t
] = AtLsim r;
(21)
We have performed an analysis of the illuminant-simulation quality for i) the multispectral image approach with 5, 7 and 10 channels, and for ii) the CIELAB space as a colour appearance model with D65 as starting reference. The methods are evaluated using ve illuminants, the CIE daylight illuminants D65 and D50, the CIE standard illuminant A (representative of a typical tungsten lighting with a colour temperature of 2856K), a normal uorescent lamp F2, and a low-pressure sodium lamp (LPS) widely used in street lighting, see Figure 5. The spectral reectances used for evaluation are those of the 64 oil pigments previously introduced in Section 2.3. The results in terms of mean and maximal E94 errors are listed in Table 1, the error histograms are given in Figure 6, and a graphical representation of the results for the case of a seven-channel acquisition system is given in Figure 7. We note that at the evident exception of D65 which serves as reference for the CIELAB model, the multispectral approach performs generally signicantly better than the CIELAB model. The CIELAB model performs reasonably well for the D50 case. This was expected since D65 and D50 have similar spectra, the change from D65 to D50 introducing only limited metameric problems. The D50 illuminant simulation using the CIELAB model is even better than the multispectral approach when using only 5 lters, the spectral reconstruction errors becoming greater than the errors induced by the CIELAB model. Almost complete failure, with a mean E94 error of 20.10, is found for the CIELAB model in the case of low-pressure sodium lamp. This was also expected, since its spectral power distribution consists almost entirely of two spectral lines at 589.0 and 589.6 nm [32]. We have also evaluated the methods with similar results using Eab as error measure, as well as applying them to different spectral reectance databases, such as the Munsell spectra [33].
300
200
150
100
50
400
450
500
700
750
Figure 5: The relative spectral radiances of the ve illuminants used in the experiment. Illuminant D65 A F2 D50 LPS CIELAB Mean Max 0.00 0.00 4.94 11.33 3.63 7.67 1.56 4.27 20.10 52.68 Multispectral (5) Mean Max 1.58 10.53 1.92 15.51 2.15 14.31 1.71 12.23 1.40 10.06 Multispectral (7) Mean Max 0.56 2.45 0.54 3.90 0.71 3.79 0.56 2.87 1.37 11.61 Multispectral (10) Mean Max 0.14 0.53 0.14 0.67 0.31 2.00 0.14 0.54 1.01 7.82
Table 1: Mean and maximal E94 errors obtained for four illuminant simulations using the CIELAB space as a colour appeareance model, and the multispectral approach using 5, 7 and 10 lters.
Histogram of E 70 60 50 Occurences 40 30 20 10 0 0 10
* 94
Histogram of E
* 94
50
40
30
20
10
20
30 * CIE E
40
50
60
0 0
94
6 * CIE E
10
12
94
Figure 6: Histograms of E94 errors for the CIELAB and multispectral (7 channel) methods. Note the difference in scale between the two histograms.
4 Conclusion
We have described a system for the acquisition of multispectral images in which a set of chromatic lters are used with CCD camera. We have proposed an efcient method for the reconstruction of the spectral
Simulations of illuminant F2
60
40
20
0 a*
20
40
60
80
60
40
20
0 a*
20
40
60
80
60
40
20
0 a*
20
40
60
80
60
40
20
0 a*
20
40
60
80
Figure 7: Simulation results with the illuminants A, F2, D50 and a low-pressure sodium lamp. The models performance is compared to direct spectral calculation of CIELAB under the simulated illuminant. The results are projected in the a*-b*-plane of the CIELAB space. The reference CIELAB values under , the values predicted by the CIELAB model by asthe simulated illuminant are marked with circles , and those predicted by the multispectral image approach by crosses . We note a clear terisks superiority of the simulation obtained by using the multispectral image approach.
()
()
( )
reectance of the object surface imaged in each pixel of the scene, from the camera responses using the set of lters. This reconstruction as well as the choice of lters are optimized by taking into account the statistical spectral properties of the objects that are to be imaged, as well as the spectral characteristics of the camera and the spectral radiance of the illuminant that is used for the acquisition. In order to reveal the modications in the colour appearance of an object or a scene when the illuminant is changed, a colorimetric simulation can be of particular interest in multimedia applications, especially in the museum eld. We have investigated two methods for such an illuminant simulation, a classical method using the CIELAB colour space as a colour appearance model, and a method using multispectral images. The multispectral image approach is found to be very performant, even when applied to illuminants that are particularily difcult to handle with conventional methods.
5 Acknowledgements
This work has been supported partially by the Norwegian Research Council.
References
[1] P. H. Swain and S. M. Davis, editors. Remote Sensing: The Quantitative Approach. McGraw-Hill, New York, 1978. [2] Henri Matre, Francis Schmitt, Jean-Pierre Crettez, Yifeng Wu, and Jon Yngve Hardeberg. Spectrophotomet ric image analysis of ne art paintings. In Proceedings of IS&T and SIDs 4th Color Imaging Conference: Color Science, Systems and Applications, pages 5053, Scottsdale, Arizona, November 1996. [3] D. L. Farkas, B. T. Ballou, G. W. Fisher, D. Fishman, Y. Garini, W. Niu, and E. S. Wachman. Microscopic and mesoscopic spectral bio-imaging. In Proc. Soc. Photo-Optical Instr. Eng, volume 2678, pages 200206, 1996. [4] Mark S. Peercy. Linear color representations for full spectral rendering. Computer Graphics Proceedings, pages 191198, 1993. [5] Yoshitsugu Manabe and Seiji Inokuchi. Recognition of material types and analysis of interreection using spectral image. In Proceedings of the 8th Congress of the International Colour Association, AIC Color 97, volume II, pages 507510, Kyoto, Japan, 1997. [6] Jon Yngve Hardeberg, Hans Brettel, and Francis Schmitt. Spectral characterisation of electronic cameras. In The International Symposium on Electronic Image Capture and Publishing, EICP98, Z rich, Switzerland, u May 1998. To appear. [7] Laurence T. Maloney and Brian A. Wandell. Color constancy: a method for recovering surface spectral reectance. Journal of the Optical Society of America A, 3(1):2933, 1986. [8] Brian Funt. Linear models and computational color constancy. In Proceedings of IS&T and SIDs 3rd Color Imaging Conference: Color Science, Systems and Applications, pages 2629, Scottsdale, Arizona, November 1995. [9] Shoji Tominaga. Multichannel vision system for estimating surface and illumination functions. Journal of the Optical Society of America A, 13(11):21632173, November 1996. [10] J. E. Farrell and B. A. Wandell. Scanner linearity. Journal of Electronic Imaging, 3:147161, October 1993. [11] A. Albert. Regression and the Moore-Penrose pseudoinverse. Academic Press, 1972. [12] Colorimetry. CIE Publications, 15.2, 1986. [13] Jon Yngve Hardeberg, Francis Schmitt, Ingeborg Tastl, Hans Brettel, and Jean-Pierre Crettez. Color management for color facsimile. In Proceedings of IS&T and SIDs 4th Color Imaging Conference: Color Science, Systems and Applications, pages 108113, Scottsdale, Arizona, November 1996. [14] Peter D. Burns. Analysis of image noise in multispectral color acquisition. PhD thesis, Center for Imaging Science, Rochester Institute of Technology, 1997. [15] Thomas Keusen. Multispectral color system with an encoding format compatible with the conventional tristimulus model. Journal of Imaging Science and Technology, 40(6):510515, 1996. [16] Michael J. Vrhel, Ron Gershon, and Lawrence S. Iwan. Measurement and analysis of object reectance spectra. Color Research and Application, 19(1):49, 1994. See ftp://ftp.eos.ncsu.edu/pub/ spectra. [17] T. Jaaskelainen, J. Parkkinen, and S. Toyooka. Vector-subspace model for color representation. Journal of the Optical Society of America A, 7(4):725730, 1990. See http://www.lut.fi/ltkk/tite/ research/color/lutcs_readme.html. [18] Hideaki Haneishi, T. Hasegawa, Norimichi Tsumura, and Yoichi Miyake. Design of color lters for recording art works. In IS&T 50th Annual Conference, pages 369372, 1997.
[19] Michael J. Vrhel, H. Joel Trussell, and Jack Bosch. Design and realization of optimal color lters for multiilluminant color correction. Journal of Electronic Imaging, 4(1):614, January 1995. [20] Gaurav Sharma and H. Joel Trussell. Optimal lter design for multi-illuminant color correction. In Proc. of the Optical Society of America Annual Meeting, Rochester, NY, October 1996. [21] Poorvi L. Vora and H. Joel Trussell. Mathematical methods for the design of color scanning lters. IEEE Transactions on Image Processing, 6(2):312320, February 1997. [22] Reiner Lenz, Mats Osterberg, Jouni Hiltunen, Timo Jaaskelainen, and Jussi Parkkinen. Unsupervised ltering of color spectra. Journal of the Optical Society of America A, 13(7):13151324, July 1996. [23] Wenjun Wang, Markku Hauta-Kasari, and Satoru Toyooka. Optimal lters design for measuring colors using unsupervised neural network. In Proceedings of the 8th Congress of the International Colour Association, AIC Color 97, volume I, pages 419422, Kyoto, Japan, 1997. [24] Kirk Martinez, John Cupitt, and David Saunders. High resolution colorimetric imaging of paintings. In Cameras, Scanners and Image Acquisition Systems, volume 1901 of SPIE Proceedings, pages 2536, 1993. [25] A. Abrardo, V. Cappellini, M. Cappellini, and A. Mecocci. Art-works colour calibration using the VASARI scanner. In Proceedings of IS&T and SIDs 4th Color Imaging Conference: Color Science, Systems and Applications, pages 9497, Scottsdale, Arizona, November 1996. [26] Mey-Chun Lo, M. Ronnier Luo, and Peter A. Rhodes. Evaluating colour models performance between monitor and print images. Color Research and Application, 21(4):277291, August 1996. [27] Karen M. Braun and Mark D. Fairchild. Testing ve color-appearance models for changes in viewing conditions. Color Research and Application, 22(3):165173, June 1997. [28] Mark D. Fairchild and Roy S. Berns. Image color-appearance specication through extension of CIELAB. Color Research and Application, 18(3):178190, June 1993. [29] R. W. G. Hunt. The Reproduction of Colour. Fountain Press, Kingston-upon-Thames, UK, 5 edition, 1995. [30] Johannes von Kries. Chromatic adaptation. Festschrift der Albrecht-Ludwigs-Univerit t, pages 145158, a 1902. Excerpts reprinted in D. MacAdam, ed. Selected Papers on Colorimetry Fundamentals, pp 4752, SPIE, 1993. [31] R. McDonald and K. J. Smith. CIE94 a new colour-difference formula. J. Soc. Dyers Col., 111:376379, December 1995. [32] R. W. G. Hunt. Measuring Colour. Ellis Horwood, Hertfordshire, UK, 2 edition, 1991. [33] J. Parkkinen, J. Hallikainen, and T. Jaaskelainen. Characteristic spectra of Munsell colors. Journal of the Optical Society of America A, 6:318322, 1989. See http://www.lut.fi/ltkk/tite/research/ color/lutcs_readme.html.