Journal of Vision (2003) 3, 369-379

http://journalofvision.org/3/5/4/

369

Surface color perception under two illuminants: The second illuminant reduces color constancy
Joong Nam Yang
Steven K. Shevell
Department of Psychology, University of Chicago, Chicago, IL, USA
Departments of Psychology and Ophthalmology & Visual Science, University of Chicago, Chicago, IL, USA

This study investigates color perception in a scene with two different illuminants. The two illuminants, in opposite corners, simultaneously shine on a (simulated) scene with an opaque dividing wall, which controls how much of the scene is illuminated by each source. In the first experiment, the height of the dividing wall was varied. This changed the amount of each illuminant reaching objects on the opposite side of the wall. Results showed that the degree of color constancy decreased when a region on one side of the wall had cues to both illuminants, suggesting that cues from the second illuminant are detrimental to color constancy. In a later experiment, color constancy was found to improve when the specular highlight cues from the second illuminant were altered to be consistent with the first illuminant. This corroborates the influence of specular highlights in surface color perception, and suggests that the reduced color constancy in the first experiment is due to the inconsistent, though physically correct, cues from the two illuminants. Keywords: surface color, color constancy, illuminant estimation, cue inconsistency

Introduction
Judging the color of a surface is easy even though the light reaching the eye results from two distinct physical properties: the surface spectral reflectance and the spectral power distribution of the illuminant. Only the reflectance is a property of the surface, so the retinal image of the surface is always ambiguous with respect to the reflectance. Yet, the visual system is able to extract a stable surface color. This is the phenomenon of color constancy. Several theoretical frameworks have been proposed to explain how the visual system achieves color constancy. MacLeod and Golz (in press) propose that the color constancy problem is readily tractable if the illuminant spectral power distribution is assumed to be appromixated by Gaussians. This assumption is based on the premise that human color perception starts with three cone types whose sensitivities can be closely approximated by Gaussian functions. Zaidi (1998) shows that the color constancy problem can be simplified by invoking heuristics based on the correlation between natural surfaces and illuminants. Brainard and Freeman (1997) use a Bayesian approach: given prior distributions, they develop a way of estimating posterior distributions for illuminants and surfaces in a given scene. In the linearmodels approach to surface color perception (Pokorny, Shevell, & Smith, 1991; Hurlbert, 1998; Maloney, 1999), the problem is mathematically solvable under the assumption that lights and surfaces can be summarized by a small number of basis functions (Maloney, 1986; Dannemiller, 1993). The subspace computation of
DOI 10:1167/3.5.4

Maloney and Wandell (1986) is an algorithm that exploits such an assumption. Other theories employ additional assumptions about the surfaces in the scene, such as reference surfaces, averages, or mutual reflection (Buchsbaum, 1980; Brill, 1978; Funt, Drew, & Ho, 1991; Lee 1986; DZmura & Iverson, 1993; Land & MaCann, 1971). Human color perception is not perfectly color constant. Empirical studies show clear deviations from color constancy (Arend & Reeves, 1986; Arend, Reeves, Schirillo, Goldstein, 1991; Jin & Shevell, 1996; Brainard, 1998; Kraft & Brainard, 1999; Nascimento & Foster, 2000; Yang & Maloney, 2001; Yang & Shevell, 2002). Color constancy performance in human vision varies also from scene to scene, which is consistent with the hypothesis that surface color perception involves errorprone estimation of the illuminant (Maloney & Yang, in press). This raises the question of how features of the retinal image are used to estimate the illuminant. Proposed cues to the illuminant include mutual reflection (Funt et al., 1991), specular reflection boundaries (DZmura & Lennie, 1986), shadows (DZmura, 1992), illuminant gradients (Ullman, 1976), brightest spots (Land, 1986), and specular highlights that reflect the illuminating light. Empirical tests show that color perception is affected by specular highlights (Yang & Maloney, 2001), mutual reflection (Bloj, Kersten, & Hulbert, 1999), binocular disparity (Yang & Shevell, 2002), and perceptual organization (Schrillo & Shevell, 2000). The present study was conducted to answer the following question: How does a second illuminant affect color constancy when both illuminants shine on part of
ISSN 1534-7362 2003 ARVO

Received December 13, 2002; published July 8, 2003

Yang & Shevell

370

the scene? Is color perception affected by cues available from the second illuminant? If so, do the added cues help or hinder color constancy? We know of no explicit solution aimed at answering these questions, although a few reports are related to the problem. DZmura and Iverson (1993) have shown in their theoretical framework that when the same scene is seen twice under two different illuminants, it is mathematically possible, with certain assumptions, to solve the color constancy problem. This is not directly applicable to the questions in the present study because here the two illuminants coexist in the scene at the same time. Second, a single-illuminant color constancy algorithm could be extended to a multi-illuminant scene (Tominaga & Wandell, 1989; DZmura & Lennie, 1986; Lee, 1986; see also Brill, 1990). These algorithms may be helpful when an entire scene is lit by two or more illuminants at the same time, which is only part of the problem here. The problem of color constancy posed by the present study is illustrated in Figure 1. In the top left (Isolating Wall) stereogram, the center-dividing wall is sufficiently high so that the illuminant on either side does not reach the opposite region. Thus, each side of the wall has its own color constancy problem involving one illuminant. This is similar to studies of simultaneous color constancy (e.g., Arend & Reeves, 1986). In the top right (High Wall) and bottom left (Low Wall) stereograms, the dividing wall is not as tall so that each illuminant also reaches part of the scene on the other side of the wall. This is a form of the two-illuminant color-constancy problem; each side now has two parts, one lit by one illuminant and the other by both illuminants. This may or may not affect
I solating Wall

color constancy. Empirically, we find that color constancy is poorer in the Low Wall scene (bottom left) than in the Isolating Wall scene (top left). In a later experiment, specular highlights (Blake & Bulthoff, 1990, 1991) were perturbed (Yang & Maloney, 2001) to investigate their role in the two-illuminant problem. Specular highlights carrying the chromaticity of the second illuminant were altered to carry the chromaticity of the first illuminant. Measurements show this improved color constancy.

General Method
Apparatus
We used a binocular CRT set-up in which each eye viewed a separate display. The images from the two displays were fused using mirrors (for details, see Yang & Shevell, 2002). Stimulus Configuration All scenes were rendered with the three-dimensional rendering software RADIANCE (Larson & Shakespeare, 1997). Lights and surfaces were rendered using 9-stepfunction rendering in order to accurately determine the light at each point in the scene (Yang & Maloney, 2001; Yang, 1999). The surfaces were chosen from the Munsell Book of Colors (Kelley, Gibson, & Nickerson, 1943): G 6/6 for the objects, BG 5/4 for the background, and N 1/ for the wall. The wall had a checkerboard pattern to make it appear more distinct. The Isolating Wall scene in Figure 1 shows the
High Wall

L ow Wall

No Wall

Figure 1. Stereograms used in Experiment 1. Each scene has many identical objects against a uniform background. From the left and right upper corners of the scene shine two separate illuminants, Illuminants D65 and A in this example. There are four different stereograms that vary in how much light from the second illuminant falls on the opposite side of the stimulus: Isolating Wall, in which the wall did not allow any light from the second illuminant beyond the wall; High Wall, in which the wall was lowered, so light from the second illuminant fell only near the far edge of the opposite side; Low Wall, in which the wall was lowered further so that more light from the second illuminant fell on the opposite side; and No Wall, in which both illuminants fell on the whole scene.

Yang & Shevell

371

stimulus (with crossed fusion) from which all other stimuli were derived. The scene includes 14 objects on each side of the wall. The background, surfaces, and objects on each side were illuminated by separate illuminants, one at the top-left corner and the other at the top-right corner. The illuminants created shadows, mutual reflections, and specular highlights, as well as attached shadows and illuminant gradients. A small square test patch for asymmetric matching was located on one of the objects (the top right-most object) on the left side of the wall; a similar comparison patch was on another object (the top left-most object) on the right side of the wall. Except in the No Wall condition in Figure 1, the second illuminant did not fall on the two patches; that is, the test patch and comparison patch were in the shadow of the wall for the other three conditions. Observers All observers were paid volunteers. They had normal color vision verified with a Neitz anamoloscope, and normal binocular vision as tested with the Titmus Stereo Test. Three observers participated in each of the experiments. A day before the experimental sessions started, observers were given 1-2 hours of practice to familiarize themselves with the task. Procedure After one minute of adaptation to the scene, observers adjusted the chromaticity of the test patch (on the left side) to match the color appearance of the comparison patch in the right region. The observers were free to move their eyes around the image. Asymmetric matches were measured for five test colors specified in MacLeod and Boynton (1979) (l,s) chromaticity space: (0.70, 0.99), (0.62, 0.99), (0.66, 1.79), (0.66, 0.22), and (0.66, 1.00). The last of these five was metameric to Equal Energy White (EEW). Note that the unit of s is arbitrary and normalized here to 1.0 for EEW. The luminance of the test patch was held constant for all scenes (19 cd/m2), and was lower than the luminance in many nearby areas. For example, the luminance of the object on which the test resided was 30.3, 20.9, 29.0, and 14.0 cd/m2 left of, right of, above, and below the test patch. Observers adjusted the appearance of the test patch using two sets of keys on the keyboard. Each measurement was repeated three times in a single session and then averaged. Each condition was repeated on three days. Measurements were averaged over the three days. The presentation order of the stimuli was randomized.

Results
Experiment 1
Purpose This experiment investigated the role of a second illuminant, which partially fell on an existing scene that already had its own illuminant. The scenes were varied in terms of how much of each region was lit by the second illuminant. Stimuli Four scenes, all stereograms for crossed fusion, were used (Figure 1). In the Isolating Wall condition (top left), the scene was divided by an opaque wall, which was high enough to block any illuminating light from the opposite side. The bottom of the wall was perpendicular to the background plane. Thus light from only one illuminant fell on each side of the wall. In the High Wall condition (top right), the wall was lowered somewhat, so that the second illuminant crossed over to the other side of the wall, but the light from the second illuminant fell in only a small area at the far edge of each region (and far from the test and comparison patches). In the Low Wall condition (bottom left), the wall was lowered further so that more of each side was lit by the second illuminant. In this case, the second illuminant contributed more to shadows, specular highlights, and mutual reflections. Note, however, that the second illuminant did not fall on the test or comparison patch. In the No Wall condition
Illuminant D65 Illuminant A
Isolating wall

High wall

Low wall

Second illuminant No wall

Figure 2. Schematic of the scenes in Figure 1. Left. A side view of the stimuli. The horizontal line indicates the background surface and the thick vertical bar, the opaque wall. Right. A top view of the scenes. The gray areas indicate how much of each side receives light from the illuminant on the opposite side.

Yang & Shevell

372

(bottom right), the wall was eliminated and the entire scene was lit by both illuminants. Figure 2 presents a schematic of the scenes in Figure 1; the left part shows a side view of the stimuli and the right part shows a top view of the scene. Note that illuminant gradients were created because the illuminants were not infinitely distant light sources. In all scenes, the illuminants were positioned in the right and left corners, and behind the observer. If the light sources were infinitely far away from the surfaces, then in the No Wall condition, a single effective illuminant would be the sum of the two illuminants. The scenes in Figure 1 have Illuminant D65 on the left and Illuminant A on the right. Another identical set of stimuli was rendered using Illuminant D65 on both sides. These two sets of scenes were used in Experiment 1. Results Figure 3 shows the asymmetric matching results in the MacLeod-Boynton cone space (MacLeod & Boynton, 1979). Each column shows results for one of the four stimulus conditions in Figure 1, and each row is for one

of the observers. The vertical axis is S/(L+M) and the horizontal axis is L/(L+M). Filled squares and circles indicate, respectively, the light reflected from a neutral equal-reflecting surface at the location of the right-side comparison patch, with Illuminant D65 on both sides (D65-D65) or D65 on the left and A on the right (D65A). The coordinates of these two points are displaced from their spectral power distributions of (0.66, 1.05) and (0.70, 0.34), respectively, due to mutual reflections in the image. For example, in the Isolating Wall scene, these coordinates are (0.68, 0.81) and (0.71, 0.51), respectively. Open squares in Figure 3 represent matches when the illuminants are the same on both sides (D65), whereas open circles represent matches when the illuminants are different (Illuminant D65 on the left and Illuminant A on the right). If the arrows showing the change in match settings with a change in the right-side illuminant (from D65 to A) were equal to the dotted arrow (the vector indicating the shift in light falling on the comparisonpatch location due to changing illuminants), then the results would indicate perfect color constancy. Except for the No Wall condition, the arrows generally are similar to

Isolating wall
2.0 1.6 1.2 0.8 0.4 0.62 0.64 0.66 0.68 0.70 2.0

High wall

Low wall

No wall

JYL

0.62 0.64 0.66 0.68 0.70

0.62 0.64 0.66 0.68 0.70

0.62 0.64 0.66 0.68 0.70

MC

S/(L+M)

1.6 1.2 0.8 0.4 0.62 0.64 0.66 0.68 0.70 2.0 1.6 1.2 0.8 0.4 0.62 0.64 0.66 0.68 0.70 0.62 0.64 0.66 0.68 0.70 0.62 0.64 0.66 0.68 0.70 0.62 0.64 0.66 0.68 0.70 0.62 0.64 0.66 0.68 0.70 0.62 0.64 0.66 0.68 0.70 0.62 0.64 0.66 0.68 0.70

XZ

L/(L+M)

Figure 3. Results from Experiment 1 plotted in the MacLeod-Boynton Chromaticity Diagram. Asymmetric matching measurements are shown. The rows indicate different observers, whereas columns show different stimulus conditions as shown in Figure 1. The vertical and horizontal axes in each plot indicate the two axes of the chromaticity diagram. Filled squares and circles are the physical illuminants reflected from a neutral equal-reflecting surface. Dotted arrows show the illuminant change. Open squares and circles show, respectively, observers matches for the scenes with both illuminants at D65 or with the left illuminant at D65 and the right at A. Solid arrows indicate changes in observers settings. SEs are indicated for each setting along both axes, though most are hidden by the symbol.

Yang & Shevell

373

the dotted illuminant vector in direction but vary in magnitude. Overall, the results show fairly good color constancy with the Isolating Wall or the High Wall scenes, less good constancy with the Low Wall, and poor constancy with No Wall. Color Constancy Index To summarize these measurements, we used a color constancy index defined along the two separate axes of the MacLeod-Boynton diagram (Jin & Shevell, 1996; Yang & Shevell, 2002). The change in the observers settings from D65-D65 to D65-A was defined as the setting vector (solid arrows, Figure 3), and the change in the two illuminants as the illuminant vector (dotted arrows, Figure 3). The setting and illuminant vectors were projected onto one axis of the cone space; the ratio of the two projected vectors was defined as the color constancy index for the axis. A value of 1.0 indicates perfect constancy; 0 indicates no constancy. The same calculation was done for each axis, resulting in two separate color constancy indexes. Figure 4 shows the index for all three observers along each axis. These plots show that the color constancy indexes are similar for the conditions with the Isolating Wall or High Wall, and smaller for the conditions with the Low Wall or No Wall (p < .05 by a binomial test, comparing the Isolating and High Wall conditions to either the Low or the No Wall condition).
L/(L+M)
1.0 0.8 0.6 0.4 0.2 JYL
Isolating wall High wall Low wall No wall

S/(L+M)
1.0 0.8 0.6 0.4 0.2

Discussion The main finding in Experiment 1 is that when a second illuminant is introduced into a region that is already illuminated by another light, color constancy declines. As discussed in the Introduction, introducing a second illuminant makes the problem of color constancy more complex. Empirically we find here that it also reduces the degree of color constancy. This may be accounted for in the framework of illuminant estimation by the visual system. When light from the second illuminant is not mixed with light from the first illuminant, it is easier to estimate the first illuminant. Asymmetric matching, then, is mediated primarily by cues to the first illuminant. The relatively distant cues to the (misleading) second illuminant seem to be ignored. As the dividing wall is lowered, more cues to the second illuminant become available (Low Wall condition). In this case, information about the first illuminant is less well segregated from information about the second illuminant, resulting in diminished color constancy. When both illuminants shine on the whole scene (No Wall condition), color constancy is low. The difference in physical illuminating lights that fall on the test and comparison patches is smaller than in the other conditions but still substantial. One reason for poor color constancy in the No Wall condition may be that removing the wall made less obvious the presence of the two distinctly illuminated regions. In addition, illuminant-gradient cues in the background and shadow cues may be more subtle than in the other three conditions. If the light sources were infinitely far away, the two sides of the scene would be physically identical, making the asymmetric matching paradigm an isomeric matching task. The measurements, in fact, show very little shift when the right-side illuminant is changed (rightmost column, Figure 3).

Color Constancy Index

1.0 0.8 0.6 0.4 0.2

MC

1.0 0.8 0.6 0.4 0.2

Experiment 2
Purpose In Experiment 1, the area lit by the second illuminant was increased as the wall was lowered. This affected the spatial average of light on each side of the scene. Could this explain the results? This was tested using a new scene in which the spatial average of light on each side was similar to that of the Low Wall scene in Experiment 1, but fewer cues to the second illuminant were available. If the average light hypothesis is tenable, the new scene should cause a drop in color constancy comparable to that found for the Low Wall condition. Stimuli A new stimulus, the Raised Wall scene (Figure 5), was rendered with the wall raised from the background plane so that the second illuminant came from under the wall. The overall area lit by the second illuminant was similar

1.0 0.8 0.6 0.4 0.2

XZ

1.0 0.8 0.6 0.4 0.2

Figure 4. Color constancy index for Experiment 1. The vertical axis shows the color constancy index and the horizontal axis indicates the four stimulus conditions. Rows indicate observers and columns are the two chromaticity axes. Error bars are SEMs.

Yang & Shevell

Illuminant D65 Isolating wall Illuminant A Low wall

374

Low wall

Raised wall Raised wall Low wall

Second illuminant

Raised wall

Figure 5. Stimuli for Experiment 2. A schematic drawing of the side view (above) and top view (below) of the conditions is shown at left. Stereograms for the Low Wall and Raised Wall conditions are at right. The Isolating Wall and Low Wall conditions were the same scenes used in Experiment 1. For the new condition (Raised Wall), the wall was raised from the uniform background plane, so that now the illuminant on one side crossed over to the other side under the wall. The wall was slightly tilted to make clear the gap between the bottom of the wall and the background plane.

to that of the previous Low Wall scene. Note, however, that in the Raised Wall scene there were many fewer cues to the second illuminant, which were mostly associated with the objects, in comparison to the Low Wall scene. The Raised Wall was slightly tilted so that the observers percept of the opening under the wall was clear. The Raised Wall scene was tested in a new set of runs that also repeated the Isolating Wall and Low Wall conditions. Results and Discussion The degree of color constancy, calculated using the same index introduced in Experiment 1, is shown in Figure 6. First, the color constancy index is greatest when each illuminant was confined to one side (the Isolating Wall condition), compared to when cues to the second illuminant were available in the Low Wall condition, a result that replicates Experiment 1 for 3 additional observers (p < .05 by sign test). Second, the color constancy index is greater when the second illuminant lit the opposite-side region from under the wall (the Raised Wall condition), compared to the Low Wall condition (p < .05 by sign test). The space-average (l, s) chromaticities for the Low Wall and Raised Wall scenes were the same (0.67, 0.76), which shows that the average chromaticity cannot explain this difference. The average luminances were also identical (14.4 cd/m2). Thus, changes in average light cannot account for the results in Experiment 2. We propose that the weaker cues to the second illuminant in the Raised Wall scene, compared to the Low Wall scene, account for the increased constancy with the Raised Wall.
Color Constancy Index

L/(L+M)
1.0 0.8 0.6 0.4 0.2 1.0 GJM
Isolating wall Low wall Raised wall

S/(L+M)
0.8 0.6 0.4 0.2

1.0 0.8 0.6 0.4 0.2

LK

1.0 0.8 0.6 0.4 0.2

1.0 0.8 0.6 0.4 0.2

SHK

1.0 0.8 0.6 0.4 0.2

Figure 6. Color constancy index for Experiment 2. The vertical axis shows the color constancy index and the horizontal axis indicates the three stimulus conditions. Rows indicate observers and columns are the two chromaticity axes. Error bars are SEMs.

Yang & Shevell

375

Experiment 3
Purpose The results in Experiment 1 showed that the second illuminant hampers color constancy, but why? We approached this question by using the cue perturbation method (Maloney & Landy, 1989; Landy, Maloney, Johnston, & Young, 1995; a detailed description of the cue perturbation method involving specular highlights is in Yang & Maloney, 2001). The specular-highlight cue to the second illuminant was perturbed, such that this cue, which carried the second illuminant chromaticity, was replaced by a cue carrying the first illuminant chromaticity. This perturbation was expected to reveal how the second illuminant influences color constancy. The perturbation of specular highlights, however, physically changes the small bright spots in the scene. To assess whether constancy depends on specular-highlight cues versus just the light from highlights, another scene was constructed. The original (unperturbed) specular highlights were changed in location so that the highlights were not in their geometrically correct locations. Thus, if the effect of highlights is due to their chromaticities and luminances rather than to their proper geometrical relations as specular reflections, changing their locations should cause no change in the observers performance, because the moved specular highlights are still somewhere in the scene at their original chromaticities and luminances. Stimuli Figure 7 shows the four scenes for this experiment. The Isolating Wall and Low Wall conditions are the stimuli used in Experiment 1. A new scene, the
Isolating wall

Highlights Perturbation condition (bottom left, Figure 7), is the same as the Low Wall scene except that the chromaticity of the specular highlights was altered. The specular highlights on each object toward its end that was closest to the wall, which carried the chromaticity of the second illuminant, were changed to carry the chromaticity of the first illuminant, consistent with the highlights on the other side of the object. Finally, a fourth scene, the Highlights Relocation condition (bottom right, Figure 7), moved the locations of the original, unperturbed specular highlights in the Low Wall condition. The location of each specular highlight from the opposite-side illuminant was moved away from its geometrically correct location while still approximately equally distant from the test and comparison patches. This was done using a pixel-by-pixel switch of each specular highlight. Note that when highlights were moved to different locations, they were not coplanar with the background plane (see lower right stereogram in Figure 7). Results and Discussion Figure 8 shows the asymmetric match settings in the MacLeod-Boynton diagram. The vertical and horizontal axes indicate the two axes of the chromaticity diagram. Rows are observers and columns are the four stimulus conditions. Filled squares and circles indicate light from a neutral reflecting surface at the comparison-patch location for, respectively, the D65-D65 and D65-A conditions. Open squares and circles indicate observers asymmetric matches for D65-D65 and D65-A scenes, respectively. The finding in Experiment 1, that introducing the second illuminant reduces color constancy, is replicated
Low wall

Higlights perturbation

Highlight relocation

Figure 7. Stereograms for Experiment 3. The two top stereograms are the Isolating Wall and Low Wall scenes used in Experiment 1. The other two stereograms are perturbed scenes. In the Highlights Perturbation scene (bottom left), specular highlights are perturbed in chromaticity so that the specular highlights that reflected the second illuminant now reflect the chromaticity of the first illuminant. In the Highlights Relocations scene (bottom right), the highlights are in geometrically incorrect locations. See text for details.

Yang & Shevell

Isolating wall
2.0 1.6 1.2 0.8 0.4 0.62 0.64 0.66 0.68 0.70 2.0 0.62 0.64 0.66 0.68 0.70 0.62 0.64 0.66 0.68 0.70 0.62 0.64 0.66 0.68 0.70

376
Low wall Highlights perturbation Highlight relocation

BPS

DTM

S/(L+M)

1.6 1.2 0.8 0.4 0.62 0.64 0.66 0.68 0.70 2.0 1.6 1.2 0.8 0.4 0.62 0.64 0.66 0.68 0.70 0.62 0.64 0.66 0.68 0.70 0.62 0.64 0.66 0.68 0.70 0.62 0.64 0.66 0.68 0.70 0.62 0.64 0.66 0.68 0.70 0.62 0.64 0.66 0.68 0.70 0.62 0.64 0.66 0.68 0.70

HJL

L/(L+M)

Figure 8. Results from Experiment 3 plotted in MacLeod-Boynton Chromaticity Diagram. The asymmetric matching data are shown. The rows indicate different observers, whereas columns show different stimulus conditions as shown in Figure 7. The vertical and horizontal axes in each plot indicate the two axes in the chromaticity diagram. Filled squares and circles are the physical illuminants reflected from a neutral equal-reflecting surface. Dotted arrows show the illuminant change. Open squares and circles show, respectively, observers matches for the scenes with both illuminants at D65 or with the left illuminant at D65 and the right at A. Solid arrows indicate changes in observers settings. SEs are indicated for each setting along both axes, though most are hidden by the symbols.

here for another 3 observers. The degree of color constancy for the Isolating Wall scene is consistently higher than for the Low Wall scene (Figure 9; p < .05 by sign test). Figure 9 also shows two new results. First, color constancy in the Highlights Perturbation condition is higher than in the Low Wall condition along the L/(L+M) axis. The perturbation of chromaticities of specular highlights was done only to those areas reflecting the second illuminant, leaving specular highlights from the first illuminant intact. When highlights that carried the second illuminant were perturbed to reflect the chromaticity of the first illuminant, color constancy improved along the L/(L+M) direction (mean difference in constancy index of 0.095, p < .05 by Tukey HSD test). Constancy in the S/(L+M) did not change significantly. The increase in the L/(L+M) direction is consistent with the view that specular highlights are cues used to infer the illuminant, and in two-illuminant scenes color constancy is reduced when the highlights contain inconsistent information about the illuminant. Second, relocating specular highlights reduced color constancy along the L/(L+M) axis, compared to either the Low Wall or Highlights Perturbation condition (mean difference, respectively of 0.102 and 0.197, both significant at p < .05 by Tukey HSD test). Thus, the specular highlight cue is not just a bright spot at the illuminants chromaticity; it also depends on location. Note, however, that this result is not as predicted for the

L/(L+M)
1.0 0.8 0.6 0.4 0.2 BPS
Isolating wall Low wall Highlights perturbation Highlights relocation

S/(L+M)
1.0 0.8 0.6 0.4 0.2

Color Constancy Index

1.0 0.8 0.6 0.4 0.2 1.0 0.8 0.6 0.4 0.2

DTM

1.0 0.8 0.6 0.4 0.2

HJL

1.0 0.8 0.6 0.4 0.2

Figure 9. Color constancy index in Experiment 3. The vertical axis shows the color constancy index and the horizontal axis indicates the four stimulus conditions in Experiment 3. Rows indicate observers and columns are the two chromaticity axes. Error bars are SEMs.

Yang & Shevell

377

illuminant estimation hypothesis. The prediction was that color constancy should improve in the Highlights Relocation condition (compared to the Low Wall) because highlights in geometrically incorrect locations should weaken the influence of the second illuminant. The observed reduction in color constancy due to relocating highlights is in the opposite direction. This is discussed below.

Discussion
Over all the observers and experiments (except the No Wall condition), the color constancy index ranged from 0.10 to 0.68 along the L/(L+M) axis and from 0.0 to 0.56 along the S/(L+M) axis. These results confirm that constancy is neither complete nor fixed in magnitude, across viewing conditions or observers (cf., Arend & Reeves, 1986; Arend et al., 1991; Brainard, 1998; Kraft & Brainard, 1999). The degrees of color constancy in all Isolating Wall conditions across the experiments ranged from 0.30 to 0.68 (average 0.44) and from 0.17 to 0.56 (average 0.36) for the L/(L+M) and S/(L+M) directions, respectively. These are similar to the ranges and averages given in previous studies, though for somewhat different color constancy indexes. For example, values ranged from 0.13 to 0.46 (average 0.30) in the study by Arend et al. (1991). We used two separate constancy indices for the two axes, L/(L+M) and S/(L+M). Calculating separate indices revealed how observers settings were influenced along each direction in color space. Recall that 3 different observers participated in each of the 3 experiments (9 observers in all), and that the Isolating Wall and Low Wall scenes were included in each experiment. These data were used to assess differences in color constancy along the L/(L+M) and S/(L+M) axes. Overall, there was virtually no difference between the two chromatic axes: the mean colorconstancy-index values over the Isolating Wall and Low Wall conditions were 0.31 and 0.30 for L and S, respectively. Analysis of variance confirmed that color constancy was better with the Isolating Wall than the Low Wall scene (mean constancy index values of 0.40 and 0.22, respectively; F(1,8)=31, p < .01), as already reported above for each of the experiments. There was neither a significant effect of chromatic axis nor a chromatic-axis by wallcondition interaction. A trend for the color constancy index to decrease more for L/(L+M) than S/(L+M) between the Isolating and Low Wall conditions (drops of 0.18 and 0.10, respectively) did not reach statistical significance (F(1,8)=4.77, 0.05 < p < .10). Overall, adding a second illuminant to a scene with an existing illuminant was detrimental to color constancy. To understand why, we altered specular highlights carrying the chromaticity of the second illuminant so that they carried the chromaticity of the first illuminant. This improved color constancy along the L/(L+M) direction in

Experiment 3, which is consistent with the hypothesis that specular highlights from the second illuminant contribute to the reduction in constancy. Within-cue inconsistency arises when the two illuminants provide incompatible cues to the illumination. There are many illuminant cues in the Low Wall scene (Figure 1), including specular highlights and shadows, and each of these cues signals both Illuminants D65 and A. For example, there are two sets of specular highlights on most of the objects in this scene, one set reflecting Illuminant D65 and the other set Illuminant A. This is an example of within-cue inconsistency as defined here. When the specular highlight cue that signaled the inconsistent illuminant was perturbed to achieve cue consistency (Experiment 3), color constancy improved. Thus, inconsistent specular-highlight cues are proposed as a factor affecting color constancy in a multi-illuminant scene. Perturbation of other cues, a topic for future research, may show similar trends. The term within-cue inconsistency needs to be interpreted with caution. Here, it means that two occurrences of the same cue point to different illuminants. For example, on the left side of a particular object in the Low Wall condition, the specular highlight signals Illuminant D65; on the right side, the highlight on the same object signals Illuminant A. This is different from a scene in which two different cues (e.g., depth cues from perspective and binocular disparity) signal inconsistent depth percepts (Landy, Maloney, Johnston, & Young, 1995), which can be called between-cue inconsistency. The scenes with two illuminants used here created inconsistency within the same cue type. In Experiment 3, the results showed that relocating specular highlights affected color constancy, implying that color constancy is influenced by both the geometry and the chromaticity of highlights. As mentioned earlier, however, the illuminant estimation framework, which provided the rationale for this experiment, implied that relocation of the highlights should increase color constancy, because relocating the inconsistent cue should reduce its influence; instead, the measurements showed poorer constancy with relocation. We can only speculate why this occurred. Note that in Experiment 3 there is an important difference between the Highlights Perturbation and Highlights Relocation conditions. In the Highlights Perturbation scene, only the highlights chromaticities were altered; everything else in the scene was unchanged. In the Highlights Relocation scene, however, relocating the highlights led to two changes in the scene: (1) disruption of the original geometrically correct locations of cues to the second illuminant, and (2) a relocated set of bright spots in nearby locations that still carried the second illuminants chromaticity. The hypothesis that color constancy would increase due to the relocation of specular highlights takes into account only the first change (disruption of geometrically correct locations). The relocated highlights, which were often isolated points

Yang & Shevell

378

clearly perceived as closer than the background plane of the scene, may actually have increased the saliency of cues to the second illuminant (a kind of Gelb (1929) effect). Therefore, these measurements may not be in conflict with the illuminant estimation framework. Results in this study support the view that human color constancy is affected by illuminant cues available in the scene, as assumed by most computational algorithms in the linear models approach to color constancy. Experiment 2 shows that constancy is affected by the number of illuminant cues in a scene. This suggests that illuminant estimation depends on combining multiple cues. The weights given to the cues may depend on the elements in the particular scene (Maloney & Yang, in press). The scenes used here have identical objects against a background. One may consider whether the same results would be obtained with scenes having more chromatic variation. Quantitative differences are expected because cues to the illuminant will not be the same for different scenes. How the various cues are weighted or combined is an important question for further research.

Acknowledgments
This work was supported by National Institutes of Health Grant EY-04802. Publication supported in part by an unrestricted grant to the Department of Ophthalmology and Visual Science from Research to Prevent Blindness. Commercial relationships: none.

References
Arend, L. E., & Reeves, A. (1986). Simultaneous color constancy. Journal of the Optical Society of America A, 3, 1743-1751.[PubMed] Arend, L., Reeves, A., Schirillo, J., & Goldstein, R. (1991). Simultaneous color constancy: Papers with diverse Munsell values. Journal of the Optical Society of America A, 8, 661-672. [PubMed] Bloj, M., Kersten, D., & Hurlbert, A. C. (1999). Perception of three-dimensional shape influences color perception through mutual illumination. Nature, 402, 877-879.[PudMed] Blake, A., & Blthoff, H. (1990) Does the brain know the physics of specular reflection? Nature, 343, 165-168. Blake, A., & Bulthoff, H. (1991). Shape from specularities; Computation and psychophysics. Philosophical Transactions of the Royal Society of London B, 331, 237-252. Brainard, D. H. (1998). Color constancy in the nearly natural image. 2. Achromatic loci. Journal of the Optical Society of America A, 15, 307-325.[PubMed]

Brainard, D. H., & Freeman, W. T. (1997). Bayesian color constancy. Journal of the Optical Society of America A, 14, 1393-1411.[PubMed] Brill, M. H. (1978). A device performing illuminantinvariant assessment of chromatic relations. Journal of Theoretical Biology, 71, 473-478.[PubMed] Brill, M. H. (1990). Image segmentation by object color: A unifying framework and connection to color constancy. Journal of the Optical Society of America A, 7, 2041-2047.[PubMed] Buchsbaum, G. (1980). A spatial processor model for object color perception. Journal of the Franklin Institute, 310, 1-26. Dannemiller, J. L. (1993). Rank orderings of photoreceptor photon catches from natural objects are nearly illuminant-invariant. Vision Research, 33, 131-40.[PubMed] DZmura, M. (1992). Color constancy-surface color from changing illumination. Journal of the Optical Society of America A, 9, 490-493. DZmura, M., & Iverson, G. (1993). Color constancy. I. Basic theory of two-stage linear recovery of spectral descriptions for lights and surfaces. Journal of the Optical Society of America A, 10, 2148-2165[PubMed] DZmura, M., & Lennie, P. (1986). Mechanisms of color constancy. Journal of the Optical Society of America A, 3, 1662-1672.[PubMed] Funt, B., Drew, M., & Ho, J. (1991). Color constancy from mutual reflection. International Journal of Computer Vision, 6, 5-24. Gelb, A. (1929). Die Farbenkonstanz den Sehndinge. In A. Bethe (Ed.) Handbuch der normalen und pathologischen Physiologie, 12. Berlin: Springer. Hurlbert, A. C. (1998). Computational models of color constancy. In V. Walsh & J. Kulikowski (Eds.), Perceptual constancies; why things look as they do (pp.283-322). Cambridge: Cambridge University Press. Jin, E. W., & Shevell, S. K. (1996). Color memory and color constancy. Journal of the Optical Society of America A, 13, 1981-1991.[PubMed] Kelley, K. L., Gibson, K. S., & Nickerson, D. (1943). Tristimulus specification of the Munsell Book of Color from spectrophotometric measurements. Journal of the Optical Society of America, 33, 355-376. Kraft, J. M., & Brainard, D. H. (1999). Mechanisms of color constancy under nearly natural viewing. Proceedings of the National Academy of Sciences, USA, 96, 307-312.[PubMed]. Land, E. H. (1986). Recent advances in retinex theory. Vision Research, 26, 7-22.[PubMed]

Yang & Shevell

379

Land, E. H., & MaCann, J. J. (1971). Lightness and retinex theory. Journal of the Optical Society of America, 61, 1-11.[PubMed] Landy, M. S., Maloney, L. T., Johnston, E. B., & Young, M. (1995). Measurement and modeling of depth cue combination: In defense of weak fusion. Vision Research, 35, 389-412.[PubMed] Larson, G. W., & Shakespeare, R. (1997). Rendering with Radiance. San Francisco: Morgan Kaufmann. Lee, H. C. (1986). Method for computing the sceneilluminant chromaticity from specular highlights. Journal of the Optical Society of America A, 3, 16941699.[PubMed] MacLeod, D., & Boynton, R. (1979). Chromaticity diagram showing cone excitation by stimuli of equal luminance. Journal of the Optical Society of America A, 69, 1183-1186.[Pubmed] MacLeod, D., & Golz, J. (in press). A computational analysis of colour constancy. In R. Mausfeld & D. Heyer (Eds.), Colour vision: From light to object. Oxford: Oxford University Press. Maloney, L. T. (1986). Evaluation of linear models of surface spectral reflectance with small numbers of parameters. Journal of the Optical Society of America A, 3, 1673-1681.[PubMed] Maloney, L. T. (1999). Physics-based models of surface color perception. In K. R. Gegenfurtner & L. T. Sharpe (Eds), Color vision: From genes to perception (pp. 387- 416). Cambridge: Cambridge University Press. Maloney, L. T., & Landy, M. S. (1989). A statistical framework for robust fusion of depth information. Visual Communications and Image Processing IV, 1199, 1154-1163.

Maloney, L. T., & Wandell, B. A. (1986). Color constancy: A method for recovering surface spectral reflectance. Journal of the Optical Society of America A, 3, 29-33.[PubMed] Maloney, L. T., & Yang, J. N. (in press). The illuminant estimation hypothesis. In R. Mausfeld & D. Heyer (Eds.), Colour vision: From light to object. Oxford: Oxford University Press. Nascimento, S. M., & Foster, D. H. (2000). Relational color constancy in achromatic and isoluminant images. Journal of the Optical Society of America A, 17, 225-231[PubMed] Pokorny, J., Shevell, S. K., & Smith, V. C. (1991) Colour appearance and color constancy. In P. Gouras (Ed.), The Perception of Color (pp. 43-61). Boca Raton: CRC Press. Schirillo, J. A., & Shevell, S. K. (2000). Role of perceptual organization in chromatic induction. Journal of the Optical Society of America A, 17, 244-254.[PubMed] Tominaga, S., & Wandell, B. A. (1989). Standard surfacereflectance model and illuminant estimation. Journal of the Optical Society of America A, 6, 576-584. Ullman, S. (1976). On visual detection of light sources. Biological Cybernetics, 21, 205-212.[Pubmed] Yang, J. N. (1999). Analysis of illuminant estimation in surface color perception. Doctoral disseration. New York: New York University. Yang, J. N., & Maloney, L. T. (2001). Illuminant cues in surface color perception: Tests of three candidate cues. Vision Research, 41, 2581-2600.[PubMed] Yang, J. N., & Shevell, S. K. (2002). Stereo disparity improves color constancy. Vision Research, 42, 19791989.[PubMed] Zaidi, Q. (1998). Identification of illuminant and object colors: Heuristic-based algorithms. Journal of the Optical Society of America A, 15, 1767-1776.[PubMed]