Why do different observers see extremely different colours in the same photo?
by Christoph Witzel
!!! Please consider doing our ONLINE SURVEY (only 2min) on the dress !!!
Summary. A photo of a dress has become famous because it has led to extremely strong disagreements between individuals about the colour of that dress, and provoked vibrant discussions around the world.[1] The main reason for these disagreements is that colours of objects are perceived relative to the colours of illuminations, and the cues about the illumination of the dress are ambiguous in that photo. What is so fascinating about this particular photo is a combination between ambiguity and realism: On the one hand, the photo displays a degree of ambiguity about the illumination that is almost never found in reality. On the other hand, the photo is realistic enough to convince observers that it depicts a real dress under a real illumination. As a result of this realism, observers automatically interpret the illumination in one or the other way so as to infer the colour of the real dress. Since observers are not aware of the relativity of object colours to illuminations they do not consider that the colour of the dress depends on their unconscious assumptions about the light that illuminates the dress. Due to this unawareness of colour relativity, observers are shocked when they realise that other observers see the dress in this photo in utterly different colours.
What is the problem?
Consider Figure 1. What’s the colour of the dress? Some say the stripes of the dress are blue and black; others insist that they are white and gold; still others might even see them as light blue and brown! What is interesting about this observation is that blue and white as well as gold and black are sufficiently different not to be attributed to differences in using colour terms to describe these colours. Instead, they indicate a very strong difference in colour and gloss appearance across different observers. Hence, the question is why do different observers see mainly two (maybe three) fundamentally different sets of colours in the stripes of the dress?
Figure 1. Photo of the dress discussed in social media. The colours of the stripes look utterly different to different observers.
What is the real colour of the dress?
The answer to this question depends on what is meant by “real colour” (cf. Figure 2). Is it the colour of the actual dress independent of this particular photo? Or is the real colour the one of this particular image of the dress in that photograph?
Figure 2. A realistic and a manipulated photo of a banana. Is the “real colour” of the blue banana on the right yellow? It depends…
The colour of the real dress. The actual dress is blue and black. This is very clear on any other photo of the same dress (cf. Figure 3). However, that photo in Figure 1 is distorted in several ways, and does not reliably represent the real dress. As a result, there is no way of saying that the colours of the actual dress are the “correct” or “true” colours of the dress in the photo. For example, if you changed colours of a photo through Photoshop you would also not say that the colours of the original, real object (yellow banana in Figure 2) are the true colours of the image on that manipulated photo (blue banana in Figure 2). The problem is the same in the case of the photo of the dress. The only difference is that the photo of the dress has not been intentionally, but accidentally distorted. Hence, the colours of the actual dress are in no way a reference for evaluating whether the colours perceived on the photo are correct.
Figure 3. Undistorted image of the same dress.[2] It shows that the real dress is black and blue. So what?
The colours of the image. When isolating the colours of the dress from the photo and showing them as uniform patches, they look like greyish (desaturated) blue and brown (cf. Figure 4).[3] However, only very few people see the colours as light blue and brown. Depending on the survey, about 50-75% judge the dress as being white and gold, about 25-50% think it is blue and black, and an only very small percentage thinks it is light blue and brown. This does not mean that the majority of people cannot perceive the true colours. On the contrary, the divergent interpretations of the dress colours are a direct consequence of how colours are perceived.
Figure 4. Colour samples from the dress. These colours correspond to a random selection of single points (pixels) from the dress in Figure 1. However, they are neither blue-black, nor white-gold, but light blue and brown. These colours are only perceived if the observer does not assume an illumination, for example because they are not convinced about the realism of the photo.
How do we perceive colours?
Colour constancy. The perception of colour is based on the properties of the light (wavelength composition) that reaches the eye. In the case of object colours, these properties depend on both, the surface of the object (the dress) and the illumination (sunshine, lamps etc.). Nevertheless, we are able to perceive the colours of objects as more or less stable even when the illumination changes, for example due to changes in sunlight during the day. This ability to perceive the colours of object surfaces as constant across changes of illumination is called “colour constancy” (cf. Figure 5).
Relativity of colour. Colour constancy requires that there are cues about the illumination in the “scene” that is a particular situation or an image. One of the most important cues about the illumination is the background and the larger surrounding of an object in a scene. Colours are perceived relative to their background and surroundings. The overall colour of background and surround provides information about the illumination that helps to disentangle which colour shades in a perceived scene are due to the object surface or to the illumination (through mechanisms of adaptation, local contrast etc.). The relativity of colour to the background is nicely illustrated by Lotto and Purves colour cube(s),[4] or in the case of lightness, by Adelson’s checkerboard.[5] Apart from background and surround, there are still other cues about the illumination, such as the colour of white surfaces or gloss, which completely reflect the illumination, or the colours of objects, whose typical colour is known, such as the yellow of a banana.
Figure 5. Illustration of colour constancy and relativity. The two photos show the same scene under red (left) and green illumination (right). The line connecting the grapes with the orange has a uniform colour, and hence illustrates the relativity of perceiving this colour depending on the interpretation of the illumination. The patch in the foreground also has the same colour, but does not look so different in the two images because it does not appear as being part of the scene.
Where does gold come from?
In general, gold is perceived when brown-yellow surfaces have strong gloss, such as the golden ball in Figure 6. Single points of this ball are beige or brown (upper row), very similar to the brown colour samples of the dress (lower row). Due to the highlight of the ball they appear golden. The dress also has highlights. They might be artificially produced by a flash and/or overexposure, or be a particular effect of the fabric. In any case, they create the impression of gold instead of simply brown as it would be the case without highlights.
Figure 6. Gold = Brown with gloss. The upper row shows patches with colours from sample pixels from the image of the golden ball on the left. The lower row reproduces the colour samples from the “golden” stripes of the dress (cf. Figure 5).
How can the dress be perceived in different colours?
In reality there is – most of the time – sufficient information about the illumination that allows for separating both factors, surfaces of objects and illuminations. This also works in many photographs or realistic images even though cues about the illumination are reduced to a smaller cut-out of the scene, and to two-dimensional information. These cues may vary depending on the image.
Ambiguity of the image. Now, this particular photo of the dress is highly ambiguous about how the dress is illuminated. In particular, the dress might be illuminated by the same illumination as the background. Alternatively, it might be illuminated by another illumination in the foreground (or hanging in the shadow, but “hanging in the shadow” results in the same effect as another illumination since it “filters” light).
Sources of ambiguity. There is actually a long list of ambiguous cues about illumination in this image. The photo is overexposed so that some pixels needed to be clipped to remain within the gamut of the sensors of the camera. As a result, not all areas in the picture correctly show the actual colour of the light reaching the sensors of the camera. Moreover, the position of the dress relative to the background plays a role. From the photo, it is not clear whether the dress is part of the background illumination, or is hanging in a separate space in the foreground. The way in which the photo is taken seems to result in particular effects of lustre and gloss, which may be due to the perspective of camera or an effect of flash and overexposure. Additionally, the material and colours of the dress might be particularly prone to produce strong shifts in colour and gloss appearance depending on the illumination. In particular, the colours of the dress coincide with colours that may be attributed to sunlight (i.e. they are lying on the daylight axis). For this reason, it is particularly difficult to determine which colours come from the daylight illumination and which from the dress.
As a result of all these ambiguous cues, there is no clear information in this particular image on whether the dress is bathed in the yellowish light of the background, or hanging in another, more bluish illumination in the front of the image, for example due to a shadow. This seems to be the fundamental ambiguity of this image, and it is to a large extent due to the fact that it does not show natural colour distributions, but artefacts that make it more ambiguous than any real scene or realistic image thereof.
Disambiguation. That the perceived colour of the dress depends on the interpretation of the illumination may be shown by “disambiguating the illumination”. For example, the appearance of the dress’s colours may be shifted to one (e.g. gold-white), or another set of colours (blue-black), by putting the dress in to a scene where the illumination is clearly defined. This is illustrated by Figure 7. In particular, showing the dress on a person in a bluish-dark shadow makes the dress clearly appear gold-white. In contrast, showing the same dress on a person in direct sunlight makes it appear blue-black, or at least blue-brown.
Figure 7. Disambiguation of illumination cues. The woman in these pictures is clearly standing in a shadow (left side), and in overexposed direct sunlight (right side). Look at the left picture when covering the right one with your hand, then look at the right one, while covering the left picture. The dress is the same as in Figure 1. Does it look different in the two images?
Interpretation. It is known that colour perception directly depends on the background and global surround of a coloured object, which is called “colour contrast”. For example, the colour shifts in Purves’ cube above may be explained – at least partly – through colour contrast. However, this is not the case for the dress. The difference in colour appearance of the dress cannot be due to colour contrast, since these differences occur for the same photo, and that is for the same background. Instead, the appearance of this dress depends on the observer’s interpretation that is how they make sense of the scene in the photo.
To illustrate this idea in Figure 6, the model in the right image is standing in front of a cave, so that the background directly adjacent to the dress is at least as dark as the background in the left image. Nevertheless, the dress appears to be blue, because the position of the model suggests that she is standing in bright sunlight.
The uniqueness of this photo. The reason why this photo of a dress is so fascinating is that it is very rare that an image is that ambiguous while still being sufficiently realistic to produce colour constancy. In order to achieve this ambiguity, the picture needs to provide at the same time convincing cues for the interpretation of one and the other illumination, while still giving the impression of being a photo of a real object.
A lack of realism prevents or at least reduces an interpretation that allows for colour constancy. For example, the patch in the foreground of the images in Figure 5 has the same colour (i.e. RGB values) as the grapes and the orange in the left and right images. However, it does not look realistic. As a result it is not subject to colour constancy, and looks the more similar on both sides of the image than do the grapes and the orange (Some residual differences in appearance may be due to colour contrast).
Moreover, Figure 8 shows the colours of the dress in a mosaic that coarsely reproduces the structure of the photo, but does not look like a realistic photo at all. In particular, the colours in that photo do not appear like illuminated surfaces in a real scene, but like a self-contained, artificial configuration of colour patches. As a result, the ambiguity breaks down, and the colours in Figure 8 simply look like the isolated patches in Figure 4. These example illustrate that an image must give the impression of being realistic in order to be interpreted in terms of colour constancy. The many cues that contribute to the realism of an image make it difficult to create or reproduce a picture that is realistic for two fundamentally different interpretations.
Figure 8. Mosaic version of the dress. The mosaic effects alienate the image of the dress so that they preclude the perceptual interpretation that the image shows a real object under a particular illumination. Hence, the ambiguity breaks down and the colours are perceived like isolated patches (cf. Figure 4).
Why are individual differences so shocking?
Individual differences. In order to make sense of this original image, an observer needs to make an assumption about whether the dress is illuminated by the yellowish background illumination, or by a second bluish illumination. At the same time, laymen (in contrast to colour scientists) are not aware of the fact that the colours of an image depend fundamentally on their interpretation of the conditions of illumination. For most observers, the dress has a real colour (as it is the case for the real dress), and the illumination is a separate issue, not connected to the dress since they are different things. For this reason, most observers will not critically interrogate what the role of the illumination is when judging the dress. In fact, they will automatically recur to one or the other possible assumption about the illumination in that photo, and judge the colour of the dress based on these assumptions. Since they do not interrogate these perceptual assumptions, they will be firmly convinced that their interpretation of the dress is the only possible, “correct” interpretation.
The surprise effect. A particular effect of this image is the surprise, or even shock, about the disagreement with other observers. Observers cannot believe that another observer can perceive the dress in such a fundamentally different colour. The crucial factor behind this “surprise effect” is that observers, and in particular lay observers (that is non-colour scientists or professionals) are not aware that they made these assumptions when looking at the image. There are three important reasons for the absence of this awareness.
First, the assumptions about the illumination(s) are made automatically when looking at the image, i.e. they are “unconscious inferences”. Second, in everyday life experience the (surface) colour of an object is independent of its background and of the illumination. For this reason, lay observers will not consider the background or the illumination as being relevant for the judgment of the colours of the dress. Third, once observers assume a certain interpretation of the illumination it will “work” with that image, i.e. it will make sense of it. As a result, the observer will firmly believe that their interpretation of the colour is the truth, and corresponds to the colour of the dress in reality. There is no reason to interrogate this believe since this interpretation works for almost all photos in everyday life.
For all these reasons, observers will not consider their implicit assumptions about the illumination when being confronted to another observer who completely disagrees about the colour of the dress.
Irreversibility of interpretation. Only a few observers may change their interpretation of the image when they know about the two possible ways to interpret the illuminations. Most observers are stuck with their initial interpretation – even when they know about the two possible ways to interpret the illumination. For this reason, some readers may even see the dress in the same colour in the two images of Figure 7 (another reason could be that my photo collage might not be as realistic as necessary; see above).
This irreversibility of interpretation may be explained by the fact that both of the two possible interpretations of that photo make sense, and lead the observer to perceive that their interpretation corresponds to the true colour of the dress as it is in reality. If an observer once perceives an image that is so sensible that it appears to be true, it is difficult to change this interpretation. Imagine somebody gives you a thing that looks like a banana, smells like a banana, and tastes like a banana. This person will probably have a hard time to convince you that this is not a real banana.
The irreversibility of consistent interpretations is further illustrated by the famous Dalmatian and Jesus images.[6] In the Dalmatian image, only dots, but no contours are visible, and initially observers do not see anything but random dots. However, once they understand that the dots correspond to the pattern of a Dalmatian dog observers cannot prevent themselves from seeing the dog in that image. They cannot see the image as random dots anymore because they have a way to make sense of it. In a similar way, the initial interpretation of the photo of the dress makes sense and hence seems to be true. This explains why this interpretation is resistant to the alternative interpretation.
The third set of colours. In reality, the observers’ ability to judge object colours independent of the colour of the illumination is possible because they make those implicit assumptions about the illumination which allow for colour constancy (see above). However, colour constancy is only possible if the observer really considers the image to show a real object under a realistic illumination. The aforementioned technical artefacts in the photo may also potentially prevent observers to assume any kind of illumination, and consider the colours of the dress in that photo as independent of any illumination. Then, they appear as what they are when the colours of the dress are isolated from the photo and shown as uniform patches. In this case, the stripes look as light blue and brown (cf. Figure 4 and Figure 8).
This is indeed a third possibility of seeing the colours of the dress. Observers who do not assume any illumination may have this perception of colours of the dress. This may happen, for example, when they are not convinced that this image is a realistic photo of a dress under an illumination because they are aware of unrealistic artefacts. Colour scientists or other professionals may be a group that is particularly prone to see this set of colours because they are particularly sensitive to the interplays between illuminations and object colours, and hence of the lack of information about the illumination.
What about alternative explanations?
The factors described above are known to play a crucial role in colour vision. However, in the case of this particular photo, still other factors might also play a role. While the final answer about the precise role may not be given for each candidate factor, we can exclude a potential role for some of them.
Technical reasons. Different observers looked at the photo on different, non-calibrated screens. As a result, colours in that photo are differ across screens because monitors have different properties, and even different settings (Brightness, contrast, colour temperature settings etc.). Moreover, colours may change depending on the perspective in which observers look at the screen. This is particularly true for modern LCD screens. However, such technical variations seem not to be a good explanation because observers even disagree about the colours of the dress when they look from the same perspective at the photo displayed on the same screen.
Colour naming. Observers differ considerably in the way they describe colour through colour names when these colours are ambiguous examples for the respective colour term. For example, an observer might call a greenish-bluish shade “green” that is called blue by another one. If the differences in dress descriptions would only be due to individual differences in colour naming, the same disagreements should occur for the coloured patches in Figure 4 – but this is not the case. Moreover, it is unlikely that colour naming combines with effects of colour constancy. Black and gold are not adjacent in colour space, and the perceived shade of blue seems to be so unambiguous that it cannot be mixed up with white. Moreover, gold fabric is glossy in a way that black fabric cannot be. Hence, it is unlikely that differences in describing the colours of the dress are due to labelling the same colour by different colour terms.
The Eye. Colour deficiencies and colour blindness might be understood as an extreme case of individual differences in the sensitivity to colour. However, disagreements about the colour of the dress also happen for observers that are all not colour deficient. Moreover, the low frequencies of colour deficiencies (<10%) do not match the frequencies for either of the two main sets of perceived colours (blue/black and white/gold). Note that the frequency of women with four instead of three photoreceptors (tetrachromacy) is still lower, which completely undermines the idea that tetrachromacy might play a role in the effects of that image.
Apart from colour deficiencies and tetrachromacy, the properties of the eyes that are relevant to colour vision (optical filtering, sensitivity of photoreceptors, relative density of photoreceptors) may vary considerably across individuals. However, the ultimate perceptual appearance of a colour does not systematically depend on those variations. Instead, it is calibrated through the observers’ experience. For example, with sustained experience observers adapt their perception of pure white and pure yellow when the light entering the eye is altered, for example by wearing coloured goggles, or after a cataract surgery. Since those differences in the properties of the eye are compensated through experience and interaction with the visual environment, it is unlikely that they can explain the differences in perceiving the colour of the dress in that photo.
The brain. Apart from the eye, the brain plays a major role for enabling perception. Hence, individual differences in seeing the colour of the dress could potentially be explained by individual differences in “how the brain is wired”. Certainly, any difference in perception is likely to be related to some differences of “what happens in the brain.” This is a rather trivial observation that does not provide any particular explanation about the individual differences in perceiving the colour of the dress. At the same time, the “wiring of the brain” might suggest that there are different kinds of brains, which differ in terms of their genetically determined or otherwise predefined structure. However, there is no reason to believe that this is the case in a way that is relevant to colour constancy. It is also possible that the interpretation of the illumination may be shaped by all kinds of experience rather than being biologically predetermined. The precise contributions of “nature” and “nurture” to the perception of this photo and to colour constancy in general are an open question.
What are the take home messages?
In sum, the cues about the illumination in that photo of a dress are ambiguous. Observers make implicit assumptions about the illumination in order to perceptually interpret the photo as a representation of a real object. Without being aware about the ambiguity of the image and the complex interplay between object colours and cues about illuminations, observers may not understand how other observers may see the dress in different colours. As a result, they are shocked when meeting other people that see utterly different colours on that dress based on different assumptions about the illuminations. This photo is so fascinating because it combines realism and ambiguity in a way that is difficult to reproduce. From a more general perspective, this photo nicely illustrates how the observer’s implicit assumptions shape their perception.
