Tag Archives: binocular rivalry

Is there a secret problem with depth maps?


It sounds like a great idea to utilize a depth map to extract information and control the depth depicted in an image or series of images. It sounds great for converting a 2D image into a 3D image. It sounds like a great tool for plenoptic cameras to interpolate the data into imagery with depth. Alpha channels are great to use for transparency mapping – so a depth map should be equally useful, shouldn’t it?

Take a look at this depth map:

icedepthmapThis is a depth map created from a plenoptic camera shot of a bunch of ice bits. It is a grayscale image with 256 shades of gray to depict the parts of the ice that are closer to the camera and the parts of the ice that are farther away from the camera. This information is used to adjust the depth of those bits that are closer and farther away by stretching or compressing pixels.

Now check out a rocking animation that uses motion parallax to depict the depth (items closer to you appear to move differently than items that are farther away).

ice

Right away you can notice a few errors in the depth map, and for complex images this is typical and can be edited and “corrected”. But there is something else. Take a close look at the parts of the image where the depth map is seemingly correct. Sure, you can see the depth but does it really look like ice? If you are like me, the answer is no. Ice reflects and scatters light in a way that is unique for each perspective. Indeed, there IS binocular rivalry where one eye sees light reflection and distortion that is not present in the other eye’s perspective. This disparity tells us something about the texture and makeup of what we are looking at. Stretching or compressing pixels eliminates this information and only provides depth cues relating to the spatial position of things. For most people, I suspect it is reasonable to assume that this creates a perception conflict in their brains. There is something perceptually wrong with the image above. It does not look like ice because the light coming off of the two perspectives looks the same. A depth map does not provide information regarding binocular rivalry and creates errors as a result. Errors that can’t be fixed. Herein you see the flaw in using a depth map. It throws away all of the binocular rivalry information. In other words, it throws away the information between perspectives that is different.

In my opinion, depth maps take the life out of an image. It removes important texture information which, I believe, is gleaned from how light shifts and changes and appears and disappears as you alter perspective.

This is the secret fundamental flaw with depth maps. Now you can subjectively look at the image above and deem it to be cool and otherwise amazing. That is all good and well, but the truth is that, compared with looking at the real ice, it is fundamentally lacking and does not depict what is seen when you look at the ice in real life.

So, people ask themselves if this is important and some will say yes and some will say no. And there are many examples where you could argue both points of view. I don’t have an argument with that. My position is only to point out that this flaw exists and it should not be ignored.

1 Comment

Filed under 1, 3D, 3D Photography, autostereoscopic, S3D, stereopsis, stereovision

I am presenting a paper at SPIE January 25, 2011 at 5:30 PM Paper 7863-49


SPIE (the International Society for Optical Engineering)  See:
http://spie.org/x16218.xml is holding a conference on 3D imaging from Jan. 23 – 27 in San Francisco, CA. My paper and presentation: “Human perception considerations for 3D content creation” is about the problem of perception conflicts as they relate to 3D imagery and what to do about them.

I first started thinking about this when I saw an old lenticular photograph of Queen Elizabeth. The photograph could be viewed with stereopsis but the Queen looked like she was dead. Watching the movie Beowulf, while not in 3D, also gave me the creeps as the characters had a dead aspect to them. I noticed some 3D lenticular photographs of people presented with a doll-like character. I then started to notice things in 3D movies that didn’t seem right. When details disappeared into blackness or got blown out to white I noticed an uneasy feeling while looking at that part of the 3D presentation.

Indeed, every time something was presented in 3D that was atypical or not possible to see in the real world, I could detect a feeling of conflict present at some level in my subconsious and I started to manifest a sensitivity to it with regards to recognizing when it was happening.

All of these observations got me thinking about the various mechanisims that we use to see and interpret depth, space and texture. Certainly vergence is the primary mechanism, but as I became more aware of supporting clues like accommodation, motion, luminance dynamic range, binocular rivalry, field of view and so on, I came to a realization.  I realized that when non-vergence depth clues weren’t complementary that those elements or perceptions in conflict required suppression to continue viewing without some sort of physical effect occurring (typically unpleasant such as headache, nausea, etc.).

My paper is a start to the investigation of the importance of supporting perception cues as it relates to stereovision.

*Vergence is the simultaneous movement of both eyes in opposite directions to obtain fixation and the ability to see depth.

*Accommodation is the automatic adjustment in the focal length of the lens of the eye to permit retinal focus of images of objects at varying distances. It is achieved through the action of the ciliary muscles that change the shape of the lens of the eye.

Leave a comment

Filed under 3D, 3D HDR, 3D Health Issues, 3D Motion Picture, 3D Photography, autostereoscopic, binocular disparity, binocular rivalry, HDR, High Dynamic Range, Perception Conflicts, S3D, stereopsis, stereovision