Category Archives: S3D

Is there a secret problem with depth maps?


It sounds like a great idea to utilize a depth map to extract information and control the depth depicted in an image or series of images. It sounds great for converting a 2D image into a 3D image. It sounds like a great tool for plenoptic cameras to interpolate the data into imagery with depth. Alpha channels are great to use for transparency mapping – so a depth map should be equally useful, shouldn’t it?

Take a look at this depth map:

icedepthmapThis is a depth map created from a plenoptic camera shot of a bunch of ice bits. It is a grayscale image with 256 shades of gray to depict the parts of the ice that are closer to the camera and the parts of the ice that are farther away from the camera. This information is used to adjust the depth of those bits that are closer and farther away by stretching or compressing pixels.

Now check out a rocking animation that uses motion parallax to depict the depth (items closer to you appear to move differently than items that are farther away).

ice

Right away you can notice a few errors in the depth map, and for complex images this is typical and can be edited and “corrected”. But there is something else. Take a close look at the parts of the image where the depth map is seemingly correct. Sure, you can see the depth but does it really look like ice? If you are like me, the answer is no. Ice reflects and scatters light in a way that is unique for each perspective. Indeed, there IS binocular rivalry where one eye sees light reflection and distortion that is not present in the other eye’s perspective. This disparity tells us something about the texture and makeup of what we are looking at. Stretching or compressing pixels eliminates this information and only provides depth cues relating to the spatial position of things. For most people, I suspect it is reasonable to assume that this creates a perception conflict in their brains. There is something perceptually wrong with the image above. It does not look like ice because the light coming off of the two perspectives looks the same. A depth map does not provide information regarding binocular rivalry and creates errors as a result. Errors that can’t be fixed. Herein you see the flaw in using a depth map. It throws away all of the binocular rivalry information. In other words, it throws away the information between perspectives that is different.

In my opinion, depth maps take the life out of an image. It removes important texture information which, I believe, is gleaned from how light shifts and changes and appears and disappears as you alter perspective.

This is the secret fundamental flaw with depth maps. Now you can subjectively look at the image above and deem it to be cool and otherwise amazing. That is all good and well, but the truth is that, compared with looking at the real ice, it is fundamentally lacking and does not depict what is seen when you look at the ice in real life.

So, people ask themselves if this is important and some will say yes and some will say no. And there are many examples where you could argue both points of view. I don’t have an argument with that. My position is only to point out that this flaw exists and it should not be ignored.

Advertisements

1 Comment

Filed under 1, 3D, 3D Photography, autostereoscopic, S3D, stereopsis, stereovision

I am presenting a paper at SPIE January 25, 2011 at 5:30 PM Paper 7863-49


SPIE (the International Society for Optical Engineering)  See:
http://spie.org/x16218.xml is holding a conference on 3D imaging from Jan. 23 – 27 in San Francisco, CA. My paper and presentation: “Human perception considerations for 3D content creation” is about the problem of perception conflicts as they relate to 3D imagery and what to do about them.

I first started thinking about this when I saw an old lenticular photograph of Queen Elizabeth. The photograph could be viewed with stereopsis but the Queen looked like she was dead. Watching the movie Beowulf, while not in 3D, also gave me the creeps as the characters had a dead aspect to them. I noticed some 3D lenticular photographs of people presented with a doll-like character. I then started to notice things in 3D movies that didn’t seem right. When details disappeared into blackness or got blown out to white I noticed an uneasy feeling while looking at that part of the 3D presentation.

Indeed, every time something was presented in 3D that was atypical or not possible to see in the real world, I could detect a feeling of conflict present at some level in my subconsious and I started to manifest a sensitivity to it with regards to recognizing when it was happening.

All of these observations got me thinking about the various mechanisims that we use to see and interpret depth, space and texture. Certainly vergence is the primary mechanism, but as I became more aware of supporting clues like accommodation, motion, luminance dynamic range, binocular rivalry, field of view and so on, I came to a realization.  I realized that when non-vergence depth clues weren’t complementary that those elements or perceptions in conflict required suppression to continue viewing without some sort of physical effect occurring (typically unpleasant such as headache, nausea, etc.).

My paper is a start to the investigation of the importance of supporting perception cues as it relates to stereovision.

*Vergence is the simultaneous movement of both eyes in opposite directions to obtain fixation and the ability to see depth.

*Accommodation is the automatic adjustment in the focal length of the lens of the eye to permit retinal focus of images of objects at varying distances. It is achieved through the action of the ciliary muscles that change the shape of the lens of the eye.

Leave a comment

Filed under 3D, 3D HDR, 3D Health Issues, 3D Motion Picture, 3D Photography, autostereoscopic, binocular disparity, binocular rivalry, HDR, High Dynamic Range, Perception Conflicts, S3D, stereopsis, stereovision

Perspective Interpolation – Specularity and Refraction Problems


So, how about converting 2D to 3D or converting two perspective 3D into multi-perspective autostereoscopic… Technology certainly should easily make that possible, right?

The answer is a bit complicated. Because for some images it is quite possible to achieve excellent results. Unfortunately, for many images and scenes it truly is impossible to create accurate 3D from 2D and/or interpolate additional perspectives for autostereoscopic displays.

Case in point? Look at the animation below:

In the background painting there are tiny bits of highly reflective particles embedded in oil paint. These dots of light reflect bright points of light depending upon the perspective. They “come on” quickly as you change perspective because of the paint occlusion where you see them in one eye but not the other. Any program that interpolates views would not know what to do with a picture like this. Morph the dots of light? In real life, they don’t morph, they pop on with the light brightening as the perspective angle changes.

Now, take a close look at the glass gems. Notice how their specularity is influenced by the perspective position relative to the background?  Notice the refraction as you see the background through the transparent glass. Unless you modeled the gems in a 3D program and rendered them, there would be no way to interpolate with a pixel warping program what is going on with the look and texture of these gems as they change perspective.

What happens typically with a conversion is an abysmal mess for items with specularity and refraction. It looks 3D for sure – but in no way is representative of reality. And this is the conundrum. There is no uniformity or consistency with regards to 2D to 3D conversions or 2 perspective to multi perspective conversions. It is completely content based and the results are dependent upon the subject matter.

Binocular disparity and as this example demonstrates, binocular rivalry where one perspective contains elements not visible in the other perspective create monumental problems for conversion.

The solution? Shoot multiple perspectives. And this is the path that I have been forced to take to create consistent and uniform results. Indeed, fewer than 10 perspectives does not yield quality, uniform results in my humble opinion. Can fewer than 10 perspectives work? The answer is yes if what you are photographing has no specularity or refraction properties and the texture is smooth and uniform. But as an artist, I find that restriction way to limiting and live in a world that consists mostly of refractive material (water) and glass and gems and metals. Indeed, just look around and the world is filled with specular and refractive content.

Even portraits pose a problem because unless the person has extremely dry eyes, they glisten as the moisture that coats the eye creates specularity and refraction. Of course, if you don’t have a close up or reduce the resolution then it isn’t that noticeable. But here again, as an artist I find that too limiting.

I do not understand the willingness of people to ignore these problems. While it is true that in many cases specularity and refraction are subtle and nuanced. But given that 3D mimics the way we see real life, shouldn’t 3D be subtle and nuanced? Perhaps the gross over emphasized poke you in the eye effects are doing the potential of 3D a disservice?

That’s my view. But what do I know?

3 Comments

Filed under 3D, 3D Photography, autostereoscopic, S3D, stereopsis

The mediocrity of most 3D lenticular prints is so realistic that it leaps off the image and completely ruins your day!


What does a person say when they are presented with a 3D image where the only redeeming quality of the image is that it is in 3D? I find it disheartening to see so much BAD 3D content. Indeed, my own work could stand improvement – but when I look at what is out there I really cringe.

I think this is a big part of the problem with the acceptance of 3D imagery. There just isn’t a high level of quality. So many are focused on the effect of 3D that they completely ignore things like good image composition, lighting for 3D, telling a story within the photograph and just creating something that is compelling to look at and study.

A good 2D photograph does not a good 3D photograph make!  With a 2D photograph you use lighting and perspective to create a sense of depth and drama. It is the abstraction that stirs the interpretation of the image. Everything about a regular photograph (2D) is processed in the brain through interpretation. If emotion is evoked from looking at a 2D photograph it is through the process of drawing upon what we have seen before and making comparisons of a sort. We make the connection through interpretation and referencing it to other things we have seen.

This dynamic falls apart with 3D imagery. For most 3D images, you notice the “gimmick” of 3D. “Look! Something is sticking out of the picture!”, “Wow, isn’t that amazing!”, etc. But like all gimmicks, they get old fast. When people try to create a 3D image using the same process as that for 2D images a problem surfaces. When we see with stereopsis (3D) we are engaging a part of human perception that relates to seeing things that are real and have a sense of occupying space. Our brain doesn’t try to interpret, but rather seeks to first experience and evaluate what we are seeing. Our brain asks: Is this image real? And then it starts a process to determine the realness which for most 3D imagery takes about a second. So, the first impression is negative (not real) and the level of interest typically fades in a dramatic fashion. “You’ve seen one 3D image, you’ve seen them all”. 

This process doesn’t exist for 2D imagery as there is no expectation that it could be real.

So, two solutions exist to the above problem. Option one would be to make every effort to have the 3D photograph appear as real and accurate as possible. Option two would be to find a way to make it obvious that the image isn’t real and have the brain process the image as referential just as it does with a 2D image.

Both options are extraordinarly difficult to achieve. And herein lies the artistic challenge of 3D imagery. My approach, to date, has been option one because I believe that until one throughly understands how to perfect option one — that option two will be significatly more difficult. I am experimenting with all aspects of multi perspective photography with the goal of achieving a type of viewing engagement that is perceived with “realness”.

Next up for blogging?  How to achieve “realness” with 3D photography.

Leave a comment

Filed under 3D, 3D Photography, autostereoscopic, S3D, stereopsis

Avatar 3D VFX Supervisor Evaluates Almont Green 3D Photo


Sometimes, it is a good idea to get a reality check and see if what you think is “good” is really good. I have tried to get as many samples as possible and see what 3D photography is out there. 

So far, I haven’t been able to find anything that I felt was good.  So I decided that perhaps I should find someone at the top of their game and send them one of my 3D photographs to see how my work stacks up.

I sent a 3D photograph to Chuck Comisky. His credits include:
3D Stereo VFX Supervisor for the motion picture AVATAR
S3D and visual effects for Ghosts of the Abyss and Aliens of the Deep IMAX 3D documentaries
T2-3D Battle Across Time theme park attraction producer
The list of producers and major studios he has worked with is like reading a list of who’s who in the motion picture industry (ILM, Disney, Fox, Sony/Columbia, Paramount, Universal…)

Today, I received an email from Mr. Comisky. He wrote:

“Yes, I did get it [sample 3D autostereoscopic photograph]. It is as stunning a stereo photo using lenticular as I have seen. It visually rivals holography in clarity… Keep up the good work.”

Leave a comment

Filed under 3D Photography, autostereoscopic, S3D

3D – Present A Different Image To Each Eye – Simple, Right? Wrong!


Those of us with “normal” vision see the world with our two eyes in 3D all of the time. We see the space between things and perceive distance, size, texture, etc.

It’s different when you go to a 3D movie or look at one of my autostereoscopic photographs because you are looking at a flat surface and perceive 3D by way of an optical illusion. There are many subtle differences (and many not so subtle) between normal “seeing” and watching a 3D movie that are important to understand and consider. The biggest difference is that for normal viewing the 3D is “real”. You see an object in front of another object and you perceive the distance. When you reach out to touch the object, that perception is verified. A 3D movie on the other hand is created with an optical illusion. It is made possible because our eyes/brain has an amazing capability to decouple focus from convergence and see and perceive the illusion as if it were real.

Some of you are asking, “What does decouple focus from convergence mean?“.

For normal viewing our eyes focus and converge on the same point in space (the thing we are looking at). Just like separate camera lenses, each eye focuses on an object independently. Then our brain processes those two retinal images into a single image with depth. Because the focusing and merging is done separately, we can perform a trick whereby we fool the brain into processing retinal images that converge at a different point from focus. The only thing the brain cares about is the alignment, size and similarity of the images on each retina. It doesn’t matter what the focusing distance is. Our brain just totally disregards this disparity… or does it? Since most people seem to be able to perceive 3D looking at 3D movies, we just make that assumption. My guess is that our brain does enable our perception of focus distance but that we suppress the conflict that happens with a 3D movie in the same way we suppress other perception conflicts (think flying in an airplane for example where our inner ear conflicts with what we see). Perhaps some of us aren’t able to suppress this conflict as easily and manifest some sort of discomfort with the experience; like getting a headache or feeling nausea.

However, it is generally thought of as a “given” that this isn’t a big deal and that perception conflicts occur in nature and the brain just “handles it”. I think that is probably true – but I’m not a scientist or doctor and it would be nice to read that my opinion has some basis in true science.  Perhaps it does and someone will comment?

You still might be wondering how a 3D movie decouples focus from convergence. The fact is that it has to do that. The motion picture screen has to be the point of focus at all times. That is the source of the reflected light and the focus point of the projector in the back of the theater. The left eye and right eye images will be offset from each other based upon whether the objects depicted are in front of the screen or going into the screen. The eyes converge or diverge to align the objects on screen but the point of focus is always the screen surface. At this point, I just have to jump up and down and say THAT IS PRETTY AMAZING!!!  Our brain is really incredible and adaptable in a way that we don’t even have to think about.

Stay tuned for more… coming soon ;^)

3 Comments

Filed under 3D, 3D Motion Picture, S3D, stereovision

Stereoscopic Multi Perspective Cross View Example


Below is an example that shows pairs of perspectives in an animated presentation. The left basketball is setup to be viewed by your right eye and the right basketball should be viewed by your left eye. The two tiny gray squares at the very top are used to help you cross your eyes. What happens is as you cross your eyes the dots will move inward and when they are on top of each other your eyes will be crossed in a way that aligns the two different images to a point where you can fuse them into a single view with depth. It can be tricky to do and many people have a great deal of difficulty but I encourage you to give it a go.  Of course, my autostereoscopic 3D photographs don’t require you to cross your eyes and you see the hoop stick out of the photograph in a natural way.

1 Comment

Filed under S3D, stereovision