Games for 3D glasses with XNA

Following on from my post about creating images for 3D glasses, here is how I modified my XNA game engine to render stereo images.

Previously my camera class simply set up the view and projection matrices based on various input. These matrices are then used by the other components in the game to draw themselves. I have now added an extra property to the class, AnaglyphType, to determine whether the scene is rendered as a left/right stereo pair. Also extra code is added to the game loop to draw the scene into two textures, then combine them with a pixel shader.


When stereo rendering is enabled, the scene must be rendered twice from slightly different viewpoints. This translates to setting a slighty different view matrix in the camera class. The orientation of the "eyes" is assumed to be orthogonal to the view direction and the "up" vector. Another variable, eyesSeparation, is used to determine the gap between the two view points. The bigger the gap the more "3D" it will look, but if you go too far your brain will get all confused and the effect will be ruined (plus it will probably give your players a bad headache!). It might be worth allowing the user to modify this value within the options menu.

The new view matrices can thus be calculated as follows:

Vector3 right = Vector3.Cross(LookAt - CameraPosition , UpVector);
right.Normalize();
if( AnaglyphType == AnaglyphType.LeftEye )
    viewMatrix = Matrix.CreateLookAt(CameraPosition - right * eyeSeparation / 2 , LookAt , UpVector);
if( AnaglyphType == AnaglyphType.RightEye )
    viewMatrix = Matrix.CreateLookAt(CameraPosition + right * eyeSeparation / 2 , LookAt , UpVector);
if( AnaglyphType == AnaglyphType.None )
    viewMatrix = Matrix.CreateLookAt(CameraPosition , LookAt , UpVector);

I automatically recalculate these whenever the AnaglyphType property is changed. This means that before a scene is rendered, you can just set the AnaglyphType property of the camera class to LeftEye, RightEye, or None, and the correct matrices will get set.

The next step is to modify your game loop to draw the scene twice, into two different textures. This is easily handled with calls to SetRenderTarget. First define two render targets in your game class:

RenderTarget2D leftEyeImage , rightEyeImage;

Create them in your Load method to exactly match the back buffer format. Then in your Draw call, when stereo view is enabled, you draw the scene twice, each time telling the camera class which eye you are drawing for. This assumes the other components in your scene will read the view matrix from your camera class somehow.

camera.AnaglyphType = AnaglyphType.LeftEye;
SetRenderTarget( 0 , leftEyeImage );
DrawScene(); // draw your scene here

camera.AnaglyphType = AnaglyphType.RightEye;
SetRenderTarget( 0 , rightEyeImage );
DrawScene(); // draw the scene again here

SetRenderTarget( 0 , null ); // reset back to original render target

Now we have two scenes rendered, we just need to combine them and draw them on the screen. This is done with a custom pixel shader and normal SpriteBatch calls. The important bit of the pixel shader is as follows:

float4 main(float4 color : COLOR0, float2 texCoord : TEXCOORD0) : COLOR0
   {
      float4 left = tex2D(LeftImage, texCoord);
      float4 right = tex2D(RightImage, texCoord);

      float leftL = left.r * 0.3 + left.g * 0.59 + left.b * 0.11;
      float rightL = right.r * 0.3 + right.g * 0.59 + right.b * 0.11;
     
      return float4( rightL , leftL , 0 , 1 );
   }

This is sampling the pixel colour from the left and right image, converting it to a greyscale luminance value, and then outputing the left image to the green channel and the right image to the red channel. Of course more sophisticated schemes could be used to better match the colour of the glasses, but this works for testing puposes.

Then in the game code, following on from the left/right rendering above, we need to do the following:

SpriteBatch.Begin();
// SpriteBatch sets the 0th texture for us, but we need to set the 1st
GraphicsDevice.Textures[1] = rightEyeImage.GetTexture();
AnaglyphEffect.Begin();
AnaglyphEffect.Techiques[0].Passes[0].Begin();
SpriteBatch.Draw(leftEyeImage.GetTexture(),Vector2.Zero,Color.White);
SpriteBatch.End();
AnaglyphEffect.Techiques[0].Passes[0].End();
AnaglyphEffect.End();

Note that you'll need to pay special attention to 2D components in your game, eg HUD information and menus. You could make these appear to float above the screen by reading camera.AnaglyphType and offsetting the graphics slightly if it's set to LeftEye or RightEye.

Even if your game engine is not designed exactly like mine, hopefully this will allow you to add 3D glasses capability to your game without the need to change much code.