Jeff Molofee - NeHe's OpenGL Tutorials

Calculating the equation of a plane looks ugly, but it is just a simple mathematical formula that you grab from a textbook when you need it.

void calculatePlane( const ShadowedObject& object, Face& face ) {

// Get Shortened Names For The Vertices Of The Face

const Point3f& v1 = object.pVertices[face.vertexIndices[0]];

const Point3f& v2 = object.pVertices[face.vertexIndices[1]];

const Point3f& v3 = object.pVertices[face.vertexIndices[2]];

face.planeEquation.a = v1.y*(v2.z-v3.z) + v2.y*(v3.z-v1.z) + v3.y*(v1.z-v2.z);

face.planeEquation.b = v1.z*(v2.x-v3.x) + v2.z*(v3.x-v1.x) + v3.z*(v1.x-v2.x);

face.planeEquation.c = v1.x*(v2.y-v3.y) + v2.x*(v3.y-v1.y) + v3.x*(v1.y-v2.y);

face.planeEquation.d = –( v1.x*( v2.y*v3.z – v3.y*v2.z ) + v2.x*(v3.y*v1.z – v1.y*v3.z) + v3.x*(v1.y*v2.z – v2.y*v1.z) );

}

Have you caught your breath yet? Good, because you are about to learn how to cast a shadow! The castShadow function does all of the GL specifics, and passes it on to doShadowPass to render the shadow in two passes.

First up, we determine which surfaces are facing the light. We do this by seeing which side of the plane the light is on. This is done by substituting the light's position into the equation for the plane. If this is larger than 0, then it is in the same direction as the normal to the plane and visible by the light. If not, then it is not visible by the light. (Again, refer to a good Math textbook for a better explanation of geometry in 3D).

void castShadow( ShadowedObject& object, GLfloat *lightPosition ) {

// Determine Which Faces Are Visible By The Light.

for ( int i = 0; i < object.nFaces; i++ ) {

const Plane& plane = object.pFaces[i].planeEquation;

GLfloat side = plane.a*lightPosition[0]+ plane.b*lightPosition[1]+ plane.c*lightPosition[2]+ plane.d;

if (side > 0) object.pFaces[i].visible = true;

else object.pFaces[i].visible = false;

}

The next section sets up the necessary OpenGL states for rendering the shadows.

First, we push all the attributes onto the stack that will be modified. This makes changing them back a lot easier.

Lighting is disabled because we will not be rendering to the color (output) buffer, just the stencil buffer. For the same reason, the color mask turns off all color components (so drawing a polygon won't get through to the output buffer).

Although depth testing is still used, we don't want the shadows to appear as solid objects in the depth buffer, so the depth mask prevents this from happening.

The stencil buffer is turned on as that is what is going to be used to draw the shadows into.

glPushAttrib( GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_ENABLE_BIT | GL_POLYGON_BIT | GL_STENCIL_BUFFER_BIT );

glDisable( GL_LIGHTING ); // Turn Off Lighting

glDepthMask( GL_FALSE ); // Turn Off Writing To The Depth-Buffer

glDepthFunc( GL_LEQUAL );

glEnable( GL_STENCIL_TEST ); // Turn On Stencil Buffer Testing

glColorMask( GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE ); // Don't Draw Into The Colour Buffer

glStencilFunc( GL_ALWAYS, 1, 0xFFFFFFFFL );

Ok, now the shadows are actually rendered. We'll come back to that in a moment when we look at the doShadowPass function. They are rendered in two passes as you can see, one incrementing the stencil buffer with the front faces (casting the shadow), the second decrementing the stencil buffer with the backfaces ("turning off" the shadow between the object and any other surfaces).

// First Pass. Increase Stencil Value In The Shadow

glFrontFace( GL_CCW );

glStencilOp( GL_KEEP, GL_KEEP, GL_INCR );

doShadowPass( object, lightPosition );

// Second Pass. Decrease Stencil Value In The Shadow

glFrontFace( GL_CW );

glStencilOp( GL_KEEP, GL_KEEP, GL_DECR );

doShadowPass( object, lightPosition );

To understand how the second pass works, my best advise is to comment it out and run the tutorial again. To save you the trouble, I have done it here:

Figure 1: First Pass

Figure 2: Second Pass

The final section of this function draws one blended rectangle over the whole screen, to cast a shadow. The darker you make this rectangle, the darker the shadows will be. So to change the properties of the shadow, change the glColor4f statement. Higher alpha will make it more black. Or you can make it red, green, purple, …!

glFrontFace( GL_CCW );

glColorMask( GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE ); // Enable Rendering To Colour Buffer For All Components

// Draw A Shadowing Rectangle Covering The Entire Screen

glColor4f( 0.0f, 0.0f, 0.0f, 0.4f );

glEnable( GL_BLEND );

glBlendFunc( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA );

glStencilFunc( GL_NOTEQUAL, 0, 0xFFFFFFFFL );

glStencilOp( GL_KEEP, GL_KEEP, GL_KEEP );

glPushMatrix();

glLoadIdentity();

glBegin( GL_TRIANGLE_STRIP );

glVertex3f(-0.1f, 0.1f,-0.10f);

glVertex3f(-0.1f,-0.1f,-0.10f);

glVertex3f( 0.1f, 0.1f,-0.10f);

glVertex3f( 0.1f,-0.1f,-0.10f);

glEnd();

glPopMatrix();

glPopAttrib();

}

Ok, the next part draws the shadowed quads. How does that work? What happens is that you go through every face, and if it is visible, then you check all of its edges. If at the edge, there is no neighbouring face, or the neighbouring face is not visible, the edge casts a shadow. If you think about the two cases clearly, then you'll see this is true. By drawing a quadrilateral (as two triangles) comprising of the points of the edge, and the edge projected backwards through the scene you get the shadow cast by it.

The brute force approach used here just draws to "infinity", and the shadow polygon is clipped against all the polygons it encounters. This causes piercing, which will stress the video hardware. For a high-performance modification to this algorithm, you should clip the polygon to the objects behind it. This is much trickier and has problems of its own, but if that's what you want to do, you should refer to this Gamasutra article.

The code to do all of that is not as tricky as it sounds. To start with, here is a snippet that loops through the objects. By the end of it, we have an edge, j , and its neighbouring face, specified by neighbourIndex .

void doShadowPass( ShadowedObject& object, GLfloat *lightPosition ) {

for ( int i = 0; i < object.nFaces; i++ ) {

const Face& face = object.pFaces[i];

if ( face.visible ) {

// Go Through Each Edge

for ( int j = 0; j < 3; j++ ) {

int neighbourIndex = face.neighbourIndices[j];

Next, check if there is a visible neighbouring face to this object. If not, then this edge casts a shadow.

// If There Is No Neighbour, Or Its Neighbouring Face Is Not Visible, Then This Edge Casts A Shadow

if ( neighbourIndex == –1 || object.pFaces[neighbourIndex].visible == false ) {

The next segment of code will retrieve the two vertices from the current edge, v1 and v2 . Then, it calculates v3 and v4 , which are projected along the vector between the light source and the first edge. They are scaled to INFINITY, which was set to a very large value.