Four in two does go

(or, Texturing a terrain in one pass using four greyscale detail textures and a lightmap with only two texture units)

INTRODUCTION

Say you want to draw a terrain heightfield using a vertex array or similar. A typical approach for making the terrain appear detailed when viewed from close to the ground is to apply detail textures.

Detail textures are textures that are mixed and tiled over the terrain at a relatively small scale. For example, a single instance of a texture may cover an area of 3*3 grid squares, or even just a single grid square. The detail textures add some variations to the colour of the terrain at a sub-polygon scale, without the need for huge texture maps.

To bring more variation to the terrain, more than one detail texture is usually used. The textures are mixed together in proportion to some local properties of the terrain. For example, mountainous areas use predominantly a rocky-looking detail texture, while water areas use a water surface texture. Depending on altitude, a grass detail texture may be mixed in with varying degrees. The more detail textures that are used, the more varied (and hopefully more realistic) the terrain will look.

There is of course a cost to pay for using many different detail textures. A graphics card can only apply a limited number of textures to a model at the same time. This number is the number of texture units of a card, and ranges from one for the oldest cards, to eight for the latest top of the line consumer cards. So if we want to use four different detail textures mixed arbitrarily but the graphics card has only two texture units, the scene would have to be drawn twice. First with one pair of textures, then with the other pair.

When rendering a scene with a large number of polygons, which a terrain typically has, multiple passes can reduce the framerate significantly. Maximising the number of detail textures applied per pass is therefore crucial. Although it may appear at first that there is no way to use more than the number of texture units per pass, with some trickery and simplification it is actually possible.

THE PROBLEM

Here is the specific problem I'm solving: apply detail textures and a lightmap to a terrain rendered using vertex arrays in a single pass on a NVidia Quadro2 graphics card. This card has all of two texture units, no dependent texture reads (so no per-pixel fiddling with the texture coordinates), and no fancy fragment shading programmability (which would pretty much remove the need for doing all this in the first place).

For those keeping score, before we even start with the detail textures we already have to accommodate a lightmap and a map describing how to mix the textures. One of these can be put in the vertex colours of the terrain (as both maps are at the scale of the terrain), and the other would have to go into a texture. So we're down one texture unit and we have barely left the starting blocks.

OPENGL MULTI-TEXTURING BASICS

Look it up on the internet.

Read it? Great. From here on I'll assume you know how to do multi-texturing in OpenGL.

THE WORLD IS BLACK&WHITE WITH A HINT OF COLOUR

The first step is to realise that detail textures need not be colour. Greyscale detail textures work quite well. The colour of the terrain can come from the lightmap. A greyscale detail texture only needs one colour channel, instead of the three (RGB) that a colour detail texture needs. A texture unit has four channels (RGBA), so four detail textures could be squeezed into one texture unit.

By putting them in the same texture unit, there are some limitations which arise. The main one is that all the detail textures in the same texture unit must be of the same size and scale. They all have to be the same width and height in pixels, and they all must be applied to the terrain at the same scale. The channels share the same texture coordinates.

Conveniently, the mixmap (which determines the amount of mixing of the detail textures at each vertex) can be four channels, thereby giving the contribution for each of the four detail textures individually.

Summarising the situation so far, we have to handle:

1 * Lightmap (RGB)
1 * Mixmap (RGBA)
4 * Detail texture (one each in R, G, B, A)

which is a total of 11 channels, while we have available:

1 * Vertex colours (RGBA)
2 * Texture units (RGBA)

which is 12 channels.

It fits!

Now the channels "just" have to be mixed and matched to produce a final colour.

MIXING AND MATCHING

The next crucial step is to set up the texture units in such a way that the detail textures are mixed together according to the weights in the mixmap, and combine it all with the lightmap.

The various maps and textures are assigned as follows:

Primary (vertex) colours : mixmap
Texture unit 0 : detail textures
Texture unit 1 : lightmap

Other orderings are also possible, as long as the detail textures are in a texture unit.

The texture combining is roughly done in two steps:

Texture unit 0 : mix detail textures according to mixmap
Texture unit 1 : combine previous result with lightmap

The mixing of the first step needs to take the values of each of the four detail textures, multiply them by their respective weights in the mixmap, and sum those to produce a single value. The closest texture combining method that does this is DOT3. However, that can only use as input the R, G, and B channels. The detail texture in the A channel needs to be handled separately:

  glActiveTexture(GL_TEXTURE0);

  glTexEnvi(GL_TEXTURE_ENV, GL_COMBINE_RGB, GL_DOT3_RGB);
  glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE0_RGB, GL_TEXTURE0);
  glTexEnvi(GL_TEXTURE_ENV, GL_OPERAND0_RGB, GL_SRC_COLOR);
  glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE1_RGB, GL_PRIMARY_COLOR);
  glTexEnvi(GL_TEXTURE_ENV, GL_OPERAND1_RGB, GL_SRC_COLOR);

  glTexEnvi(GL_TEXTURE_ENV, GL_COMBINE_ALPHA, GL_MODULATE);
  glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE0_ALPHA, GL_TEXTURE0);
  glTexEnvi(GL_TEXTURE_ENV, GL_OPERAND0_ALPHA, GL_SRC_ALPHA);
  glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE1_ALPHA, GL_PRIMARY_COLOR);
  glTexEnvi(GL_TEXTURE_ENV, GL_OPERAND1_ALPHA, GL_SRC_ALPHA);

The RGB channels of both the mixmap and the detail textures need to be pre-processed first to make them work with DOT3. This is explained later.

The second step is to combine it with the lightmap. There is a problem however. Both the values in the RGB channels and the alpha channel from the first step need to be combined with the lightmap to produce an RGB value. That's a total of three sources, two of which are in RGB channels and one in an alpha channel, which need to result in an RGB output. Texture combining does not provide for such functionality, or at least not in one step. GL_INTERPOLATE comes close in that it takes three sources, but it results in Arg0 * Arg2 - Arg1 * Arg2 + Arg1. There will always be an Arg1 component which is not modulated by the lightmap. Arg1 could be set to be the lightmap itself. That would mean that either the previous RGB or the previous alpha is in Arg2. Arg2 would dominate in the sense that if it is zero, Arg0 will not contribute to the final result at all. This is not desired, as the previous RGB and alpha values should be treated as being independent (being the results of independent detail textures).

There is another way of modulating in some grey: by using blending. The blend function can be set up to mix in shades of grey with the terrain according to the value of the alpha channel of the textured terrain:

  glBlendFunc(GL_ONE_MINUS_SRC_ALPHA, GL_ZERO);
  glEnable(GL_BLEND);

So now adding in the alpha channel created in step 1 can be left to the blending stage. We only have to combine the RGB result of step 1 with the lightmap:

  glActiveTexture(GL_TEXTURE1);

  glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_COMBINE);
  glTexEnvi(GL_TEXTURE_ENV, GL_COMBINE_RGB, GL_MODULATE);
  glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE0_RGB, GL_TEXTURE1);
  glTexEnvi(GL_TEXTURE_ENV, GL_OPERAND0_RGB, GL_SRC_COLOR);
  glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE1_RGB, GL_PREVIOUS);
  glTexEnvi(GL_TEXTURE_ENV, GL_OPERAND1_RGB, GL_ONE_MINUS_SRC_COLOR);

Using GL_MODULATE can cause the terrain to become too dark. It can be replaced with a GL_ADD_SIGNED, or even a GL_INTERPOLATE with the Arg2 set to GL_CONSTANT to control the overall degree with which the detail textures affect the terrain:

  glActiveTexture(GL_TEXTURE1);
  glTexEnvfv(GL_TEXTURE_ENV, GL_TEXTURE_ENV_COLOR, mixcol);

  glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_COMBINE);
  glTexEnvi(GL_TEXTURE_ENV, GL_COMBINE_RGB, GL_INTERPOLATE);
  glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE0_RGB, GL_PREVIOUS);
  glTexEnvi(GL_TEXTURE_ENV, GL_OPERAND0_RGB, GL_ONE_MINUS_SRC_COLOR);
  glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE1_RGB, GL_TEXTURE1);
  glTexEnvi(GL_TEXTURE_ENV, GL_OPERAND1_RGB, GL_SRC_COLOR);
  glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE2_RGB, GL_CONSTANT);
  glTexEnvi(GL_TEXTURE_ENV, GL_OPERAND2_RGB, GL_SRC_COLOR);

  glTexEnvi(GL_TEXTURE_ENV, GL_COMBINE_ALPHA, GL_MODULATE);
  glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE0_ALPHA, GL_PREVIOUS);
  glTexEnvi(GL_TEXTURE_ENV, GL_OPERAND0_ALPHA, GL_SRC_ALPHA);
  glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE1_ALPHA, GL_CONSTANT);
  glTexEnvi(GL_TEXTURE_ENV, GL_OPERAND1_ALPHA, GL_SRC_ALPHA);

Another set of variations can be made by using GL_SRC_COLOR instead of GL_ONE_MINUS_SRC_COLOR. Experiment to get the right balance.

PREPARING TEXTURES FOR DOT3 COMBINING

The formula for GL_DOT3_RGB combining is:

  t = 4 * ((Arg0_r - 0.5) * (Arg1_r - 0.5) +
           (Arg0_g - 0.5) * (Arg1_g - 0.5) +
           (Arg0_b - 0.5) * (Arg1_b - 0.5))

It is really intended for use with so-called normal maps, hence the slightly strange subtractions of 0.5 and the multiplication by 4. For our texturing purposes however, we only want to use it to multiply the matching channels of the arguments and add up the results.

The range of values produces by DOT3 is -3 <= t <= 3. Furthermore, if both arguments are (0,0,0) the result is the same as if they both were (1,1,1). This is obviously not desirable if (0,0,0) is meant as no contribution of any detail textures and (1,1,1) is meant as fully adding all detail textures.

The solution is to restrict the RGB values of the two arguments to the range 0.5 to 1, which limits t to the range 0 <= t <= 3. Note that any value of t about 1 will be clipped to 1 though.

The texture maps involved in the DOT3 computation in the first texture step (the detail textures and the mixmap) are preprocessed by squashing the values in the RGB channels into the range of 0.5 to 1.0. Note that the alpha channel isn't touched, as it does not take part in the DOT3. If each colour channel is a byte, converting to the needed range can be done using something like:

  channel_value = (((int)channel_value) + 255) / 2;

RESULTS

These are screenshots taken from a simple terrain renderer. It uses a 512 * 512 heightmap, a cubemap for the sky rendered with Terragen, and the multitexturing methods described above. There is no level of detail control (the whole thing is done as vertex arrays in a display list), and runs at 20+ frames per second.

detail textures are numbers, showing the mixing (PNG 472kB)

without detail textures (PNG 254kB)
with detail textures (PNG 358kB)

without detail textures (PNG 195kB)
with detail textures (PNG 299kB)

CONCLUSION

In conclusion, you can indeed do four greyscale detail textures mixed arbitrarily plus a lightmap in a single pass using only two texture units on a GeForce2-type card.

The disadvantages are that the detail textures all have to be the same size, and they all use the same texture coordinates. In practise I've also found that it can be tricky to get a good balance between the strength of the detail textures and the strength of the colours in the lightmap. Lastly, blending (like is used to add in the detail texture in the alpha channel) is done after fog in the OpenGL pipeline. This means that the fourth detail texture can not have any fog applied to it. As seen in the above results, this isn't too bad if that texture is used for water.

Last Modified: 16-Jun-2003 by Jarno van der Linden