Having cartoon style rendering for D2X-XL had been lingering in the back of my mind for quite a while. I very much like that look in games, and I believe that Descent with its simplistic level designs and abstract textures would lend itself very well to such a graphical style. However, learning about cel shading (which is the most commonly used term for cartoon rendering) from various sources on the internet deterred me from earnestly pursuing the issue: It wasn't quite as simple as the resulting graphical effect might suggest.

I never quite gave up on the issue though, and I finally figured a simple way to make D2X-XL's renderer produce a cartoon style output.

The first hurdle to overcome was to generate textures that looked simple enough yet still had enough detail to look appealing. My first attempt at texturing was simply filling a face with the average color of its texture, but that looked to bland. After a while of thinking about it, I started to tinker with textures in GIMP and eventually achieved a pleasant result by first heavily blurring textures and then massively reducing their color depth.

The next step was to implement blurring and posterization in D2X-XL. I chose to apply these to textures right before they were uploaded to the graphics driver, as that was the central code area all textures had to pass on their way to being rendered, thus requiring the least changes to D2X-XL's texture handling. The blur method I am using is called box blur, which has the advantage of being very fast and converging to a nice gaussian blur if applied several times. I tried two variants of it, one clamping texture accesses at the texture borders, the other wrapping around to the opposite side of the texture. Wrapping turned out to provide the better result, as it produced as good as no visible seams between tiled textures.

Black outlines are an important element of cartoon rendering. I started with outlining level geometry. The most basic way to do it is to outline contours; i.e. draw a dark (black) line where two faces meet and where one of the faces is visible and the other is not. This wouldn't create a lot of outlines though, leaving a lot of visible edges without an outline that intuitively should have been outlined. I therefore developed a solution for such edges, too: They will have thinner outlines. Depending on the angle of the two faces forming the edge, the outlines may be shorter than the edge, or be split in two short lines. This makes the outlines look much more like being hand drawn by an artist and gives the level a more "natural" look. Edges between faces at a very flat angle will not be outlined to avoid cluttering the view with lines.

A problem I had to overcome with the outlines was z fighting (i.e. a face's texture blotting out its outline). Since I chose to draw lines, and not to render the outlines by drawing back face culled faces in line mode, I couldn't use OpenGL's polygon offset functionality. Instead, I solved the issue by adding a small offset towards the viewer position to the lines' vertices. A smaller issue was that the line ends looked blocky and didn't smoothly overlap. I solved this by rendering a circle the size of the lines' thickness on top of each line's ends.

Rendering the outlines for the game's 3D models caused a lot of headache for me and it took me a lot of time to get it to work (mostly because of absolutely stupid coding mistakes I made ... talk about "optimizations" breaking your code). Once I had the basic edge collection done, rendering 3D model outlines boiled down to reusing the geometry outline rendering code, which thanks to C++'s class inheritance and polymorphy was very simple.

A problem with using OpenGL's rather primitive line drawing for painting the outlines became very obvious with 3D models: OpenGL's lines do not become thinner with distance, so a distant robot or player ship appeared almost completely black, is it was covered in the constantly thick lines of its outline. I solved this problem by scaling a model's outline's line width with model distance.

Tweaking the effects applied to achieve a nice cartoon style rendering actually took up the majority of time I have spent with implementing it. I am quite fond of my outlining code: It can make 3D models almost look like being drawn by a human artist - all that by a well devised algorithm.

Since most of the effect is achieved by preprocessing data, the impact on frame rate should be minimal.

I have to say that despite my solution is only a compromise, I am very pleased with the result - particularly given the short time I have spent to create it.

Here are a couple of screenshots - don't they look nice?

One thing my cartoon style renderer couldn't do at the beginning was handling a peculiarity of the underlying stone age Descent render engine, and that was outlining contours within textures. Just think of e.g. fans or gratings with the holes and gaps they model. Sirius had brought this issue up indirectly, and I had to started to think about a solution for it until I finally found a simple one. Depending on texture resolution, it doesn't look that pretty (particularly close up), because texture filtering can only do so much, but it adds yet more cartoon feeling to the renderer.