Volumetrics - fast real-time volume rendering

[Oogst]

I am working on a technique that can render volumetric objects in real-time. Usually, smoke in games is made out of flat particles that do not have real volume. With the technique I am working on, these particles can be rendered with real volume, so objects that move into them will slowly fade out based on how far they are inside the volume. This is useful of rendering smoke, explosions, localised mist and clouds.

I am not sure whether this is really a new technique. I have not found anything like it so far in literature, but I have not read everything ever written either, so I cannot be sure. So my question is: is this a new technique?

This technique works as follows:
-render scene to a rendertexture that contains the distance to the camera at every pixel
-render scene normally, without volume
-render volume, ignoring the zbuffer, by raytracing a sphere in the pixel shader and comparing the resulting thickness of the sphere with the depth in the depth texture
-this might be combined with lighting on the spheres and with texturing

Just to be sure it is clear: I am NOT doing a raymarch through a voxel grid.



These images show the result:

Large particle rendered with volume. Note how the red bars fade out as they enter the volume:


The particle, rendered traditionally, so without volume. Note the sharp edges where the particle boxes intersect:


The raytraced volumetric sphere:


The same volumetric sphere, with lighting:


The one above, combined with a texture:


Moving the camera inside the volume also works correctly, so when close to the red bars, thee become more visible. As the technique is currently implemented, moving beyond the centre of the volumetric particle does not work correctly. A fix for that is planned, but not implemented yet.


A real-time adjustable demo (including source code) can be found here:
Volumetrics demo version 3 (ZIP, 4.0mb)

Ow, by the way, I am working on this for my thesis for the Computer Science Master "Game & Media Technology" at Utrecht University. The first half of my thesis was Interior Mapping, which also used raytracing in the pixel shader (which is the topic of my thesis).

I am looking forward to hearing any ideas for this that you folks might have! (And I am hoping it really is a new technique.)

[Vectrex]

That's really cool. I think soft particles use a similar depth difference to smooth out the edges.
Not sure what you mean about the raytraced sphere. Couldn't you store a kind of depth map just as a texture of the smoke density and use that as the comparitor? That way you could easily get fluffy patches of different densities. Would work for clouds and could be use to calculate the amount of light getting through. But I guess you mean a true 3d volume.
Nice work, runs fast too

[beaugard]

The Crytek soft particles do not have fallof, they mention other approaches used it, but reach the conclusion that an artist can achieve something better (and cheaper) by using alpha. There was another paper (it's somewhere in the forums, actually) that use a sphere. No lighting though.

here, I found it:
http://www.iit.bme.hu/~szirmay/firesmoke_link.htm

[Oogst]

One thousand bombs and grenades, that paper does exactly what I am also doing! Maybe I can still expand on it, I think that would also be okay for my thesis, but still, that makes my idea old and that is a pity.

Some variations I can still work with that they do not seem to be doing (I need to read the paper more thoroughly to be sure):
-more with lighting of the spheres, including normal maps for cool lighting
-more with texture mapping to the spheres (perspective distortion of the particle texture might add some depth)
-combining more than one sphere to get boolean-like operations
-rendering front and back facing polygons to get the depth of real geometry instead of spheres (might be a lot faster as it can use much simpler pixel shaders)
-good examination of the performance of this technique versus voxel ray marching and versus volume slices
-fog areas and clouds

That's really cool. I think soft particles use a similar depth difference to smooth out the edges.
Not sure what you mean about the raytraced sphere. Couldn't you store a kind of depth map just as a texture of the smoke density and use that as the comparitor? That way you could easily get fluffy patches of different densities. Would work for clouds and could be use to calculate the amount of light getting through. But I guess you mean a true 3d volume.
Nice work, runs fast too

I store the scene in a depth map and use that for rendering the sphere-particles. So the sphere particles themselves are not stored in a depth map, they are just pasted over back to front per particle using the density as the alpha.

Ow, wait, that might not be what you mean. I think I could store the front and back of the smoke in a texture and use that in run-time to not do any raytracing. It would not be fully correct, but I think it might work well. Sounds like it would be a lot faster that what I am doing and would support more complex shapes. Hmm, interesting.

I still need to lookup how soft particles are done exactly, but I think they are not done using spheres.

[Moe]

Hi,

your technique sounds somehow familiar to me. In my bachelor thesis, I am working with volume rendering, too, but it is not my main issue, so I'm sorry, if I tell you something wrong.

But back to your questions: Your technique seems to me like an optimized ray casting algorithm, which was described in Real-Time Volume Graphics (http://real-time-volume-graphics.org/ - this book and the homepage is nice for information about volume rendering; I can't check for details, I had to return this book to my adviser already) and in GPU Gems 3, Chapter 30 (http://developer.nvidia.com/object/gpu-gems-3.html).

For ray casting, there is single detail, what does not fit: You said, you do not raymarch through a voxel grid. How do you calculate the volume image data? If you march along the rays in a sampling distance step width until you reach the limitator and sum all the values, it should be ray casting (discrete ray casting integral). If you calculate the density and color of the volume by a different way, I did not yet hear of this technique.
I hope this was helpful.

Anyway, your results look already really cool!

bye
Moe

[Oogst]

But back to your questions: Your technique seems to me like an optimized ray casting algorithm, which was described in Real-Time Volume Graphics (http://real-time-volume-graphics.org/ - this book and the homepage is nice for information about volume rendering; I can't check for details, I had to return this book to my adviser already) and in GPU Gems 3, Chapter 30 (http://developer.nvidia.com/object/gpu-gems-3.html).

I think the one in GPU Gems does a ray march, so that is much more expensive. That book on volume rendering seems to be all about raymarching. I do not have the book either, but I checked the articles it references and they seem to be about ray marching and voxels exclusively.

For ray casting, there is single detail, what does not fit: You said, you do not raymarch through a voxel grid. How do you calculate the volume image data? If you march along the rays in a sampling distance step width until you reach the limitator and sum all the values, it should be ray casting (discrete ray casting integral). If you calculate the density and color of the volume by a different way, I did not yet hear of this technique.
I hope this was helpful.
...

I go much simpler: colour is taken from a 2D texture, as is done with standard particles. Density is simple the thickness of the sphere for the view ray. So no ray marching is done here.

Anyway, your results look already really cool!

Thanks you.

[sinbad]

I believe Crackdown on the 360 uses this approach for fire & smoke. You can tell by the fact that geometry within the explosions shimmers a little and has some subtle pixellation which is a tell-tale sign of a slightly smaller non-HD RTT being used to combine the particle effects and the scene geometry. Nevertheless, a cool effect.

[Oogst]

I believe Crackdown on the 360 uses this approach for fire & smoke. You can tell by the fact that geometry within the explosions shimmers a little and has some subtle pixellation which is a tell-tale sign of a slightly smaller non-HD RTT being used to combine the particle effects and the scene geometry. Nevertheless, a cool effect.

The bad thing of games using it, is that they usually do not tell what and how. I read a text by the folks of Crackdown about their tech and that was pure marketing blabla, without any real info.

[beaugard]

-rendering front and back facing polygons to get the depth of real geometry instead of spheres (might be a lot faster as it can use much simpler pixel shaders)

I like this idea! Imagine the effects you could achieve with animated meshes...

[Moe]

But back to your questions: Your technique seems to me like an optimized ray casting algorithm, which was described in Real-Time Volume Graphics (http://real-time-volume-graphics.org/ - this book and the homepage is nice for information about volume rendering; I can't check for details, I had to return this book to my adviser already) and in GPU Gems 3, Chapter 30 (http://developer.nvidia.com/object/gpu-gems-3.html).

I think the one in GPU Gems does a ray march, so that is much more expensive. That book on volume rendering seems to be all about raymarching. I do not have the book either, but I checked the articles it references and they seem to be about ray marching and voxels exclusively.

Yes, both books use the volume integral (a ray march), that's why I asked you about your technique for calculation. It is more expensive, but on small volumes and with some optimization, you can gain really nice fps
As long as you just want to have nice looking effects without regarding all details, your method is fine in my eyes and hopefully also running on older hardware (in opposite to ray marching)

[bharling]

Well, if its any consolation, this is the only 'volumetrics' demo I've seen that works on my ATI X1650, the vast majority seem to be NVIDIA only, so perhaps that's something new?

works very nicely too.

[Zeal]

Neat trick!

At first I thought, 'wait, there is nothing volumetric about that, its just a soft particle'. But really I guess its a hybrid simifakevolumetricsoftparticle.

You should make a demo with a animated sprite (say a explosion). Then start the 'volume sphere' very small, scale it larger as the animation plays, and then fade the whole thing out.

I would be very curious to see how that looks. If it really looks like the flames are engulfing the surrounding geometry, this could be a VERY useful technique.

[Oogst]

I still have to check, but I think soft particles are not really volume based. They just fade a bit at the edges, I think, while this really calculates volume. The explosion sounds like a neat idea. I have some good animated explosion textures on my hard disc, so I will give that one a go and try it!

[PolyVox]

But really I guess its a hybrid simifakevolumetricsoftparticle.

Great, stick a name like that in a paper and it's bound to get published

[Zeal]

I still have to check, but I think soft particles are not really volume based.

True, as far as I understand the only thing 'special' about a soft particle is that it uses a depth read to alpha blend the hard edges. When I call your technique 'semifakevolumetric' I just mean the data set (the texture) is 2d, nothing volumetric about that.

One other interseting article I found when doing reasearch for my volumetric work - Megaparticles. You asked if your technique was 'new', and I have yet to see anything exactly like it, but mega particles seems the be the closest thing...

http://www.inframez.com/events_volclouds_slide01.htm

Let us know when you get that animated scaling explosion working!

[Oogst]

It's been a while, but I finally have the testing application done.

I found out that what I was doing already existed and is in use in games under the name soft particles. To still have a decent research for the final chapter of my thesis, I am doing a comparison of different ways to achieve this effect. The results can be seen below.

Also, for those interested, the source and executable of the demo can be downloaded here:

Volumetrics versions 6



Let me know what you think!

[jacmoe]

I love you, Oogst!

What a seriously cool thesis you're doing!

[Nauk]

Wow

[_tommo_]

Really cool!

Just... the effect isn't that "cool" aesthetically speaking: the smoke looks like a grey bush... especially the grayscale one makes no sense
Maybe you could tweak better the depth/transparency to have an improved depth perception?
The first screens you posted were nearly perfect in this...

[xadhoom]

I think this billboard is especially used to show the differences
between the techniques.

Oogst: BTW did you consider using the alpha value as depthmap?
Ofcourse this would decrease flexibility but in terms of a uniform
smoke cloud ("every discreet cloud voxel has the same opacity")
the transparency gives information "how long" the view ray is in the
cloud.

[my.name]

super =)

[Kencho]

Awesome, Oogst. Can't add anything to what's already said

[xadhoom]

I tried to compile your showcase with shoggoth, but ended up with
CG script error at startup.


BTW: I got the same strange compiler error with the OIS::MB_RIGHT flag. Very mysterious...
I had to remove it completely to get it compiled.

[Ruud v A]

Cool! I love volumetrics.

My benchmark results:

22:04:55:
BENCHMARK
System information:
Allow NVPerfHUD: No
Anti aliasing: None
Floating-point mode: Fastest
Full Screen: Yes
Rendering Device: %NVIDIA_G84.DEV_0407.3%
VSync: No
Video Mode: 1024 x 768 @ 32-bit colour
Results:
Different materials Planar unshaded particle without texture 1 3735
Different materials Planar mesh with 5 slices without texture 1 2737
Different materials Planar mesh with 9 slices without texture 1 2165
Different materials Planar mesh with 19 slices without texture 1 1213
Different materials Planar mesh with 39 slices without texture 1 726
Different materials Planar mesh with 79 slices without texture 1 412
Different materials Planar mesh with 5 slices without texture (depth texture still rendered) 1 1812
Different materials Planar mesh with 9 slices without texture (depth texture still rendered) 1 1537
Different materials Planar mesh with 19 slices without texture (depth texture still rendered) 1 981
Different materials Planar mesh with 39 slices without texture (depth texture still rendered) 1 633
Different materials Planar mesh with 79 slices without texture (depth texture still rendered) 1 379
Different materials Unshaded constant volume without texture 1 1876
Different materials Unshaded lineair volume without texture 1 1810
Different materials Unshaded spherical volume without texture 1 1626
Different materials Unshaded depth map volume without texture 1 1856
Different materials Shaded spherical volume without texture 1 1325
Different materials Planar unshaded particle with texture 1 3338
Different materials Planar mesh with 5 slices with texture 1 2403
Different materials Planar mesh with 9 slices with texture 1 1812
Different materials Planar mesh with 19 slices with texture 1 1035
Different materials Planar mesh with 39 slices with texture 1 604
Different materials Planar mesh with 79 slices with texture 1 337
Different materials Planar mesh with 5 slices with texture (depth texture still rendered) 1 1656
Different materials Planar mesh with 9 slices with texture (depth texture still rendered) 1 1343
Different materials Planar mesh with 19 slices with texture (depth texture still rendered) 1 858
Different materials Planar mesh with 39 slices with texture (depth texture still rendered) 1 538
Different materials Planar mesh with 79 slices with texture (depth texture still rendered) 1 314
Different materials Planar mesh with 5 slices with texture, rendered to floating buffer 1 1073
Different materials Planar mesh with 9 slices with texture, rendered to floating buffer 1 848
Different materials Planar mesh with 19 slices with texture, rendered to floating buffer 1 525
Different materials Planar mesh with 39 slices with texture, rendered to floating buffer 1 317
Different materials Planar mesh with 79 slices with texture, rendered to floating buffer 1 175
Different materials Unshaded constant volume with texture 1 1863
Different materials Unshaded lineair volume with texture 1 1806
Different materials Unshaded spherical volume with texture 1 1627
Different materials Unshaded depth map volume with texture 1 1837
Different materials Shaded spherical volume with texture 1 1300
Finished benchmarking!

I have one suggestion:
compose the smoke/dust out of more particles so that the effect also looks volumetric when moving. It now looks smooth and soft, but when you walk you can still clearly notice this is a billboard. Making multiple volumetric smoke billboards would solve this I think.

[Oogst]

Thanks for the feedback!

By the way, what I forgot to mention: I think the depth map version wins: it has better performance than the sphere-version and the sphere version can almost exactly be made by altering the depth map, plus the depth map can do other shapes than spheres.

Really cool!

Just... the effect isn't that "cool" aesthetically speaking: the smoke looks like a grey bush... especially the grayscale one makes no sense
Maybe you could tweak better the depth/transparency to have an improved depth perception?
The first screens you posted were nearly perfect in this...

Yeah, I know, I already spent a lot of time making this smoke and it is still not good. Making a good photorealistic smoke texture is just terribly difficult. I went through hundreds of smoke and explosion photo's on Google and spent a lot of time editing in Photoshop to get is to this level. I guess I am just not good enough a texturer for the next level.

...

Oogst: BTW did you consider using the alpha value as depthmap? Ofcourse this would decrease flexibility but in terms of a uniform smoke cloud ("every discreet cloud voxel has the same opacity") the transparency gives information "how long" the view ray is in the cloud.

The problem of using alpha as thickness is that the alpha needs to have a lot of variation for that, while all the other materials look better with an alpha that is pure white for most of the cloud, except at the edges, where it fades out. If I made a full texture for the depth map version only, I could do what you suggest and win a little bit in performance. So for completenes' sake I am going to do that before running the final benchmarks. Thanks for the idea!

I tried to compile your showcase with shoggoth, but ended up with
CG script error at startup.


BTW: I got the same strange compiler error with the OIS::MB_RIGHT flag. Very mysterious...
I had to remove it completely to get it compiled.

Yeah, I did not switch to Shoggoth for this one, so I guess that might give some random compile errors. Also, I worked with Visual Studio 7.1, but that should not give any problems.

Cool! I love volumetrics.

My benchmark results:
...

I have one suggestion:
compose the smoke/dust out of more particles so that the effect also looks volumetric when moving. It now looks smooth and soft, but when you walk you can still clearly notice this is a billboard. Making multiple volumetric smoke billboards would solve this I think.

Thanks for the benchmarking! You are a little bit early though, as I just found out I need to still optimise some small things. I will post it here as soon as I am ready for the benchmarking.

For a real smoke effect, a lot of particles would be needed of course. The testing is done with a single particle, because it exemplifies the different techniques better, but just outside the screen is a particle system with several particles being emitted. You can move around with WASD to get it into view.