CT Volumetric Visualization

[Praetor]

For a while we've been working on volumetric rendering of CT and MRI scans. There's several challenges with working with this kind of non-ideal, real-world data. There's a lot of it (scans are often larger than video memory) and it is not all good (noise sucks). There's a ton to do, not least of which is packaging the 3D viewer into a larger application for use by actual medical professionals. But, I'm getting fairly happy with the quality of the rendering.



There are several rendering modes and quality settings for peeking into the body.




You can see some blood vessels spidering out from the chest.



I like the definition on the arteries running down the legs.

[mkultra333]

That looks excellent. Really nice, must be difficult with all the transparency. Those bottom two of the torso with the blood vessels would look great as gore in an FPS, hehe.

[xadhoom]

Wow, great colouring. Wondering about the lighting of the volumes I remember this paper: http://www.mpi-inf.mpg.de/~ritschel/Papers/VolumeSH.pdf
which has some astonishing effects you can see here: http://www.mpi-inf.mpg.de/~ritschel/Papers/VolumeSH.avi

xad

[metaldev]

wow this is super impressive. nice work.
what kind of fps do you get atm?

[Praetor]

This depends highly on the video card's capabilities. On the target machine with a FX3800 the framerate stays above 15fps. Obviously there are better cards out there which can do it even faster.

As for lighting, the one problem with doing anything with these volumes is load times. Unless we somehow hook into the entire CT/MRI pipeline and attach some kind of processors to the data coming out (can you say FDA nightmare?) we really need to work with the raw data coming in. Any processing that must be done needs to be really fast and happen on-the-fly. Caching can only help a little, since a lot of studies will only be loaded once. I have at least one idea that I want to try that will make the rendering faster at the same quality as well as providing a possible ambient occlusion approximation.

The coloring also needs a lot of work. I only spent about 5 minutes making this particular CLUT.

@justboo Yeah you can fly around and inside the volume. You can also control "slicing" from any cardinal direction so you can control what slabs of the data you see. Right now it is only a simple orbit camera which makes fly-throughs difficult. The problem is that any free-look style camera is dead-simple for one of the video game generation to use but you'd be amazed how painful it is and how confused the average medical professional becomes when you try to explain mouse-look with WASD. I would dread trying to put an xbox controller in their hands...

[Rambus]

Looks great!

What size volumes can your system visualize?

Are you coloring the volumes your self or automatically deciding colors based on known densities of tissues? (e.g. blood vessels = red)

If you haven't read about GigaVoxel yet it might be worth checking into: http://artis.imag.fr/Publications/2009/CNLE09/
I know I was saddened to see they beat me to a few things in my own system.

[Praetor]

Most scans are 512x512 with variable numbers of slices. Most are in the hundreds. Full-body scans can have thousands of slices but in practice full-body scans are rare.

I tried giga-voxel like systems. The indirection-type mechanisms are powerful but very complicated and there's a lot of manipulation you need to do to manage the data. For now I've opted for a much simpler mechanism for managing my data.

[Rambus]

Thanks for the quick response.

I'm curious as to what file formats your system supports? (e.g. Raw, Rawvox, Binvox, 3b)

Do you feel these formats are limited and if so what issues do you take with them?

I've implemented my own format specific to game use and would be interested in some of the constraints you have with visualizing medical datasets.

For instance-
Do you favor breaking the dataset into subspaces / bricks ?
Is mipped data a requirement?

[Praetor]

The data always comes in DICOM format which is the medical standard that comes out of all imaging devices.

[Praetor]

We will stay far away from the networking stuff. Yes DICOM has tons of data in it. Like most "standards" it is somewhat dangerous to rely on parts, but we work it out.

[Rambus]

My 'Invented' file format was designed for entertainment applications. I recall looking at the DICOM format when I was implementing my first version of software and being scared away by the design specs. Its good to have confirmation that it is indeed the industry standard format- I'll do some more research into it. Thank you.

[Praetor]

I encourage you to find a reusable library for dealing with the DICOM format. Not only do you need to deal with the various non-image data fields but you also need to be able to handle a couple different data formats (different bit lengths and packings). Also more studies are being compressed with JPEG2000 these days. I use dcmtk, which has a commercial JPEG2000 system. But, the basic system is good enough for most starting needs.

[sinbad]

Looks awesome, great work. Volume rendering is really hard to get running at reasonable rates without obvious undersampling artefacts, but you seem to have cracked it.

[jacmoe]

This is impressive work!
Very exiting to see how 3D rendering can assist in the medical field.

[Praetor]

I'll be doing a lot more work on the rendering method coming up. Quality is good, but the actual shading technique needs some tweaking. It is too ghosted and "soft" and the shapes inside the body really need to be brought out with some trickier shaading methods. Got a few ideas. I'll post back if I come up with something cool.

[AfyDriver]

i am new too ogre, and have to do sth like this!
would you plz explain some more about the progress?

[stefanbanev]

Indeed, it's an excellent result for GPU VR; still way below then CPU VR may deliver for the same price; good luck...