[GSoC 2012] Volume Rendering with LOD aimed at terrain

Threads related to Google Summer of Code
Post Reply
User avatar
jacmoe
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 20570
Joined: Thu Jan 22, 2004 10:13 am
Location: Denmark
x 179
Contact:

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by jacmoe »

PhilipLB wrote:Plugin: Yep, this is something I'd like to do when everything is in a somewhat "demoable" state.

Patent: See above. :)
Mattan Furst wrote:@jacmoe

From the wikipedia page philipLB link to:
The Marching Cubes algorithm is claimed by anti-software patent advocates as a prime example in the graphics field of the woes of patenting software. An implementation was patented (United States Patent 4,710,876[4]) despite being a relatively obvious solution to the surface-generation problem, they claim. Another similar algorithm was developed, called Marching Tetrahedrons, in order to circumvent the patent as well as solve a minor ambiguity problem of marching cubes with some cube configurations. This patent expired in 2005, and it is now legal for the graphics community to use it without royalties since more than 17 years have passed from its issue date (December 1, 1987[4]).
I think this means its in the clear.
Wow, time flies!
I am referring to a discussion we had in 2004 I think, where the algorithm were still patented.
Glad to learn that it's now truly free. :)
/* Less noise. More signal. */
Ogitor Scenebuilder - powered by Ogre, presented by Qt, fueled by Passion.
OgreAddons - the Ogre code suppository.

PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

First rough LOD.
  • Finished the triangulation via Marching Squares for the open parts of the chunks.
  • Updated the roadmap, added the plugin point.
  • Changed the Octree grid split method to use some kind of geometric error.
  • Moved Mesh Generation to the class Chunk.
  • Finished a first LOD version with chunks and approximated pixel error.
Image
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.

PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

Making things more usefull.
  • Made the TextureSource working with some real data coming from the editor Acropora.
  • Added the possibility (switched on per default) to use trilinear interpolation of the normal in TextureSource.
  • Some big loading time optimizations.
  • Added a CSGUnarySource as abstract parent class, CSGNegateSource is now a child class of it.
  • Added a CSGScaleSource (CSGUnarySource is the parent class) which scales the given Source. Good for scaling a TextureSource to the desired size.
Image
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.

dermont
Orc Shaman
Posts: 785
Joined: Thu Dec 09, 2004 2:51 am
x 35

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by dermont »

Good job, it looks like you are making really good progress on this. There are a couple of build issues on Linux which you probably haven't got round to fixing:

Code: Select all

diff --git a/Samples/VolumeTerrain/CMakeLists.txt b/Samples/VolumeTerrain/CMakeLists.txt
--- a/Samples/VolumeTerrain/CMakeLists.txt
+++ b/Samples/VolumeTerrain/CMakeLists.txt
@@ -5,8 +5,8 @@
 	include/CSGSource.h
 	include/DualCell.h
 	include/DualGridGenerator.h
-	include/Isosurface.h
-	include/IsosurfaceTables.h
+	include/IsoSurface.h
+	include/IsoSurfaceTables.h
 	include/MeshBuilder.h
 	include/QEF.h
 	include/TextureSource.h
diff --git a/Samples/VolumeTerrain/include/IsoSurface.h b/Samples/VolumeTerrain/include/IsoSurface.h
--- a/Samples/VolumeTerrain/include/IsoSurface.h
+++ b/Samples/VolumeTerrain/include/IsoSurface.h
@@ -107,4 +107,4 @@
 }
 }
 
-#endif
\ No newline at end of file
+#endif
diff --git a/Samples/VolumeTerrain/include/Source.h b/Samples/VolumeTerrain/include/Source.h
--- a/Samples/VolumeTerrain/include/Source.h
+++ b/Samples/VolumeTerrain/include/Source.h
@@ -2,6 +2,7 @@
 #define __Source_H__
 
 #include "OgreVector3.h"
+#include <float.h>
 
 namespace Ogre {
 namespace Volume {
@@ -39,4 +40,4 @@
 }
 }
 
-#endif
\ No newline at end of file
+#endif
diff --git a/Samples/VolumeTerrain/include/VolumeTerrain.h b/Samples/VolumeTerrain/include/VolumeTerrain.h
--- a/Samples/VolumeTerrain/include/VolumeTerrain.h
+++ b/Samples/VolumeTerrain/include/VolumeTerrain.h
@@ -2,10 +2,10 @@
 #define __VolumeTerrain_H__
 
 #include <stdio.h>
-#if OGRE_PLATFORM_WIN32
+#if OGRE_PLATFORM == OGRE_PLATFORM_WIN32
 #include <fcntl.h>
+#include <io.h>
 #endif
-#include <io.h>
 #include <iostream>
 #include <string>

User avatar
duststorm
Minaton
Posts: 921
Joined: Sat Jul 31, 2010 6:29 pm
Location: Belgium
x 80
Contact:

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by duststorm »

Very, very, very nice work! I like :)
I love that you can come up with things to show us so quickly too.
Developer @ MakeHuman.org

PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

Thanks. :)

I'll apply those patches right today. And also setup a build environment soon on a MacBook so this small glitches won't happen again. :)
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.

User avatar
Cygon
Halfling
Posts: 70
Joined: Thu Nov 07, 2002 9:36 pm
Location: Germany
x 5
Contact:

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by Cygon »

You're fast _and_ you're a crowd pleaser with all those interesting screenshots :)

Reading about those CSG classes makes me wonder how terrain will in the end be defined. I admit haven't looked at those CSGUnarySource and CSGUnaryScaleSource classes yet -- are they interfaces through which the mesh generator (or how it is called) queries a 3D region of the world as a point cloud or does it work differently?

I'm wondering if in the end it will be possible to stream data into the your component from some kind of sparse voxel data structure (like a grid, rtree or octree). And if it would be a good idea to turn hand-modelled structures into some kind of higher frequency point cloud instead of keeping them independent from the terrain (I'm guessing they need to be separate models since the terrain uses triplanar texturing and textures can only change based on compass orientation and steepness).

User avatar
petrocket
Gremlin
Posts: 178
Joined: Tue Mar 20, 2007 3:29 am
x 9
Contact:

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by petrocket »

I'm so freakin excited about this GSoC project! Thumbs up!! :D
Ogre API & Manual | Ogre Wiki

blog | website | work

Follow me on twitter @ twitter.com/petrocket

PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

Thanks both. :)
Reading about those CSG classes makes me wonder how terrain will in the end be defined.
The volumedata / density function is just based on a single interface, Source. The CSG classes implement this.
This CSG stuff is actually just a nice-to-have and some easy way for me to create debug-data like spheres. They were easy to implement and every now and then, I add some classes here (somewhere in my idea notes: CSGMirror). The actual terrain data is coming from 3D texture files exported from Acropora [1] loaded via TextureSource.
One nice thing is: As the CSG classes just work on the Source interface, you can freely combine them with any Source implementation. This means, adding a sphere to your TextureSource is no problem. :)
I'm wondering if in the end it will be possible to stream data into the your component from some kind of sparse voxel data structure (like a grid, rtree or octree).
Should be possible, just implement the Source interface. :) Where "stream" implies here to have terrain streamed in the runtime which sadly is not (easily) possible because of the loading process (for each chunk: generate octree, generate dualgrid, marching cubes on dualgrid). So a realtime editor is currently out of the scope of this project. :(
And if it would be a good idea to turn hand-modelled structures into some kind of higher frequency point cloud instead of keeping them independent from the terrain (I'm guessing they need to be separate models since the terrain uses triplanar texturing and textures can only change based on compass orientation and steepness).
Hm, this comes again down to "just" implement the Source interface. This implementation would be responsible for the higher frequencies. A CSGPerlinNoisePlane is totally possible. As it turns out, that this project makes a somewhat fast process without "search the corrupt pointer for one week"-bugs (yet...), this is a class I'd like to implement. :) Time will show. Or a Mesh-to-volume converter (seen in an implementation of another isosurface algorithm). Subtracting some spheres from the Ogre Head would be nice. Or integrating him in the terrain like Mount Rushmore [2]. :D But those things are far away. :)

Material is currently this triplanar texturing, hacked together in a single shader without any configuration. This is one of the very next steps, making it more flexible. Don't know yet how. I'd like to have it like in the terrain component with it's gazillion fallbacks. But this would take a long time. I'd love to discuss this in a few days in a separate topic, how to make a simple, generic triplanar texturing material creation class. What fallbacks are needed, etc.

[1] http://www.voxelogic.com/index.php?opti ... &Itemid=21
[2] http://upload.wikimedia.org/wikipedia/c ... -2_new.jpg
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.

User avatar
Mattan Furst
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 260
Joined: Tue Jan 01, 2008 11:28 am
Location: Israel
x 32

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by Mattan Furst »

Material is currently this triplanar texturing, hacked together in a single shader without any configuration. This is one of the very next steps, making it more flexible....
I think that a better implementation of the triplanar texturing can be provided through texture array.

Texture array allows you to pack several textures into a single texture. Then shaders can refer to the texture just like 3d texture using the w texcoord to move between the original textures. The difference between texture arrays and 3d textures is that in a 3d texture mipmaps are done for the entire texture as a whole, in texture arrays mipmaps are done per "original" texture.

You can also do it with atlas textures, but with atlas textures you will always have problems with repeating textures. Plus it's much easier to compile and change texture arrays at runtime.

The downsides are:
1. texture arrays are not supported on DirectX9. Only in DirectX11 and openGL
2. DirectX11 is not fully developped (one of the GSoCs this year). however I think that the texture array feature is already supported.
3. Not really in the scope of the GSoC
it's turtles all the way down

PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

Sounds interesting. But is there any big disadvantage of just using three textures like now?
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.

User avatar
Mattan Furst
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 260
Joined: Tue Jan 01, 2008 11:28 am
Location: Israel
x 32

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by Mattan Furst »

Sounds interesting. But is there any big disadvantage of just using three textures like now?
No. as I said it's out of the scope of your GSoC project. The three textures is what you committed to and is perfectly fine.
it's turtles all the way down

User avatar
Mattan Furst
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 260
Joined: Tue Jan 01, 2008 11:28 am
Location: Israel
x 32

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by Mattan Furst »

Hello philip,

I am posting the PM you wrote me here as it is just the sort of questions which can benefit from the expirience of the Ogre community developers. If you want me to I'll remove it.
what do you think? Currently, the volume rendering uses chunks made out of Chunk which derives from SimpleRenderable. The batchcount is naturally quite high...
So currently: A lot of SceneNodes (one per Chunk) and the LOD mechanism switches on/off the visibility of the Chunks.
I could convert it to not use Chunks but build up (the same amount of) StaticGeometries.
Would this make a difference?
...
My computer Keeps giving me giving me blue screens of death so I haven't been able to take a very good look at you're code. I'll try to write a more informed post by tomorrow. however:
  1. I haven't worked with StaticGeometry but as far as I know StaticGeometry is used to generate instances of static objects. For instance a forests made of same looking trees. I'm not aware of any benefit that moving you're code through StaticGeometry mechanism will give you.
  2. In the sample you created I can only see a single sphere generated in a single batch? does the terrain in you're example have more batches. I can't find the texture it derives the terrain from ("bigTerrain2.dds").
  3. Why is the batch count high. Since all of the terrain is made from the same material can't you just push everything to a single SimpleRenderable and draw everything in one batch (I'm not saying you should (i.e. see reasons below), I'm saying from a programing stand point it should be possible to do so).
  4. Why do you have multiple scene nodes. I it might be better to have one scene node and have all the chunks attached to it. The only reason to have multiple scene nodes is if you want each chunck to start from a diffrent world position. This might be problematic as it might cause stiching problems between chunks (thin lines were you can see the between the chunks).
  5. This is more of a side note. Are you aware of a program called perfhud? It is part of NVidia's prefSDK. It allows you, amoung other things, to look at how the scene is rendered batch-by-batch. It works as long as you have a GEForce card to work with. There might be batches you're not aware that you are rendering that you are rendering. (You need to select running Ogre with NV perfhud allowed in the dialog setting to make it work).
  6. As for dividing your terrain to chunks. If what I said at point number 1 is true than you can divide your terrain to chucks in whichever way you want. Generaly there are 2 ways, I'm aware of, in which terrains are devided to chunks.
    The first is have a single batch or a set number of batches which are rendered relative to the camera position. This is more appropriate to terrains based on heightmap textures where the textures are fed directly to the graphic card for processing.
    In most other terrains, the terrain are devided into a fixed size grid or a progressive grids. The reason to divide a terrain to diffrent renderables (batches) even when you can render them in a single batch is this. If you want to create a terrain in which you can roam freely, At some point you will need to update the geometry of the terrain. If the geoemtry is to big it will create a big framerate hickup as it is loaded. If you can divide your terrain to a grid you can update only the tiles in the grid that have actually been updated plus each individual tile will be much smaller than whole grid so it make much less of a hickup.
    Note: There are a couple of tricks to implementing this type of grid implementation. if this is the way you want to proceed I'll post about them separately.
*progressive grid - grid in which where areas closer to the camera get devided recursively by 2 (much like quad trees).

If anything I wrote here doesn't make scene than I am probably missing something. I will try to keep my skype open this week if you want to call.
it's turtles all the way down

PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

Hi,

thanks for the answer, I'll dive into it in the next days. For the very next thing, I'd like to have a nicer material with some bumpmapping on it. :)

Just pushed an updated version which loads the volume data from the 3d texture provided in Samples/Media/volumeTerrain/volumeTerrains.zip
Last edited by PhilipLB on Sun Jun 24, 2012 8:49 pm, edited 2 times in total.
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.

PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

Optimizations and getting a bit towards something for the real world.
  • Updated the roadmap as the LOD stuff was done one week early and I'm happy for now with the Octree.
  • Changed the density function Source::getValueAndNormal to Source::getValueAndGradient.
  • Using a new OctreeNodeZhangSplitPolicy, some kind of this method extended: http://www.andrew.cmu.edu/user/jessicaz ... n/meshing/
  • Don't generate those border/skirt triangles on the border of the world. Not needed here and saves a lot of triangles.
  • Put the volume textures in a zip to decrease IO loading time.
  • Don't generate chunks which don't contain triangles and don't patch cracks. This decreases the batchcount.
  • Implemented a CacheSource. Might be usefull for very expensive (nested CSG) sources.
  • Using far better normals for the crack patching marching squares.
  • Added a debug/documentation visualization of the MC configurations, the key m switches through all 256.
  • Switched off the handling of ambigous cases in marching squares as it caused a crack (this took some hours of debugging...).
Image
Image
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.

User avatar
Mattan Furst
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 260
Joined: Tue Jan 01, 2008 11:28 am
Location: Israel
x 32

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by Mattan Furst »

Continuing my post about your PM.
  • I don't know why I didn't see it previously. I just saw how you generate your terrain (the 2 sphere experiment) through PrefHud. you're generating all the geometry of the sphere. including both the insides of the sphere and for each cube-like renderable (batch) all the walls of the cube from all sides. This is a huge waste of geometry. I thought the way this algorithm worked it only generated the hull of a shape.
    VolumeRendering halfway through the build process
    VolumeRendering halfway through the build process
    Half build.png (752.24 KiB) Viewed 5568 times
  • it should be easy to unite the batches into a single or a couple of chunks.
    • Add all the vertexes from a given set of chunks to a new chunk.
    • While adding the vertexes to the new chunk only add unique vertexes.
    • Assign each new vertex added to a new index number (starting from 0). Create a map that will translate old index values to new index values
    • add the indexes from the set of chunks to the new chunk using the translation map you constructed.
it's turtles all the way down

User avatar
Mattan Furst
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 260
Joined: Tue Jan 01, 2008 11:28 am
Location: Israel
x 32

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by Mattan Furst »

There is something I wanted to share.

I had helped write a terrain engine based on a mesh file like format. The following is a general description of the algorithm we ended up using:
(I am assuming here that you don't have many materials to deal with and those that you do can be pre-constructed. I will keep steps involving materials out of the description)

We have 2 threads. The main thread which does the rendering and the work thread which prepares the buffers to be rendered for the main thread. There are 2 main ideas behind this technique. The secondary thread's job is to run over the terrain and extract / prepare it's geometry for the main thread. Anything that can be done on the secondary thread is done on the secondary thread. The second idea is to have a single point at which data is exchanged and synced between the 2 threads, with as few as possible or even no locks. This is done to improve performance.
  1. Main thread: We start by saving all the data upon which the terrain will be updated to a side structure. In your case this is just the source of the terrain (name of the terrain) from which it is loaded and the position of the camera/s on the terrain.
  2. Worker thread: Once the data is saved we unpause the work thread. The work thread reads data structure the main thread saved.
  3. Worker thread: The secondary thread then proceeds to scan the terrain appropriately and captures all the geometry of the terrain into various small buffers (analogus in your case to the information in chunks).
  4. Worker thread: The small buffers are combined (batched together) into big buffers. Anything that has the same material can be added to a single buffer, however you need to be carefull here, big buffers will cause hickups when they are loaded to the graphic card.

    We gather the buffers in a list (which is known as a work list).

    While constructing the combined buffers a hash value is created from the data of the vertex and index buffers. This value is used latter in the algorithm for optimization purposes.

    Note: You can use a grid structure huristic to limit the size of the batches. Batch together only data that can fit more or less in a given tile of a grid.

    Also note: We do not load the constructed buffers into hardware (render system) buffers. This is becuase certain graphic cards as well as Render systems like OpenGL do not handle threading well (at least not in Ogre), Plus as far as I know only directX11 is truly multi-threaded and only on newer cards.
  5. Worker thread: Once the buffers are ready the worker thread raises a flag which tells the main thread it has new information. The worker thread then puts itself to sleep.
  6. Main thread: The main thread holds a list of renderables called a display list. This list contains renderables derived from previous batched buffers in the working thread. Once the main thread discovers that the work thread has finished loading it marks all the renderables in the display list as "under examination". No visible change is performed due to this marking.
  7. Main thread: At this stage the main thread slowly loads the data gathered in the work thread into the graphic card (to prevent hiccups). Every frame we take a single buffer from the work list and:
    1. Using the buffers hash value we check whether we already have a renderable displaying the information of such a buffer. If so we mark the renderable as "in use" delete the buffer and its entry in the work list and continue on.
    2. If no such renderable is found we place the data from the buffer into a hardware buffers inside a new renderable object. We do this for both index and vertex data.
    3. we mark the newly constructed renderable as "in loading".
    4. we remove the buffer from the old work list and place the renderable in the display list.
    5. At this point even if you buffers are defined as dynamic (belonging to d3d unmanaged pool) the buffers are not automaticly loaded to the graphics card. We need to force them to load one by one. To do this we must use them in a rendering operation. A code is added so renderables marked as "in loading" will be placed in the position of the camere with a 0 scale matrix. That way the new rederables are drawn but not visible.
  8. Main thread: once all the buffers have been converted to renderables. Any renderables in the display list still marked as "under examination" is deleted. Any renderable marked as "in loading" is switched to "in use" (for normal rendering).
  9. Main thread: Repeat 1
There are a couple of other small tricks like using pools to save on new and delete operations, scanning only parts of the terrain that you know are going to change, etc... .But on the whole that's it.

The point I'm trying to make is that using static geometry might be helpful (I haven't really worked with that mechanism to work). But if you leave everything under your control you can end up with a better working mechanism. I'm not saying you have to implement this. But at least consider to leave the code open enough so other people can write this type of algorithm.


Update:
I would very much like to hear some input from you in the next couple of days.
Did my comments answer your original question? Are you in anyway stuck? Do you know where you want to take your code to?
Also, your wiki states 5 weeks of "Testing, bells & whistles, refactoring, optimizing, documentating, buffer". Since you are very close to finishing your original proposal content this is not prohibited. But a little clarification would be appreciated.
it's turtles all the way down

PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

Hi,

yes, I think the question is answered. :) So the way to go will be to use some kind of shared geometry between chunks (Maybe one single big vertex buffer and the chunks hold just indices). Tomorrow I'll have a look how this can be done and have a deeper look to your detailed answers.
You asked why I'm using one SceneNode per chunk: This way, culling works and throws away lots of unseen geometry, depending on the viewers position. So:
- First, I refactor the Chunks now to share geometry.
- Based on the gained knowledge there, I might re-introduce some kind of multiple SceneNodes (like only for the first half of the chunk tree) to experiment a bit around whether this is usefull.

Your dynamic terrain system sounds really impressive. Something like this could be one subproject of this whole thing in the feature (as I'm planing to further develop this after the GSoC). :)

To the roadmap: I just updated it with this refactoring coming up as the very next task from tomorrow on. One of the buffer-weeks is gone now. They are exactly for this kind of changes and for some more features which might be nice but don't have any priority for now. For example, a XML format for the definition of the source (nesting all types of Sources). Or the above mentioned Source which (somehow) converts a mesh to a volume. There is much to imagine. :)
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.

User avatar
Mattan Furst
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 260
Joined: Tue Jan 01, 2008 11:28 am
Location: Israel
x 32

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by Mattan Furst »

Just a couple of quick things:
  • You must remove the unneeded geometry before batching the geometry together. Once you batch things together it will be very hard to see it. (see Perfhud image 3 posts up).
  • About having more scene nodes. I think individual renderables can be removed based on their own bounding box. This might not be something the SimpleRenderable implements but I think other renderables do it during the call to _notifyCamera(). I'll get back to you on that.
  • both vertex and index buffer need to be aggregated together. Otherwise it wont be the same batch. If you have 2 chunks containing a single triangle: Batch 1 with vertexes {v0,v1,v2} and indexes {0,1,2}, Batch 2 with vertexes {v1, v3, v4} and indexes {0,1,2}. The final batch needs to be {v0,v1,v2,v3,v4} with indexes {0,1,2,1,3,4}.
p.s.
You must have done your post at 1:40 at midnight. Do you actually go to sleep at some point? :)
it's turtles all the way down

PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

This unneeded geometry is there to patch the cracks occuring between DMC chunks with different resolutions and different octrees. Just like skirts in heightmap terrain. But until now, it naivly just generated triangles at the whole side (as you have seen too) which actually wasn't needed.
So now I changed the marching squares function which does this and added a maxDistance parameter. If all 4 corners of the square are inside the volume, it now checks if their distance to the surface is greater than the maxDistance. If so, no triangles are generated.
This maxDistance is based on the geometric error of the chunk with a factor.
The practical effect is, that the chunks aren't closed anymore but have some kind of skirts.
The number of saved triangles are far greater than expected! The current testscenario with the terrain and the bridge made out of CSGSpheres from the last screenshot:
Before: 329,601 triangles generated for all LOD levels
After: 190,973 triangles generated for all LOD levels
42% less triangles, yay!

ps:
I usually go to bed quite late but sleep long. The time of your posting would be horrible for me. :D
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.

User avatar
Mattan Furst
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 260
Joined: Tue Jan 01, 2008 11:28 am
Location: Israel
x 32

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by Mattan Furst »

Is there a way to easily turn of the skirts. I would like to investigate this issue a bit if possible?
it's turtles all the way down

PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

Chunk.cpp, comment out everything of the loop-body below line 52 so it reads just:

Code: Select all

    for (size_t i = 0; i < count; ++i)
    {
        DualCell cell = dualGridGenerator.getDualCell(i);
        corners = cell.getCorners();
        values = cell.hasValues() ? cell.getValues() : 0;
        is.addMarchingCubesTriangles(corners, values, mb);
    }
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.

strich
Gnoblar
Posts: 10
Joined: Fri Jun 29, 2012 5:22 am

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by strich »

Seriously excited about this! Amazing work so far Phillip, keep it up.
It disappoints me a little that Dual Contouring isn't used as the main algorithm for your proposal scope as sharp features would be really nice. Do you think it'd be hard to implement post-GSoC? Is it high up on your TODO?

P.S. Is this project using PolyVox as storage for the volume? It would seem like the perfect fit.

PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

Thanks! :D
The Dual Contouring part would be the (performance heavier) feature isolation from the octree with QEFs. This might be indeed something for later. Currently not really on my list as it isn't really needed for terrain imho. But I'm aware of it. :)

Polyfox isn't used at all as the volume data isn't needed anymore after generating the chunks.
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.

PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

Scratching itches.
  • Changed the chunks from being closed with marching squares to just some skirts, also with marching squares. This saves 42% of the triangles for the testscenario!
  • A bit more tweaking on the skirt normals. Some blending can be configured now from the outside.
  • Introduced ChunkParameters as parameter of Chunk::load to clean up this giant amount of parameters a bit.
  • The drawing octree leaves and their parents have the same size now. So the node before the leave doesn't split up into eight children but into one. The eight would be all drawn anyway so they can be one mesh. Saves again a lot of triangles (no more skirts here) and reduces the batchcount to something usable.
  • Found an error in the OctreeNodeZhangSplitPolicy causing those big differences between chunks.
  • Found an error in the Triplanar Texturing shader causing some bad lighting.
  • Added the point "Loading basic (non-CSG) terrain from a config file." to the roadmap.
  • Added the point "Have a deep look at the WorkQueue to parallelize loading of chunks." to the roadmap.
As the geometry and lighting is now nice and smooth, I initially wanted to post a video today. But unfortunatly, my router died yesterday due to a thunderstorm here and I'm on a crappy UMTS connection for the next days. So only a screenshot again.
Image
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.

Post Reply