[GSoC 2012] Volume Rendering with LOD aimed at terrain

Threads related to Google Summer of Code
Post Reply
CABAListic
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 2903
Joined: Thu Jan 18, 2007 2:48 pm
x 58
Contact:

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by CABAListic »

Basically the question is: Does a user (programmer) have to include header files and link to the library to use it? If so, it's useless as a plugin, because the one thing a plugin allows is loading a library at runtime. But if you need header files etc., then you already have to link at compile time, so this gains you nothing.

There is kind of a runtime interface for plugins, so you can interact with them without having to use their header files and link to their library explicitly, but this interface is limited and should only be used for very, very simple things. If you need more, then a component it is :) Components are still separate from OgreMain, so they don't harm anyone who doesn't use it.
PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

Ok, that's good. I separated it today as a component because of this very reasons. An user has to include some header files to actually use it. :) That's also how I came up with the idea to make plugins possible for the Source interface with this factory pattern suggested on the wiki page where specific/new factories can be registered at runtime coming from plugins (similar to how new SceneManagers or RenderSystems are registered at runtime). This is something I'd like to do after the RTTS. :)
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.
PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

Some cleanup and separation.
  • Updated the roadmap (again) and moved the plugin creation upfront the RTTS.
  • Just hashing the vertex position in the MeshBuilder, removed the normal here. Gives a nice loading time reduction and should be OK in this case.
  • Removed the not working QEF octree split policy. Removed the virtual part of this class for a small performance gain and also removed the dependency to the CML.
  • Renamed all files according to the Ogre conventions.
  • Moved all files except the actual Sample ones to a new Component "OgreVolume".
(No screenshot today, nothing new visually)
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.
PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

Hi,

I wanted to setup everything for the upcoming RTSS triplanar texturing but ran into a strange problem:
For now, my factory just always returns 0 (as the SRS doesn't exist yet). I'm setting everything up like in the fog demo:

Code: Select all

    RTShader::ShaderGenerator* mGen = RTShader::ShaderGenerator::getSingletonPtr();
    mGen->setTargetLanguage("cg");
        
    RTShader::RenderState* pMainRenderState = 
        mGen->createOrRetrieveRenderState(RTShader::ShaderGenerator::DEFAULT_SCHEME_NAME).first;
    pMainRenderState->reset();

    mTriplanarTexturingFactory = new RTShader::TriplanarTexturingFactory();
    mGen->addSubRenderStateFactory(mTriplanarTexturingFactory);
    pMainRenderState->addTemplateSubRenderState(
        mGen->createSubRenderState(RTShader::TriplanarTexturing::Type));	
        
    mGen->invalidateScheme(Ogre::RTShader::ShaderGenerator::DEFAULT_SCHEME_NAME);

    // Make this viewport work with shader generator scheme.
    mViewport->setMaterialScheme(RTShader::ShaderGenerator::DEFAULT_SCHEME_NAME);
For all already existing parts, I thought the FFP emulation of the RTSS takes over. In this case, the Skydome and the UI. But they are completly white now (see attached screenshot)?
Attachments
screenshot08032012_004215583.png
screenshot08032012_004215583.png (154.53 KiB) Viewed 6156 times
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.
User avatar
Mattan Furst
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 260
Joined: Tue Jan 01, 2008 11:28 am
Location: Israel
x 32

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by Mattan Furst »

I have a splitting head ache so I'll keep this short.

I think this means your on the right track. If you look at the funtion of the SRS getExecutionOrder() it returns FFP_TEXTURING. This means that it replaces the previous SRS (the one meant to emulate texture color polling of the fixed pipeline). This means that you don't get the color of the texture. The lighting stage still works which is why you still get darker and brighter areas.

If you want your SRS to only influence your terrain you need to remove the SRS from automatically adding itself to all materials. meaning remove the lines

Code: Select all

mGen->addSubRenderStateFactory(mTriplanarTexturingFactory);
    pMainRenderState->addTemplateSubRenderState(
        mGen->createSubRenderState(RTShader::TriplanarTexturing::Type));
than add the SRS specifically to your material by adding the lines in the terrain material like

Code: Select all

material {
   technique {
      pass {
        ...
        ...
        rtshader_system
        {
             volume_terrain true
        }
      }
   }
}
Than read the code in the factory function FFPTexturingFactory::createInstance. it should be something like:

Code: Select all

SubRenderState* XXXXXXXX::createInstance(ScriptCompiler* compiler, PropertyAbstractNode* prop, Pass* pass, SGScriptTranslator* translator)
{
	if (prop->name == "volume_terrain")
	{
		if(prop->values.size() == 1)
		{
			bool active;

			if(false == SGScriptTranslator::getBool(prop->values.front(), &active))
			{
				compiler->addError(ScriptCompiler::CE_INVALIDPARAMETERS, prop->file, prop->line);
				return NULL;
			}

			if (active)
			{
				return createOrRetrieveInstance(translator);
			}
		}		
	}

	return NULL;
}
As for implementing the shader. The inside implementation is up to you, but the "output" of the SRS should contain a line that defines the output color of the shader.

Code: Select all

MyOutputParam = psMain->resolveOutputParameter(Parameter::SPS_COLOR, 0, Parameter::SPC_COLOR_DIFFUSE, GCT_FLOAT4);
And this value needs to be filled with the color of texture you want.

Update:
P.S.
There is a wiki article on the RTShader in http://www.ogre3d.org/tikiwiki/tiki-ind ... der+System.
(I went over it and added a couple of things in the past week. I'm not sure everything is up to date. but all in all it's also a good starting point)
it's turtles all the way down
PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

Thx. :)
Ok, I removed those lines, now my initialization method of the RTSS looks like this:

Code: Select all

    RTShader::ShaderGenerator* mGen = RTShader::ShaderGenerator::getSingletonPtr();
    mGen->setTargetLanguage("cg");
    RTShader::RenderState* pMainRenderState = 
        mGen->createOrRetrieveRenderState(RTShader::ShaderGenerator::DEFAULT_SCHEME_NAME).first;
    pMainRenderState->reset();
    mTriplanarTexturingFactory = new RTShader::TriplanarTexturingFactory();
    mGen->invalidateScheme(Ogre::RTShader::ShaderGenerator::DEFAULT_SCHEME_NAME);
    mViewport->setMaterialScheme(RTShader::ShaderGenerator::DEFAULT_SCHEME_NAME);
But the implementation of createInstance(ScriptCompiler* compiler, PropertyAbstractNode* prop, Pass* pass, SGScriptTranslator* translator) is never called. I think it's because now the RTSS actually doesn't know anything about the TriplanarTexturingFactory. Also, when I add this line:

Code: Select all

mGen->addSubRenderStateFactory(mTriplanarTexturingFactory);
To just register the factory after the reset()-call, it never gets called.

Ou, and get well soon. :)
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.
User avatar
Mattan Furst
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 260
Joined: Tue Jan 01, 2008 11:28 am
Location: Israel
x 32

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by Mattan Furst »

I'm sorry, I miss wrote.

You still need the line

Code: Select all

mGen->addSubRenderStateFactory(mTriplanarTexturingFactory);
so the RTSS will be aware of you
you don't need the line

Code: Select all

 pMainRenderState->addTemplateSubRenderState(
        mGen->createSubRenderState(RTShader::TriplanarTexturing::Type));  
but that probably won't work either. You need the first line to be called before the materials are parsed from the disk.
Try to add the SRS as part of the RTSSs' extended SRS list (as part of the RTSS system not as part of your sample).

Update:
You can see the extended SRSs' factories being added to the RTSS system in the function "void ShaderGenerator::createSubRenderStateExFactories()" in the file ShaderGenerator.cpp.
it's turtles all the way down
PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

I think that did the trick. :)
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.
PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

Fixing stuff and preparing the RTSS-part.
  • Fixed a crash bug when stopping the sample.
  • Fixed a giant memory leak when loading a terrain from a config file.
  • Fixed a smaller memory leak when stopping the sample where the Chunk tree wasn't freed.
  • Fixed a memory leak where the geometry memory wasn't freed on destruction of the chunks.
  • Changed the sleeping in the wait-for-threads-loading-loop from OGRE_THREAD_SLEEP(50) to OGRE_THREAD_SLEEP(0) yielding in 33% smaller loading time!
  • Some foundation work for the RTSS triplanar texturing.
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.
PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

I just pushed a first working Triplanar Texturing SRS. Could you have a look at it if everything is cool?
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.
PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

Another thing: I'd love to do some procedural terrain generation with noise.
Here is an easily portable implementation of Improved Perlin Noise: http://cs.nyu.edu/~perlin/noise/
But: I emailed him twice about the terms and conditions, whether it would be allowed to be ported to Ogre under the MIT. But I got no answer. :(
So the options are:
a) Just to port it anyway.
b) Find another (Improved) Perlin Noise implementation where the license is clear.
c) Use a totally different noise algorithm.
What do you think? If c, do you know some? :)

// Edit: This looks good: http://stackoverflow.com/questions/6963 ... -algorithm A big explanation PDF is linked and in that PDF, a public domain 2012 Java implementation is linked. This could be ported easily!
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.
User avatar
aguru
Goblin
Posts: 236
Joined: Tue Feb 26, 2008 5:48 pm
x 3

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by aguru »

I have heard good things about "simplex" noise.
[edit]
Also, maybe this topic is of interest to you: http://www.ogre3d.org/forums/viewtopic.php?f=1&t=41293#
Noise++ source is here: http://sourceforge.net/projects/noisepp/

And they mention the "Accidental Noise Library", especially this might be of interest: http://accidentalnoise.sourceforge.net/ ... orlds.html
User avatar
Mattan Furst
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 260
Joined: Tue Jan 01, 2008 11:28 am
Location: Israel
x 32

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by Mattan Furst »

1.
I will take a look in the RTSS code this evening

2.
When I first was introduced to Ogre it had an LGPL license. The then Ogre team had worked really hard to move it to MIT license. non-MIT compatible code cannot be accepted to Ogre. And adding code that does not come with a specified license opens Ogre to future lawsuits. So no, you can't add code that you cannot positively know is MIT compatible.

3.
We are coming up to the official GSoC deadline (less than 2 weeks). Before the code can be added to the trunk it should go through a bit of testing. please ensure that:
  • The code compiles without unity build
  • The code compiles and runs without boost -The issue with non boot compilation has returned, at least It did a month ago.
  • The code compiles and runs in double precision mode - This seems to be a very commonly used setup which creates a lot of problems.
  • The sample runs in both directX and OpenGL - If for some reason this is not possible at least prevent the sample from loading an incompatible render system is working.
  • Ogre as a whole (the samplebrowser and all it's samples) is not affected.
I'm not stressed about when we merge and check-in the code, as far as I'm concerned it can be after the official dead line. so you don't have to do these things right away.

I've already given you writing permissions as per our PM exchange and discussion with the Ogre team. If you want to merge the code yourself go ahead. You will probably be able to better handle merging issues due to your knowledge of the code. Just give me a 1 days heads up if you want to do it and make sure that the above list is done.
it's turtles all the way down
User avatar
Mattan Furst
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 260
Joined: Tue Jan 01, 2008 11:28 am
Location: Israel
x 32

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by Mattan Furst »

Clarification to my last post:

My point was that except for merging the code to the trunk as far as I'm concerned you've completed all your responsibility to the GSoC project (and then some).
If you want to start another big development (perlin noise) now it might be better to first merge the code to the trunk before starting another project.
it's turtles all the way down
PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

Alright, makes sense. Then the very next things will be smaller development stuff, testing, optimizing, adding finally the MIT headers, etc.. :)

// Edit: Went through the bullet points and fixed some stuff with Unity, Boost and double precision. :)

// Edit2: Noticed that the loading times without Boost don't seem to be worse. I'll investigate further, maybe I can remove the dependency completly. :)

// Edit3: After some intensive tests: Ogres Maps reduce in this case the loading time by 25%. -> No more dependency to Boost and a significant faster loading time. Win-win!
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.
FlorianGeorge
Halfling
Posts: 86
Joined: Tue Sep 01, 2009 7:15 pm
Location: Cologne, Germany
x 4

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by FlorianGeorge »

Cool Project!

I saw this on the wiki page:
CSGXML

A xml format to define "scenes" outside of the code. Something like this:
CSGXML

<csg>
<union>
<sphere radius="5" x="5" y="6" z="7" />
<texture src="terrain.dds" width="200" height="200" depth="200" />
</union>
</csg>
That looks quite similar to dotScene.

I guess it would make sense to port this Terrain technique to Ogitor, either natively or as a plugin, then you'd have a great edit/import/export framework around this project.
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by Assaf Raman »

Can you fill this form and send it to e-mail at the end of the forum - so we can merge your work to the main repo?

Post on this thread after doing it - so I will know that you have.
Watch out for my OGRE related tweets here.
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by Assaf Raman »

This is the last week of the project, mentor and student - try to finish this project this week and merge it into the trunk if the code it ready for it.
Watch out for my OGRE related tweets here.
PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

Signed form is mailed. :) Now the weekly report:

Last actual roadmap feature done, optimizations and refactoring, right on time with the "pencils down" date.
  • Some microoptimizations to decrease the loading time a tiny bit.
  • First working triplanar texturing SubRenderState for the RTSS!
  • Moved some hardcoded triplanar texturing parameters to uniforms in the reference shader.
  • Removed the normalmapping part of the triplanar texturing RTSS roadmap as this would require to implement the whole FFP_LIGHTING execution. This is a bit over the top for this GSoC when looking at the NormalMapLighting SRS.
  • Sample Thumbnail and text updated.
  • Fixed build for non-unity-builds
  • Using Ogres maps as default and removed Boosts unordered maps, yielding in about 25% less loading time and removed the (optional) dependency to Boost!
  • Introduced a global scale parameter for easy huge terrain generation with lower resolution data, in this case a 2560x2560x2560 world from a 256x256x256 source.
  • Moved the max accepted pixelerror from static to regular class member and made it loadable from the config file.
  • Refactored the two methods of OgreVolumeUtils to other classes and so removed the now obsolete file.
  • Using 16Bit indices now with a fallback to 32Bit if the current mesh is bigger (doesn't happen in the testscenario), reducing memory and further 5% faster loading time.
  • Stripped down the sample and removed the development UI and the MC debug display. As I still need the MC debug display for some nice pictures in my thesis, I moved this to an own, separate application not in this repository.
  • Supporting visibility settings in the Chunk (Chunk::setVolumeVisible(), Chunk::getVolumeVisible()).
  • Refactored the debug visualization API of the octree and the dualgrid in the same way.
No new visuals again, sorry. But I plan a nice GSoC-done video. :)
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.
User avatar
Mattan Furst
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 260
Joined: Tue Jan 01, 2008 11:28 am
Location: Israel
x 32

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by Mattan Furst »

Hi PhilipLB,

I had mislead you in a previous exchange we had about comitting code to Ogre. This is a response from CABAlistic to a question I had on commiting chagesets and responsibilities:
By default, we merge upwards, that means that commits should be done in the lowest applicable branch. Bugfixes go to v1-8, (relatively) stable features to the next major version, i.e. v1-9. Due to our roadmap that exceptionally encompasses two major version, we also have v2-0 for anything targeted at that. default is only to be used for unstable changes (although these may be better to do in a separate repository and merged when somewhat stable).


This means that your code will go to v1-9 and not like I previously though to the trunk.

Given that you now you have check-in privileges to Ogre we need to agree on who is doing the merge, me or you. I no longer have a preference in the matter. What do you want to do?
it's turtles all the way down
User avatar
Mattan Furst
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 260
Joined: Tue Jan 01, 2008 11:28 am
Location: Israel
x 32

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by Mattan Furst »

Just a thought

Is there anything that I can provide you to help you to pass the thesis better?
it's turtles all the way down
PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

Starting today with some preparations for the merge and I'll do it tomorrow. :)
Thesis-Help: Thanks a lot for the offer, I might come back to this. :)
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by Assaf Raman »

When is the merge planned? I would like to see it soon.
Watch out for my OGRE related tweets here.
PhilipLB
Google Summer of Code Student
Google Summer of Code Student
Posts: 550
Joined: Thu Jun 04, 2009 5:07 pm
Location: Berlin
x 108

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by PhilipLB »

Hi,
the merge is on its way. The main repro was merged into the fork. The development happened in the default branch of the fork which was just merged into the 1.9 branch. This is now checked and when everything is ok, the fork will be merged into the main repro which should be painless.
Google Summer of Code 2012 Student
Topic: "Volume Rendering with LOD aimed at terrain"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Mattan Furst


Volume GFX, accepting donations.
User avatar
Mattan Furst
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 260
Joined: Tue Jan 01, 2008 11:28 am
Location: Israel
x 32

Re: [GSoC 2012] Volume Rendering with LOD aimed at terrain

Post by Mattan Furst »

It seems that adding merging the volume rendering GSoC project with Ogre's repository will increase the source control size from around 150 to 350 megs because of history with large volume textures.

1. Do you think that this is a price worth paying? or given that PhilipLB is the only one for which the history has any meaning should we remove it to conserve the space?

Personally I would say that given that we will always have the history in the original GSoC branch, I'd rather conserve the space.

2. Does anyone have any expirience with the mercurial collapse extension? especially within the bitbucket environment?
it's turtles all the way down
Post Reply