Editable Terrain Manager [v2.2 released]

[CABAListic]

Ok, I've finally released the reworked version 2 of the Terrain Manager
See the first post for updated details.

To be honest, I was lazy and didn't test everything in full detail, but I think that the more important features should work well enough. I've also finally added some bits of documentation so as not to let you struggle completely in the dark.


I'm still having issues with OpenGL rendering, for the sake of it, I can't get splatting to work with 6 textures per pass without producing artifacts in the OpenGL rendering. The included demo will show this if you run it in OpenGL. Any insight on this would be much appreciated.

[KungFooMasta]

Awesome CABAListic!

Downloading now. . .

If I knew anything about how splatting worked I would definately help you out, the more textures per pass the better.

Thanks for releasing version 2! Hopefully you can tolerate my questions, since I will be working on terrain collision as soon as I read over/digest some of ETM.

KungFooMasta

[KungFooMasta]

Wow, great job commenting the code, it looks very slick!

I'm trying to figure out how to attack making a terrain collision object using ETMv2.

Here is the function OgreOpcode uses on the TSM:

Code: Select all

void OgreOpcodeTerrainExample::pageConstructed(TerrainSceneManager* pSceneMgr, size_t pagex, size_t pagez, Real* heightData) 

pSceneMgr is used to get the terrain Scale, TileSize and PageSize.

I don't understand the difference between a page and a tile, but I know that ETM does not support Paging. This is fine, but I need to figure out how to convert this function to ETM equivalent.

I think we need ETM::TerrainManager::getTileSize() function.
Also ETM::TerrainManager::getNumberOfTiles().

Also, what is "Ogre::IndexData", in the context of

Code: Select all

Ogre::IndexData* Tile::getIndexData();

I think I will have to iterate through each tile and create a collision object using the index data of the tile.

Will LOD morph affect the indices? If so, I would either have to recreate these collision objects a lot, or disable LOD Morph.

KungFooMasta

[CABAListic]

Well, I hope there's a different way to set up an OpCode collision object than to access the tiles themselves, because that would get kinda ugly. They are not even exposed publically in the DLL, and I would rather keep it that way.
Your best bet would be to use the heightmap data from a TerrainInfo object.

About LOD morphing: It alters the vertices, but only for display. This does not affect anything logic-wise, so will not hurt you in any way

[KungFooMasta]

Ok, that sounds like a more simple approach anyway.

I guess I will experiment with trying to make one collision object for the entire terrain, iterating through the heightmap data and multiplying each height by the terrain scale, or something like that.

If I'm lucky, the OgreOpcode master (Jacmoe) will drop in and give me some help.

KungFooMasta

[jacmoe]

'pageConstructed' is a member of a TerrainListener and is called by the TSM when a page is constructed (Duh!).

I think we need ETM::TerrainManager::getTileSize() function.
Also ETM::TerrainManager::getNumberOfTiles().

These functions are generally implemented through the generic getOption's, but for some reason they are not implemented in the TerrainSceneManager.
This might be an oversight,..
The getOption scheme allows us to get information from a specific SM without linking to it (well, one way of putting it).
The DotSceneOctree SM implements that quite nicely - see the OgreOpcode DSO demo for a more elegant Ogre-like way of querying for world fragment data.
I hope CABAListic will go that route too.

But, I'll have to check out a TerrainInfo object..

[CABAListic]

Yeah, but ETM is no longer a scene manager, so it does not implement getOption or setOption, instead it exposes anything it wants to as a public member function

But from my point of view the terrain tiles and even more so their data are internals of the library which are not meant to be exposed, and it would, to some extent, screw its design to do so. So unless for very, very good reasons I'd rather not do that.

But since I don't know Opcode or OgreOpcode, I can't really comment on its functioning and alternatives...

[jacmoe]

Well, we will cook something up! ETM is very desireable to us!

[KungFooMasta]

I'm trying to setup my apps to create and use ETM terrain. Using the demo as an example, I don't see the relation involving the splatting images, between the Splatting Manager and the Terrain Manager.

Code: Select all

 // create the splatting manager
      mSplatMgr = new ET::SplattingManager("ETSplatting", "ET", 128, 128, 3);
      // specify number of splatting textures we need to handle
      mSplatMgr->setNumTextures(6);

Code: Select all

// load the terrain material and assign it
      MaterialPtr material (MaterialManager::getSingleton().getByName("ETTerrainMaterial"));
      mTerrainMgr->setMaterial(material);

I see that the splatting textures are defined in the material "ETTerrainMaterial", and also that the Splatting Manager says it will be using 6 textures for this terrain. But how does the SplattingManager know what splatting textures are defined for use? How does the Splatting Manager know what material is being used for the terrain?

Also, does this mean every splatting texture needs to be defined in a material before use?

Thanks for any information!

KungFooMasta

[CABAListic]

I see that the splatting textures are defined in the material "ETTerrainMaterial", and also that the Splatting Manager says it will be using 6 textures for this terrain. But how does the SplattingManager know what splatting textures are defined for use? How does the Splatting Manager know what material is being used for the terrain?

Also, does this mean every splatting texture needs to be defined in a material before use?

Simple answer: The SplattingManager doesn't know! It has nothing to do with actual materials, all it does is manage the coverage textures in a way allowing you to edit them. What you do with them is entirely up to you.
So yes, you need to set up a material which uses the coverage maps handled by the SplattingManager and which defines the actual splatting textures. For that, you can either use a material script or write your own dynamic material handler, whatever you need - it's up to you

[KungFooMasta]

So can we assume that each coverage map represents a material pass? (Sorry for extremely noob questions, hopefully others benefit as well! )

So in the demo, you set 3 textures per pass, and use 6 textures, resulting in 2 passes, and 2 coverage maps.

I see there is ETcoverage.0.png and ETcoverage.1.png.

But the material looks like this:

Code: Select all

texture_unit
      {
        // first coverage map, dynamically managed
	texture ETSplatting0
      }
      texture_unit
      {
        // second coverage map, dynamically managed
	texture ETSplatting1
      }

Are those just dummy placeholders to be replaced with coverage maps?

I like the flexibility you've added in. More for the user to manage, but we can manage as we like. I just need to figure out how it all works.

KungFooMasta

[CABAListic]

Nah, the SplattingManager does not care how you use the generated coverage maps. You could create a 4-channel map and use a shader with it and 4 splatting textures to splat 4 textures per pass.

But I like to use 3-channel maps, because this allows for two things:
- You can make it backward-compatible to pixel shader 1.1 versions, because using all 4 texture units you can splat 3 textures per pass
- You can use 2 maps and 6 splatting textures to splat 6 textures per pass on pixel shader 2.0

The demo material implements both. If you have a PS 2.0 capable card, then you'll get 6 textures per pass, i. e. all 6 textures are rendered in one pass. If it is not 2.0, it will fall back to do 2 passes with 3 textures each.

However, the 6-texture pass is buggy on OpenGL and will distort the rendering of anything else except the terrain, for reasons I do not understand (but I would tend to call it a bug in the OpenGL renderer ).


EDIT: The names of the splatting maps as manage by the SplattingManager are of the form BaseNamex, where the BaseName is what you initialise the SplattingManager with (in the demo: "ETSplatting") and the x is the index of the map. So therefore the two textures are named ETSplatting0 and ETSplatting1, which is what I reference in the material script. The ETcoverage.x.png are just the saved offline versions, but are not used by the material script.

[CABAListic]

Just took a quick glance at OgreOpcode. I can't compile the demos, so I didn't test it, but from that code part in the terrain demo, I figure something like this should work:

Code: Select all

void OgreOpcodeTerrainExample::constructTerrainCollisionShape(const ET::TerrainInfo& info) 
{
	Vector3 offset = info.getOffset();
	Vector3 scale = info.getScaling();
	//Create indices 
  size_t index_size = (info.getWidth()-1)*(info.getHeight()-1)*6;
	int* indices = new int[index_size]; 
	for (size_t x=0;x<info.getWidth()-1;x++)
	{ 
		for (size_t y=0; y<info.getHeight()-1;y++) 
		{ 
			indices[(x+y*(info.getWidth()-1))*6] = (x+1)+(y+1)*info.getWidth(); 
			indices[(x+y*(info.getWidth()-1))*6+1] = (x+1)+y*info.getWidth(); 
			indices[(x+y*(info.getWidth()-1))*6+2] = x+y*info.getWidth(); 

			indices[(x+y*(info.getWidth()-1))*6+3] = (x+1)+(y+1)*info.getWidth(); 
			indices[(x+y*(info.getWidth()-1))*6+4] = x+y*info.getWidth(); 
			indices[(x+y*(info.getWidth()-1))*6+5] = x+(y+1)*info.getWidth(); 
		} 
	} 

	// Create vertices
  size_t vertex_size = info.getWidth()*info.getHeight();
	float* vertices = new float[vertex_size*3];
  size_t index = 0;
	for(size_t z = 0; z < info.getHeight(); z++)
	{
		for(size_t x = 0; x < info.getWidth(); x++)
		{
			Real height = offset.y + info.at(x, z)*scale.y;
			vertices[index + 0] = offset.x + x*scale.x;
			vertices[index + 1] = offset.y + info.at(x,z)*scale.y;
			vertices[index + 2] = offset.z + z*scale.z;
			index += 3;
		}
	}

	// create new shapes here!
	PtrCollisionShape* tempTerrainShape;
	CollisionObject* tempCollObject;
	String shapeName = "terrainShape";
	tempCollObject = mCollideContext->createObject(shapeName);
	tempTerrainShape = CollisionManager::getSingletonPtr()->createPtrCollisionShape(shapeName);
	tempTerrainShape->load(vertex_size, index_size, vertices, indices);
	tempCollObject->setCollClass("level");
	tempCollObject->setShape(tempTerrainShape);
	mCollideContext->addObject(tempCollObject);
	delete[] vertices;
	delete[] indices;
}

As stated, I didn't test this, I don't even know if it compiles Also, in contrast to the Opcode terrain demo, this creates one shape for the entire terrain instead of per tile basis. I don't know if that creates performance problems. You can naturally cut it into pieces any way you like.

[KungFooMasta]

Wow, thanks for looking into this! I will post back with the results of trying this method, or any modifications to make it work, when I get there. Still working on creating/loading/saving ETM terrain.

KungFooMasta

[jacmoe]

CABAListic, you're the man. Thanks!

[KungFooMasta]

I'm thinking up the design for a "TerrainMaterial" class that I want to implement. Basically it creates and manages a material in memory, and can save to and load from disk. I think it will contain the Splatting Manager inside it. Before I get too far, I want to point out some information, please tell me if it is inaccurate.

- Splatting Manager creates, configures, saves and loads coverage maps to/from disk.
- Materials define use of coverage maps and splatting textures. Coverage maps can make use of 1-4 splatting textures, based on RGBA channels.
- Within each material pass, the first set of texture unit states should be of the coverage maps. The following texture unit states are of the Splatting textures. Each pass must have reference to the vertex/fragment programs, before texture unit states are defined.
- Using the supplied shaders/programs, the max amount of splatting textures per material pass is 4.
- After painting, the Splatting Manager updates the coverage map in memory, which should be used by the current material applied to the terrain. The Splatting Manager can't re-apply the material using setMaterial because it does not know what material is being used, and has no knowledge of the terrain.

KungFooMasta

[CABAListic]

That is more or less correct, I'd say (even though technically, coverage maps don't make use of anything, they are used by materials or more precisely shaders, usually).
But re to the last point: There's no need to touch the material after painting. If the material reference the dynamic textures that the splatting manager manages, then the changes to those textures will automatically apply to the rendering.

[KungFooMasta]

I'm trying to create an initial material for the terrain. In the demo, in the createScene method, you have:

Code: Select all

MaterialPtr material (MaterialManager::getSingleton().getByName("ETTerrainMaterial"));
      mTerrainMgr->setMaterial(material);

In my scenario, the user will choose to create a terrain, and will supply:

1) either defaultHeightmap or .raw file
2) terrain dimensions, ie 1500,300,1500 by default
3) choose a base splatting texture for the entire terrain

So the process is that when you create a *terrain* object, you also create a default material. This material will by default be configured with the baseTexture chosen by the user. The problem is, I'm not sure how to create this default material.

Here is my starting code:

Code: Select all

void Terrain::createNewMaterial(const Ogre::String& name, const Ogre::String& baseTexture)
	{
		mTerrainMaterial = Ogre::MaterialManager::getSingleton().create(name,"Terrain");

		Ogre::Technique* t = mTerrainMaterial->getTechnique(0);
		Ogre::Pass* p = t->getPass(0);
		p->setLightingEnabled(false);
		p->setVertexProgram("ET/Programs/VSLodMorph2");
		p->setFragmentProgram("ET/Programs/PSSplat2");
	}

The first problem is that I don't know how to set the FragmentProgram parameters. Also, if I tell the splatting manager to setNumTextures to 1, it will create a coverage map.. do I need to write this map out to file, and then add it to the material via createTextureUnitState?

ETTerrainMaterial:

Code: Select all

pass
    {
      // splatting pass
      
      lighting off
      
      vertex_program_ref ET/Programs/VSLodMorph2
      {
      }
      
      fragment_program_ref ET/Programs/PSSplat2
      {
        param_named splatScaleX float 20
	param_named splatScaleZ float 20
      }
      
      texture_unit
      {
        // first coverage map, dynamically managed
	texture ETSplatting0
      }
      ...

Thanks in advance,

KungFooMasta

[KungFooMasta]

I think this is somewhat closer, just missing the fragment program arguements.

Code: Select all

void Terrain::createNewMaterial(const Ogre::String& name, const Ogre::String& baseTexture)
	{
		// creates a coverage map in memory - name + "0"
		mSplattingManager->setNumTextures(1);
		mTerrainMaterial = Ogre::MaterialManager::getSingleton().create(name,"Terrain");

		Ogre::Technique* t = mTerrainMaterial->getTechnique(0);
		Ogre::Pass* p = t->getPass(0);
		p->setLightingEnabled(false);
		p->setVertexProgram("ET/Programs/VSLodMorph2");
		p->setFragmentProgram("ET/Programs/PSSplat2");

		// add coverage map
		p->createTextureUnitState(name+"0");

		// add splatting texture
		p->createTextureUnitState(baseTexture);
	}

[CABAListic]

First off, if you intend to use the PS2.0 version of the shader from the example, you will have to give 2 coverage maps, or you won't get what you are looking for.
So call mSplattingManager->setNumMaps(2); instead, and add the second map as the second texture unit state before adding the splatting texture.

As for setting the parameters, look at the original ETSM v1 in the material handler, there's code for that there, I don't remember it in detail

[KungFooMasta]

Thanks CABAListic, I found the code in the ETSM v1 source, and used that as a base. This is what I'm currently seeing:



Here is my code:

Code: Select all

void Terrain::createNewMaterial(const Ogre::String& name, const Ogre::String& baseTexture)
	{
		// creates a coverage map in memory - name + "0"
		mSplattingManager->setNumMaps(2);
		mTerrainMaterial = Ogre::MaterialManager::getSingleton().create(name,"Terrain");
		
		// create lightmap
		mTerrainLightmapTexture = Ogre::TextureManager::getSingleton().createManual(mName+".Lightmap", "Terrain", Ogre::TEX_TYPE_2D, 128, 128, 1, Ogre::PF_BYTE_RGB); 
		ET::createTerrainLightmap(*mTerrainInfo, mTerrainLightmap, 128, 128, Ogre::Vector3(1, -1, 1), Ogre::ColourValue::White,Ogre::ColourValue(0.3, 0.3, 0.3));
		mTerrainLightmapTexture->getBuffer(0, 0)->blitFromMemory(mTerrainLightmap.getPixelBox(0, 0));

		mTerrainMaterial->setLightingEnabled(false);
		Ogre::Technique* t = mTerrainMaterial->getTechnique(0);
		Ogre::Pass* p = t->getPass(0);

		// FIRST PASS

		// add lightmap
		p->createTextureUnitState(mTerrainLightmapTexture->getName());

		// add coverage map
		p->createTextureUnitState(name+"0");
		p->createTextureUnitState(name+"1");

		// add splatting texture
		p->createTextureUnitState(baseTexture);

		p->setFragmentProgram("ET/Programs/PSSplat2");
		Ogre::Real val = 20.0; 
		Ogre::GpuProgramParametersSharedPtr fpParams = p->getFragmentProgramParameters();
		if(fpParams->_findNamedConstantDefinition("splatScaleX")) fpParams->setNamedConstant("splatScaleX", val);
		if(fpParams->_findNamedConstantDefinition("splatScaleY")) fpParams->setNamedConstant("splatScaleY", val); 

		p->setVertexProgram("ET/Programs/VSLodMorph2");
		Ogre::GpuProgramParametersSharedPtr vpParam = p->getVertexProgramParameters();
        vpParam->setNamedAutoConstant("worldViewProj", Ogre::GpuProgramParameters::ACT_WORLDVIEWPROJ_MATRIX);
		vpParam->setNamedAutoConstant("morphFactor", Ogre::GpuProgramParameters::ACT_CUSTOM, 77);
	}

And here is when I actually apply the material:

Code: Select all

mSceneManager = CurrentSceneManager;
			mTerrainManager = new ET::TerrainManager(mSceneManager,mName);
			mTerrainManager->setLODErrorMargin(2, mSceneManager->getCameraIterator().getNext()->getViewport()->getActualHeight());
			mTerrainManager->setUseLODMorphing(true, 0.2);
			mTerrainManager->createTerrain(*mTerrainInfo);
			mTerrainManager->setMaterial(mTerrainMaterial);

You can see in my UI window the minimal options to creating a terrain. I want to use those to create a default material and apply it to the terrain.

KungFooMasta

[CABAListic]

Well, the lightmap does not belong in the same pass as the splatting, the shader is not programmed to deal with it. The way you are setting up your texture units, the first two texture units (which to the shader present the coverage maps) are now the lightmap and the first coverage map, whereas they really should be both the coverage maps.

To use the lightmap, setup an additional pass at the end - or use a different shader which is written to incorporate the lightmap. You could use the one from the original ETSM, that one is meant for one RGBA coverage map with four splatting textures and the lightmap...

[KungFooMasta]

Yah, looking into the v1 and v2 materials, I see some differences.

I have implemented a different method, although the visual results are still the same! Basically, I have a function called "_rebuildMaterial" which is called whenever the user adds or removes a splatting texture. It should clear everything in the material and recalculate/recreate the passes, texture unit states, etc.

Code: Select all

void Terrain::_rebuildMaterial()
	{
		// Remove all passes
		Ogre::Technique* t = mTerrainMaterial->getTechnique(0);
		t->removeAllPasses();
		mTerrainMaterial->setLightingEnabled(false);

		// Determine the number of coverage maps needed - 4 textures per pass
		int numSplattingTextures = static_cast<int>(mSplattingTextures.size());
		int numPasses = 1;
		if( numSplattingTextures > 8 ) numPasses = 3;
		else if( numSplattingTextures > 4 ) numPasses = 2;
		
		// 2 coverage maps per pass
		mSplattingManager->setNumMaps(numPasses * 2);

		// Created calculated number of passes
		while( t->getNumPasses() < numPasses )
			t->createPass();
		
		// Iterate through passes and add:
		// lightmap, coverage maps, splatting textures, vertex/fragment programs
		for( int i = 0; i < t->getNumPasses(); ++i )
		{
			Ogre::Pass* p = t->getPass(i);

			// lightmap
			//p->createTextureUnitState(mTerrainLightmapTexture->getName());

			// coverage maps - PS 2.0 version requires 2 coverage maps per pass
			p->createTextureUnitState(mName + Ogre::StringConverter::toString(i * 2));
			p->createTextureUnitState(mName + Ogre::StringConverter::toString((i + 1) * 2));

			// splatting textures
			for( int j = i*4; j < (i+1)*4 && j < static_cast<int>(mSplattingTextures.size()); ++j )
				p->createTextureUnitState(mSplattingTextures[j]);

			if(i != 0) p->setSceneBlending(Ogre::SBT_ADD);

			p->setFragmentProgram("ET/Programs/PSSplat2");
			Ogre::Real val = 20.0; 
			Ogre::GpuProgramParametersSharedPtr fpParams = p->getFragmentProgramParameters();
			if(fpParams->_findNamedConstantDefinition("splatScaleX")) fpParams->setNamedConstant("splatScaleX", val);
			if(fpParams->_findNamedConstantDefinition("splatScaleY")) fpParams->setNamedConstant("splatScaleY", val); 

			p->setVertexProgram("ET/Programs/VSLodMorph2");
			Ogre::GpuProgramParametersSharedPtr vpParam = p->getVertexProgramParameters();
			val = 77;
			if(vpParam->_findNamedConstantDefinition("worldViewProj")) vpParam->setNamedAutoConstant("worldViewProj", Ogre::GpuProgramParameters::ACT_WORLDVIEWPROJ_MATRIX);
			if(vpParam->_findNamedConstantDefinition("morphFactor")) vpParam->setNamedAutoConstant("morphFactor", Ogre::GpuProgramParameters::ACT_CUSTOM, val);
		}
	}

For now, I'm taking out all references to the lightmap. I'm still seeing the textures stretched out, I have no idea why this is. Also, my FPS is suddenly 20 this time around. I don't know if I have too many visual studio projects open, my computer is acting up, or this new code is doing bad things. Do you think the removeAllPasses function properly cleans up the texture unit states created? Although, I am only calling this function once, so the performance I'm seeing shouldn't be related.

Right now I'll be happy to get a terrain loaded with just one splatting texture..

KungFooMasta

[KungFooMasta]

FPS problem solved..

Code: Select all

p->createTextureUnitState(mName + Ogre::StringConverter::toString((i * 2) + 1));

Simple mistake!

Now getting 300 FPS.

But the terrain still looks striped! Also, I'd like to note that without the if statements in the following code, the program crashes:

Code: Select all

if(fpParams->_findNamedConstantDefinition("splatScaleX")) fpParams->setNamedConstant("splatScaleX", val); 
         if(fpParams->_findNamedConstantDefinition("splatScaleY")) fpParams->setNamedConstant("splatScaleY", val);

This kind of tells me that "splatScaleX" and "splatScaleY" are not defined in "ET/Programs/PSSPlat2"?

Code: Select all

fragment_program ET/Programs/PSSplat2 cg
{
  source PSSplat2.cg
  entry_point main
  profiles ps_2_0 arbfp1
}

No idea if that is related.. Hope you can spot the problem.

[CABAListic]

It's splatScaleZ, not splatScaleY

Your calculation for the number of passes seems strange. You are using 3-channel maps, are you not? Then with the PSSplat2 shader you are actually rendering 6 splatting textures per pass, so wrapping at 4 or 8 makes no sense.

As for the look, that's most likely the error above that you're missing to set splatScaleZ.