New Terrain Early Shots

[aguru]

I can report that shaders are working fabulously on my Linux box with a 8800gt.

I have a question concerning the creation of my .dat/.page files. Right now I just load in my pretty big terrain (10x10 a 1024, with 3 splatmaps and colourmap, about 1.1gig raw data) into the example and wait for the .dat files to be generated. Even then the frame rate is quite decent I then use the dats in my engine. But now I'm thinking about writing a proper, stand alone converter because my target resolution is much higher, I think something like 20x20 a 1024.

The problem is, I cannot just load in the whole dataset, it's just too large, and I cannot just generate the dats one page after another because for proper shadows the terrain component needs to know about neighboring pages. Any suggestions? Or should I just be a little more patient?

Thanks for any input

[sinbad]

@Xplodwild: simple: you're just using too many layers to be able to use shadows too. Clearly you were already using all the texture units up to 15 just with your existing options, you have no room for shadows too. You'll just have to live with fewer layers if you want shadows too. For example you could drop 1 layer and add a 2-split (not 3 as in the demo) PSSM option, since each layer uses 2 texture units. I'll be putting in some more obvious exceptions to raise that kind of condition to the user's attention.

Before you ask, yes it is possible to do shadows as a second pass instead. But, it's less efficient and I don't have time to implement that right now - you can do it as a separate material generator if you want. There are clearly infinite trade-offs between efficiency and flexibility and individual types of render approaches, which is why the material generators are pluggable. I only intend to provide a general-purpose one that covers the cases I'm interested in - anyone else can add new ones too.

@aguru: I suggest doing a once-off process of loading in all the pages in groups of 9 and saving them as .dat files. The neighbour relationships only typically rely on the immediate neighbours so you don't need all 20x20 in memory at once to properly process them, only 3x3. Once they've fully calculated, you can save the 'middle' ones (or the edges) and they are ready to go for real-time use and completely independent of each other without needing to all be loaded.

[Lee04]

I just tried the paging demo.

I get very simple green wire frame tiiles that dosn't resemble a landscape in both OpenGl and DirectX mode.

@Sinbad: Is this some test wire frame set up you have?

[theotherhiveking]

Thats the paging Demo, you must load the Terrain demo, set the slider to 26.
Also, uncomment the paging in the include file of the Terrain demo.
I hope that was what you meant.

[sinbad]

Yes, I'll be dropping the paging demo now I no longer need it, it was just a quick proof.

The terrain paging isn't really fast enough for my liking yet which is why it's disabled by default. I've implemented some optimisations with buffer pooling which have removed the majority of the latency on the GPU update (which is in the main thread) but I'm still getting a hiccup during the loading process that I can't explain at the moment, given that it's happening in a background thread and other background tasks (like dynamic updating of lightmaps) are smooth as silk. I'm out of time for the moment, it'll need to be refined over the life of 1.7.

[Lee04]

I get 13-15 FPS with the terrain almost over the total screen 1600x1000. It's a NVidia Quadro Go FX1400 in a Precision Dell laptop (Well the name sounds fast at least.)
It was framerates like these that started me looking at HOQs and other means of optmizations. One thing that caught my eye was in GPU Gems 2 Chapter 28 "Mipmap-level Measurement.", by Iain Cantlay

during the loading process that I can't explain at the moment

Perhaps some deadlock?

[sinbad]

I get 300-500 fps on my 9800 GX2 with only hiccups at page boundaries if I enable paging. Even my MacBook Pro with an 8600M can manage 110 fps. So, not sure why those framerates are so awful, totally different to what I see. I don't know how quick the Quadro FX 1400 is though, I'm not familiar with them. Make sure you wait until the "Building terrain..." prompt at the top disappears, otherwise it's calculating a lot of data in the background - the second run it uses data it calculated in the first run so skips this step.

Occlusion won't make much difference. All the terrain chunks beyond a configurable distance are rendered using a 'composite map' which is a pre-baked version of everything. So no normal mapping, no parallax, no specular etc. Therefore areas beyond the immediate vicinity are very cheap. When I fly up a little way I can get 700fps+ on my desktop because of this.

[theotherhiveking]

I get 300-500 fps on my 9800 GX2 with only hiccups at page boundaries if I enable paging. Even my MacBook Pro with an 8600M can manage 110 fps.

:/ At what resolution? With my 9500gt overclock (core clock 660, i think the normal one is 550) is between 70 and 90 when wandering arround normally, and 145~ when looking direcly at the floor.

With colour shadows i lose between 30 and 40 fps (settings aliasing to 4 reduces the loss to 15) even if there is nothing that projects a shadow on screen, depth shadows same but have jagged edges and are even slower, to the point i can not even reach 30 fps. on 1280x1024.

The 9500 isnt really more powerful that the 8600, thats why i ask.

[sinbad]

That was at 1024x768 4xFSAA, which is the standard resolution I use for testing - obviously the higher the resolution, the more you're going to hit the fillrate limit. But I can get 60fps at 1440x900 4xFSAA (that's the highest I can go on my MacBook).

The filtered depth shadows are a bit much for the 8600M with the parallax too, you'd really need to drop some filtering and/or the parallax. I intend to make the filtering level configurable when I get time.

[Tango]

So I'm about 80% done integrating the new terrain system into our engine. So far I absolutely love working with it. Great job Sinbad.

However, I'm changing over our terrain painting tools and I've run into an issue.

As far as I can tell there doesn't seem to be way to identify existing layers within a terrain. More specifically, I can't tell if a layer has already been added to a terrain.

What I have is a global sorted list of terain layer/splats loaded from an XML file. I'd like to be able to paint any layer onto a new terrain and automatically insert it (maintaining the global layer sort) on the fly.

1) I can't easily determine if the layer exists in the terrain without iterating all layers and all textures doing string compares on the texture names

2) The only way to add a layer is at the end of the list. I'd like to be able to insert into any position.

Would it be possible to change LayerInstance to have a 'uint8 layerId'? Then add something like these methods to Terrain...

Code: Select all

bool hasLayer( uint8 layerId ) const;
addLayer( uint8 layerId, Real worldSize, const StringVector* textureNames );
removeLayer( uint8 layerId );

Of course the interface is debatable, I'm just trying to get the concept across.

An alternative might be to just return mLayers non-const and allow inserts into the vec... However this seems messier I think?

Code: Select all

LayerInstanceList& getLayers();

Thoughts?

[sinbad]

Re the ID, I see your problem, although in the context of the terrain component an identifier like this doesn't actually have any meaning so I'm loath to include it, because it would require the interface for adding a layer to either include it, or to generate a unique one as a default. For something that's not used in the component itself, that feels wrong. In any case, any finding of a layer this way would still result in an iteration (since the layers are not going to be keyed on it).

I think the only way I might include this in a sensible, generic way is as a hash of the texture names, but that probably isn't of use to you if you have some external IDs.

About being able to insert a layer at any point in the stack, this is a reasonable request and I'll see about including it.

[Praetor]

Inserting a layer, yes, but can't you just store layers associated with terrains externally to the system? Doesn't seem like it would be too much bookkeeping to do the storage and association yourself.

[sinbad]

FWIW layer insertion at an arbitrary point is now supported in svn and will be in the 1.7 final.

[DanielSefton]

The terrain is dynamically lit if you disable lightmapping...?

[CrimsonGT]

Has anyone integrated physics with this yet? I implemented Bullet last night and plan on switching over to 1.7 tommorow. I followed this example when implementing it with Heightmaps, and im wondering if its basically the same.

http://www.ogre3d.org/forums/viewtopic.php?f=5&t=51330

[fatcoder2]

The terrain is dynamically lit if you disable lightmapping...?

Does this mean the terrain will cast shadows as well if lightmapping is disabled? Does it use PSSM or something like that? If that is correct then we should be able to have the terrain casting shadows onto objects placed on the terrain as well right?

[DanielSefton]

Does this mean the terrain will cast shadows as well if lightmapping is disabled? Does it use PSSM or something like that? If that is correct then we should be able to have the terrain casting shadows onto objects placed on the terrain as well right?

Yep, give it a go:

Code: Select all

TerrainMaterialGeneratorA::SM2Profile* matProfile = 
	static_cast<Ogre::TerrainMaterialGeneratorA::SM2Profile*>(
		Ogre::TerrainGlobalOptions::getDefaultMaterialGenerator()->getActiveProfile());
matProfile->setLightmapEnabled(false);

I'm not sure about your last question yet, I haven't tried it myself.

[sinbad]

About lighting, I mainly went with the CPU-calculated approach so that it could be done realistically in the background in small chunks while you're editing, and it works extremely well for that. For dynamic sun position, it's less useful.

I have plenty of ideas for alternative approaches. A GPU version would be faster for dynamic sun but slower for everything else. There are optimisations that can be done to the CPU version too, such as doing the update along the light direction and storing the intersection height at each point, as described here: http://gpwiki.org/index.php/Faster_Ray_ ... hadow_Maps . Also for a dynamic sun position it would be good to have the option of not baking the lighting into the composite map and using the global normal map. As ever, having options is the key.

Unfortunately, I'm short of time so I don't know if I'll get to these things in the near future. The terrain system does what I intended it to do (I'm particularly pleased with batching efficiency) and I think it's a very solid base, but there's always lots of enhancements that can be done. Over time, I'm sure I'll get to some of them but patches are always welcome.

[Praetor]

Did you ever get aroun to PSSM or any other depth shadow maps?

[sinbad]

Did you ever get aroun to PSSM or any other depth shadow maps?

Sure, they're both in there already. Check out the shadows drop-down in the demo.

[Praetor]

I probably just wasn't up to date. I'll check it again, thanks.

[yubgipenguin]

Man I'm using old legacy geforce fx 5500 and it's extremely slow(15fps highest 8fps average) with everything off. Is there any way to improve performance?

I'm not interested in anything else but paging since PLSM2 isn't option for me... (I'm using PythonOgre)

[Tango]

FWIW layer insertion at an arbitrary point is now supported in svn and will be in the 1.7 final.

Much appreciated Sinbad. I also found an acceptable workaround to determining which layers have been applied to the Terrain.

[sinbad]

Man I'm using old legacy geforce fx 5500 and it's extremely slow(15fps highest 8fps average) with everything off. Is there any way to improve performance?

I'm assuming you've disabled normal mapping, parallax mapping & specular mapping?

Other things you can do:

• Pull in the composite map distance so that less blending is needed TerrainGlobalOptions::setCompositeMapDistance
• Increase the pixel error allowed TerrainGlobalOptions::setMaxPixelError
• Use a lower resolution base terrain
• Reduce the size of the textures in use
• Reduce the number of layers
• At an extreme, write a new TerrainMaterialGenerator::Profile to generate simpler materials

Personally I'm not really designing the terrain for anything less than a 6x00 (which is still a 5-6 year old card let's not forget) but there's no reason that with sensible settings you couldn't get good performance with that hardware. But you certainly won't be able to use the kinds of features & settings I use & have primarily designed for with the defaults.

[rubasurex]

There seems to be a minor issue with the positioning of the terrain. I just tried creating a terrain at 512x512 in size. I want to center the terrain at the origin, so I call:

Code: Select all

mTerrain->setPosition(Vector3(256, 256));

Everything works fine so far. I then save the terrain to a data file. When I try to reload the terrain, something strange happens. From a data point of view, the terrain is centered at the origin. In other words my entities walking on the terrain behave as if it is there but they are in fact walking up in the air. The actual visuals of the terrain are rendered back in the default location (i.e. with the corner of the terrain at 0,0). I hope that makes sense.

So it means that when I load the terrain, I have to always call setPosition to ensure it is rendered correctly. It appears as though the underlying geometry is set in the correct location, but the scene node is not updated to ensure it is rendered in the correct location.