New Terrain Early Shots

[sinbad]

I don't know why I didn't see that before, but it's fixed now.

[BTolputt]

Much appreciated, sinbad, rebuilding now

[Yacoby]

I have been playing around with the terrain component and from what I have seen it is great. The one thing I haven't seen any samples of (and presume it wouldn't work) is the ability to load vast terrains (although with the majority of it at a very low detail).

For example, I can load 16, 65 vertex wide terrain "blocks" fine, however much more than that and I start running out of resources. I need to load the equivalent (as they shouldn't be at full detail) of up to 4000+. Does it (or will it get to the point) where is supports this sort of vast landscape (although using pregenerated data ofc, e.g. pre-generated images rather than alpha blending)

Is there any chance of it ever being supported, is it in the pipeline or have I missed a major feature and it is already possible?

*EDIT*:
The simplistic solution would be to have multiple TerrainGroup instances which each use a landscape of varying detail. The issue is that the transitions would look really ugly.

*EDIT2*:
I can put some time into (trying to) create something but I doubt it would be very good and probably have a "hacked in" feel as I don't have much experience with working with other peoples code. There is no point me sinking time into something if Sinbad is working (or plans to work) on it.

[Sharma]

Not sure that this is the right place to discuss your problem (maybe the Help forum :p). But...vast terrains... did you tried paging ? ^^'

[sinbad]

Yes, there is more work for us to do on the new terrain to support larger overall terrains.

In fact the rendering already uses pre-generated composite maps for distant renders, it only does actual alpha splatting up to a certain distance. This saves a vast amount of fillrate.

But geometry wise, there are future plans to support staged loading of geometry data rather than always loading the full detail. The quadtree system that stores the terrain geometry already splits and distributes the vertex data to different parts of the quadtree at different resolutions, in order to support arbitrary sized terrains efficiently while still using only 16-bit indexes. This gives the opportunity to 'slice' the quadtree at a certain level (or levels) and only load the geometry for those lower LOD levels. This isn't implemented yet but is one of the TODO items.

We also want to make the paging more efficient, which works already but we're not happy with the speed yet, which is why we're not really advertising it despite the implementation being there already.

It sounds like you're partitioning your terrain too much though if you really meant that you're loading 65 vertex wide blocks. The new terrain really doesn't work very efficiently unless you use larger sizes for each terrain instance, because it splits the block down and then hierarchically batches it. If the blocks are already separate it can't do that. You can actually load 4,000+ vertex terrains into one instance provided you have the VRAM available, although in practice it would be better to use 1,000+ blocks.

[thesaint]

Hi,

first I have to say that this new terrain engine is really amazing!

Now, my problem is that the lightmap is computed correctly, but the terrain doesn't seem to respond to any dynamic lightning?!
Did I miss something or does the terrain only support one directional light which is also not to be changed?

Since I worked on my own terrain implementations in the past I am a little bit confused here. I thought the lightmap would only be used to compute the static shadow. This lightmap would normally be interpolated with the dynamic lightening computed by all important scene lightsources using the terrain normals (which seem to be available).

The thing is that the shadows in my case are the same as ambient lightening, resulting in "flat" terrain in the shadow areas, which looks really ugly. It would be nice if the lightmap would do something like a 50:50 interpolation between the existing shadow map and another lightmap without shadows. this way the "shadowed" areas would not be "flat" but instead shaded smoothly by using terrain normals.

After all it would be an option for me to generate the correct lightmap myself, but this would require additional work, so if it is already implemented, please tell my what I can do to activate this feature(s) .

regards
chris

[Praetor]

Basically you want to be able to change the shadow "color" so that it isn't necessarily full dark?

[thesaint]

Well I changed the calculateLightmap() method in the orge source code so now I get the shadow effect I wanted . I am still not sure wether this will provide me with dynamic lightening, but I will try it.

It seems as if I need to further modify the terrain source code to get all the performance and setting tweaks I would like to have . I am not sure wether or not they already exist but with the current API I couldn't figure out how to do it:

1) Skip the lightmap/normal map generation when creating a terrain with ImportData. Since I will probably store the lightmap with the heightmap, and calculate the normal map on the GPU, it would save a lot of time, if OGRE skips the normal map and lightmap calculation, wouldn't it? Of course I could use the builtin save/load methods but since the heightmap/lightmap pair is compressed really efficiently, I think using those methods would probably multiply disk usage compared to my compressed data, which already is around 100 MB.

2) Add properties to influence the resulting lightmap shadows.

Oh well that would be all for now...

regards
chris

[sinbad]

I think basically you're looking to replace ambient lighting with 'skylight' style lighting, which is not purely ambient or directional, it's hemispherical - a good idea. I actually have a big list of things I'd like to add support for, either in the standard material generator or alternate ones, but I just ran out of time for 1.7. I hope to come back to the terrain because there's loads more I can think of doing to it - or maybe we'll get some ideas for this year's Summer of Code.

On to your points:

1) you could do this with your own material generator (they're pluggable), just ask the terrain not to calculate its own lightmap and use your own approach. The idea is that the Terrain instance can calculate certain things, but it's dependent on what the material generator requests from it for rendering

2) Patches welcome

[thesaint]

Thanks for your reply ... I did some more testing with the terrain and also some source code debugging. But I still can't figure out how this strange bug can be avoided:

http://www.ogre3d.org/forums/viewtopic.php?f=2&t=56356

Maybe you could have a look at it?! It is really important for me to get the terrain working, since it is the basis for my project.

I already eliminated a wrong heightmap. The heightmap data is exactly what it should be. If I load the heightmap from an image, the results are even more weird.

regards
chris

[oskEE]

Hi!

I'd like to ask - everybody is writing about how they are already using the new terrain/paging system, posting pictures etc. but I actually couldn't find any information about what sort of data to feed into it, what are the naming conventions for terrain heightmap and custom data files, how the .page fileformat looks and so on. I'm aware that this is a RTFM question, but I've really went through all the "M"s I could think of - Ogre manual, API docs, relevant Wiki pages - and found no solid consistent info on that. The "how" is apparent from the SDK demo, that is quite clear, but "what" is not, since the paging is disabled by default and the media folder doesn't even contain a single .page file, and plain commenting-out of the "PAGING" flag and defining the MIN and MAX page limits doesn't have any apparent effect. I haven't been watching this thread from the beginning, so the info may as well be buried here, but reading through those 23 pages of posts is not a half-an-hour task I don't want anyone to waste their time explaining these things to me, I'd just be grateful if somebody could link me to the specification of those things, or even to the forum thread (post) that explains that.

Just for the context - currently, I'm using Myrddin's plugin, which isn't bad, but the huge memleaks are preventing me from staying with it. So I'm also naturally interested in how to use the data for MLP (which are .raw for heightmaps and 4-channel .png-s for splatting maps) in the sinbad's terrain system.

Big thanks
oskEE

[xadhoom]

As you might already know Ogitor uses the terrain paging system which you can test easily with editing, splatting etc. If you don´t want to use it you definitely can get a lot of answers from the forum and the code though.

xad

[vladdracula]

Hi,

I have managed to load .page files and the terrain appear just fine.
However, i have been getting loads of handled exceptions about FileNotFound
My page files are named as:
Area_00000000.page

the page file is generated from saveAllTerrains

Although it managed to load, i can't help but notice the amount of exception occurred when looking for the files.
Namely it was looking for some variety of naming convention which changes when i try to match up with the filename
1. World_Area_00000000.page
2. World_Area00000000.page

I have tried changing the file's name to match however, either it repeats the FileNotFound problem or the terrain didn't show up at all.
How can i prevent these exceptions from occurring? or rather what is the name convention for page files?
I could barely identify the convention as (WorldName)_(SectionName)_XXXXXXXXXX.page but sometimes it looks for (WorldName)_(SectionName)XXXXXXXXXX.page

I have coded it in this order:

Code: Select all

m_pTerrainGroup->setFilenamePrefix("Area");

m_pTerrainGroup->setFilenameExtension("page");

m_pWorld = m_pPageManager->createWorld("World");

m_pWorldPagedSection = m_pTerrainPaging->createWorldSection(m_pWorld, m_pTerrainGroup, 128, 513, 0, 0, 0, 0);

[sinbad]

You've misunderstood a bit. The terrain data file and the page data file are not the same thing. Terrain keeps its own dedicated data files which is just simpler for the purposes of supporting multiple pages directly through TerrainGroup rather than via the paging component. The '.page' files are general world section data files which can embed or refer to any other kind of content for that page, but are distinct from terrain data files. You've tried to make them the same thing which is why you're having a problem.

Unless you have some page data other than terrain, you won't have any .page file loading anyway. That's why our paging demo just implements a procedural page generation (and does nothing) - if we were loading other content such as entities etc, then we would need .page files.

I know this isn't well documented but I"ve explained why we're not pushing the paging just yet anyway, it needs more work.

[sinbad]

In 1.8 (default branch in Mercurial) I've implemented vertex compression, which has dropped the size down from 28 bytes per vertex to just 16 (in other words a 43% saving). I could probably shave another 2 bytes off that by encoding the delta information differently, but this is a bit more difficult and prone to precision issues so I haven't done it yet.

As an illustration, it means you'd save over 3MB of VRAM on a 513x513 page of terrain, and over 12MB on a 1025x1025 terrain. The only cost is a few extra instructions in the vertex shader, but on the 8600M that I'm currently testing on the frame rates were unchanged, I just got a lower VRAM footprint...

[stealth977]

Very nice news indeed

Any plans of merging the changes to v1.7?

[BTolputt]

That is some very good news, sinbad. I'd be looking at it now, but my tweet-feed has you saying "don't use" - is this still valid?

[sinbad]

Any plans of merging the changes to v1.7?

No - new feature, new version. It works in my tests (Mac OS X & Windows, nvidia so far) but the rule is bugfixes only for the stable branch to minimise the chance of breaking issues.

That is some very good news, sinbad. I'd be looking at it now, but my tweet-feed has you saying "don't use" - is this still valid?

That was a few days ago, it's working now & I posted updates saying that it worked since then

[BTolputt]

Apologies, might have missed them. Twitter is not something I read "religiously"

[Kukanani]

I'm sorry, but I really have a need for a material generator that works on lower graphics cards. I'm willing to take a shot at coding my own custom generator, but I'm not even sure where to start. I assume that I can take out all references to NormalMap, LightMap, and ParralaxMap if I don't need those features in my material, right?

[KungFooMasta]

Is there any way to get a pointer to a Terrain instance from a standard raycast?

In OgreTerrainQuadTreeNode.h I see the following code:

Code: Select all

		/** MovableObject implementation to provide the hook to the scene.
		@remarks
			In one sense, it would be most convenient to have a single MovableObject
			to represent the whole Terrain object, and then internally perform
			some quadtree frustum culling to narrow down which specific tiles are rendered.
			However, the one major flaw with that is that exposing the bounds to 
			the SceneManager at that level prevents it from doing anything smarter
			in terms of culling - for example a portal or occlusion culling SceneManager
			would have no opportunity to process the leaf nodes in those terms, and
			a simple frustum cull may give significantly poorer results. 
		@par
			Therefore, we in fact register a MovableObject at every node, and 
			use the LOD factor to determine which one is currently active. LODs
			must be mutually exclusive and to deal with precision errors, we really
			need to evaluate them all at once, rather than as part of the 
			_notifyCurrentCamera function. Therefore the root Terrain registers
			a SceneManager::Listener to precalculate which nodes will be displayed 
			when it comes to purely a LOD basis.
		*/
		class Movable : public MovableObject
		{
		protected:
			TerrainQuadTreeNode* mParent;
		public:
			Movable(TerrainQuadTreeNode* parent);
			~Movable();
			
	        // necessary overrides
			const String& getMovableType(void) const;
			const AxisAlignedBox& getBoundingBox(void) const;
			Real getBoundingRadius(void) const;
			void _updateRenderQueue(RenderQueue* queue);
			void visitRenderables(Renderable::Visitor* visitor,  bool debugRenderables = false);
			bool isVisible(void) const;
			uint32 getVisibilityFlags(void) const;
			uint32 getQueryFlags(void) const;

		};

When I perform my raycasts, which can hit any movable object, I am hitting objects of movable type "OgreTerrainNodeMovable":

Code: Select all

	const String& TerrainQuadTreeNode::Movable::getMovableType(void) const
	{
		static String stype("OgreTerrainNodeMovable");

		return stype;

	}

Although there is a TerrainQuadTreeNode::getTerrain API, mParent is protected, and I'm unable to access it. Is there another way I can get a pointer to the terrain, from the Movable instance? Why not make the TerainQuadTreeNode public, or provide an accessor method?

I'm using 1.7RC1, apologies if this has already been addressed. (I'll be happy if it has!)

[sinbad]

I'm sorry, but I really have a need for a material generator that works on lower graphics cards. I'm willing to take a shot at coding my own custom generator, but I'm not even sure where to start. I assume that I can take out all references to NormalMap, LightMap, and ParralaxMap if I don't need those features in my material, right?

You probably still want the lightmap, since that doesn't require any shaders (it's just a greyscale texture), and you'll still want the composite map. You just request the features you want the Terrain instance to generate in your Profile::requestOptions implementation:

Code: Select all

	//---------------------------------------------------------------------
	void TerrainMaterialGeneratorA::SM2Profile::requestOptions(Terrain* terrain)
	{
		terrain->_setMorphRequired(true);
		terrain->_setNormalMapRequired(true);
		terrain->_setLightMapRequired(mLightmapEnabled, true);
		terrain->_setCompositeMapRequired(mCompositeMapEnabled);
	}

So you could omit things you won't need the terrain to generate for you (like the normal map). Then when generating the material, you can reference just the textures that you want to include in your material. The Terrain instance provides you with the data, and the TerrainMaterialGenerator (and nested Profiles) build the material itself referencing data from the Terrain.

Supporting shader model 1 cards has been on our TODO (not so interested in fixed-function really since no morphing could be implemented but it's still possible) so if you wanted you could implement this as a new Profile inside the TerrainMaterialGeneratorA which kicks in when shader model 2 isn't available and submit it as a patch...

Is there any way to get a pointer to a Terrain instance from a standard raycast?

It's generally not a good idea to do it this way, because the Movable representation of the terrain will have a very large AABB which isn't very useful for ray casting, you would be better to use the dedicated ray intersection methods on Terrain and TerrainGroup. If you want to stop the terrain movables showing up in your general ray query, just call TerrainGlobalOptions::setQueryFlags(0) before creating any terrains so that all terrains pick it up.

[Praetor]

I have made a low-end material generator for my current project. I created the entire terrain in L3DT Pro with all the maps. Therefore I don't need the generated ones. My material literally is just a few overlaid texture units with some tweaked color ops for my taste. 0 shaders (until I need to add shadows).

[KungFooMasta]

In my engine I want to have an API to raycast through the scene and return a list of RayIntersection objects, so I need to be able to use the RaySceneQuery to get Terrain instances. I found a work around for this, by using a map of type <SceneNode*,Terrain*>. When I hit a movable object of type "OgreTerrainNodeMovable" I get the Parent SceneNode, and again its Parent SceneNode, and retrieve the Terrain instance using the map. Then I do another raycast to get the actual terrain collision, if it does collide with the ray. The hardest part of this API is trying to maintain a distance sorted list of intersections, especially since the Terrain Tile AABBs will usually appear first in the ray query results, even though Terrain is usually intersected last by Rays (object picking). (It wasn't too tricky, except I have a bool firstHit parameter, which kicks out on first intersection, which might appear to be the Terrain Tile, but actually be an Entity)

[Bostich]

Hi guys,

iam currently running into a small problem, mb someone is able to help me.

For a simple water plane iam currently develope, i need the depth of the scene within a texture.

On my depthRenderTarget viewport, the Terrain LOD looks diffrent, then in my main viewport. Do i have to inform the terrain about a new viewport?


thanks,

//bostich