Editable Terrain Manager [v2.2 released]

A place to show off your latest screenshots and for people to comment on them. Only start a new thread here if you have some nice images to show off!
Post Reply
User avatar
ahmedali
Gnome
Posts: 302
Joined: Fri Feb 20, 2004 8:52 pm
Location: Lahore, Pakistan

Post by ahmedali » Wed Jun 27, 2007 7:42 am

I tested with 50 raytests only, still it makes the whole game slow.

I checked rayIntersects source, it uses incremental height tests. Most of the time is wasted in While Loop. And the more rays the more long loops.

But if I direct ray down and very close to terrain, it still becomes slow with 50 ray tests only.
0 x

CABAListic
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 2903
Joined: Thu Jan 18, 2007 2:48 pm
Contact:

Post by CABAListic » Wed Jun 27, 2007 9:13 am

Yes, I took that more or less from TSM (though I did optimise some parts away), and it could certainly be improved, though not quite as easily.

Anyway, might I ask why you need that many ray queries?
0 x

BSer
Halfling
Posts: 93
Joined: Fri Dec 23, 2005 8:46 pm
Location: Niskayuna, NY
Contact:

Post by BSer » Thu Jul 12, 2007 4:29 pm

Is there color map support built in, for coloring regions of the terrain?

I would think it would be economical to merge the shadow map and a color map in a common texture, keeping them as separate data for loading and saving but blending the data for display purposes.

If its not in already it is something I'll try to add eventually unless I'm beaten to it.
0 x

CABAListic
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 2903
Joined: Thu Jan 18, 2007 2:48 pm
Contact:

Post by CABAListic » Thu Jul 12, 2007 5:02 pm

Texturing the terrain is entirely up to you. You define the material to use, you pass it in. If you want to use colour maps, go ahead, you just have to write a fitting material and possibly the necessary shaders :)
There is, however, no support for real-time colouring, if that's what you are after. Wouldn't be too hard to do, though, I guess. You can look at the splatting manager and create something similar for colour maps, if you like. If you do, send me the code and I'll include it with ETM :)
0 x

Trosan
Greenskin
Posts: 101
Joined: Sun Jun 10, 2007 4:57 pm

Post by Trosan » Wed Jul 18, 2007 10:30 pm

Short question, in the current demo, the ETTerrain.cfg is obviously not read, is there any way to specify the "TextureRepeat" values manually? (or rather: how can I do that? :))
Last edited by Trosan on Wed Jul 18, 2007 11:39 pm, edited 1 time in total.
0 x

CABAListic
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 2903
Joined: Thu Jan 18, 2007 2:48 pm
Contact:

Post by CABAListic » Wed Jul 18, 2007 11:11 pm

Yes, it's in the material :) It's a shader parameter.
0 x

Trosan
Greenskin
Posts: 101
Joined: Sun Jun 10, 2007 4:57 pm

Post by Trosan » Wed Jul 18, 2007 11:57 pm

Thank you! Now it's working as I want it to.

But... the "solution" doesn't quite satisfy me. I am dealing with different world sizes (for different map files). The format I am using is designed so that there is one texture reference per height map entry. Therfore, I'd set the TextureRepeat values to the terrain dimensions (if I could ;)).
Afaik I could pass the shader parameter in the program, but wouldn't this somehow draw the material script useless?
Unfortunately, that seems like the only solution to me atm, unless I (or someone else) come(s) up with a better idea. Help is always welcome! :wink:
0 x

CABAListic
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 2903
Joined: Thu Jan 18, 2007 2:48 pm
Contact:

Post by CABAListic » Thu Jul 19, 2007 3:41 am

I'm afraid that is what you'll have to do (set/modify it in-program). The texture repeat is a property of the terrain material, and there's no way to change that. TSM and older ETSM simply have one dynamic material where they manage that setting for you. ETM gives you the freedom to decide how your material looks like. You'll either have to create your material dynamically, too, or you need separate material scripts for each terrain :)
0 x

REiNDEeR
Greenskin
Posts: 149
Joined: Sat Apr 14, 2007 1:27 pm

Post by REiNDEeR » Thu Jul 19, 2007 8:43 am

Trosan wrote:Afaik I could pass the shader parameter in the program, but wouldn't this somehow draw the material script useless?
yes it would, but you can always edit the shader code so it doesnt splat useless images.

btw, I also started with a single texture generated by L3DT but the problem is your terrain size has to be relativly small(like max 2048x2048), else your texture will stretch out too much.

With a terrain scale you can make it much bigger and still have a nice coverage using several textures and a shader of course.

Cabalistic thx for sharing this, thanks to your shader I managed to make my own (remember you told me to take a look at it :)).
I have a modified it a little to support fog, so does releasing the changes here comply with the LGPL license?

Anyway here it is (i use GLSL & HLSL tho):
vertex shader

Code: Select all

varying vec2 texcoord;
varying vec4 position;

void main( void )
{
   //gl_Position = ftransform();
   
   gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;
   position = gl_Position;
   texcoord = gl_MultiTexCoord0.xy;
}
pixel shader

Code: Select all

uniform sampler2D splatter1;
uniform sampler2D texture1;
uniform sampler2D texture2;
uniform sampler2D texture3;
uniform sampler2D texture4;

uniform vec4  fogColor;
uniform float fogStart;
uniform float fogEnd;

uniform vec2 scale;

varying vec2 texcoord;
varying vec4 position;

void main( void )
{
    //apply splatting
    vec4 sp1 = texture2D(splatter1, texcoord);
    
    vec2 tc = texcoord * scale;
     
    gl_FragColor  = sp1.r *  texture2D(texture1, tc);
    gl_FragColor += sp1.g * (texture2D(texture2, tc) - gl_FragColor);
    gl_FragColor += sp1.b * (texture2D(texture3, tc) - gl_FragColor);
    gl_FragColor += sp1.a * (texture2D(texture4, tc) - gl_FragColor);
    
   
    //apply fog
    float fi = (fogEnd - position.z) / (fogEnd - fogStart);
    gl_FragColor = fogColor + max(0.0, fi) * (gl_FragColor - fogColor);
}
0 x

CABAListic
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 2903
Joined: Thu Jan 18, 2007 2:48 pm
Contact:

Post by CABAListic » Thu Jul 19, 2007 11:39 am

so does releasing the changes here comply with the LGPL license?
Eh, don't worry about that, the LGPL applies to the ETM library. The demo and demo material would make no sense under the terms of the LGPL. I'll have to sort that out in the next release and change them to something else, possibly public domain.
0 x

User avatar
Loockas
Kobold
Posts: 28
Joined: Wed Aug 02, 2006 11:45 am
Location: Poland

Post by Loockas » Thu Jul 19, 2007 5:00 pm

@CABAListic: Are you planning to support the shadow casting on terrain in next releases? Shadows seem not working so far..
0 x

CABAListic
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 2903
Joined: Thu Jan 18, 2007 2:48 pm
Contact:

Post by CABAListic » Thu Jul 19, 2007 7:12 pm

What do you mean? Shadow casting works fine for me, but you have to be careful with your terrain material since if you are using shaders, you need to specify the extra shadow_receiver shaders.
0 x

User avatar
Loockas
Kobold
Posts: 28
Joined: Wed Aug 02, 2006 11:45 am
Location: Poland

Post by Loockas » Thu Jul 19, 2007 7:20 pm

Well I'm using default materials and shaders from the example with all the original code. I've just enabled texture shadows and what I saw was a completely dark terrain with somewhat flickering textures.. Hmm could you tell me what exactly should I do?
0 x

CABAListic
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 2903
Joined: Thu Jan 18, 2007 2:48 pm
Contact:

Post by CABAListic » Thu Jul 19, 2007 10:36 pm

I'm afraid I don't have a working material example right now since for our own project we'll use some kind of custom shadow setup, so I didn't bother. The main issue is that the material needs a shadow_receiver vertex program which the demo material does not have. Maybe you can try your way around with a search in the forums. I'll try to change the demo material for the next release if I find the time.
0 x

Trosan
Greenskin
Posts: 101
Joined: Sun Jun 10, 2007 4:57 pm

Post by Trosan » Thu Jul 19, 2007 11:58 pm

CABAListic wrote:I'm afraid that is what you'll have to do (set/modify it in-program). The texture repeat is a property of the terrain material, and there's no way to change that. TSM and older ETSM simply have one dynamic material where they manage that setting for you. ETM gives you the freedom to decide how your material looks like. You'll either have to create your material dynamically, too, or you need separate material scripts for each terrain :)
Thank you REiNDEeR and CABAListic.

I was already up to writing a program which writes the material script when I came to the conclusion that I don't need the splatting stuff and so on anyway. All I need is one texture per tile - no splatting images, no detail textures, even shadow can wait.
Unfortunately, even the smallest map I load has 64 tiles in each direction. With a texture size of 128 this map would make up a 8192x8192 texture if "compiled" into one terrain texture which I'd then load into the material. Unfortunately, this is way more than today's graphic hardware can handle (I believe the maximum texture size even on newer cards is 4096x4096), so I can't go this way. (or does Ogre split the material up into several small pieces itself perhaps?)
What other options do I have, is there need for shaders to solve this problem (which would require me to learn more about them) or does someone know another workaround?
Currently I think of the map material as one big texture - perhaps that is even wrong and there are other ways to access it.

At the moment, I have roughly 380 different terrain textures (including transitions from one ground type to another - that's why there are so many) consuming 17 MB of memory (128x128, 24bit depth).
Now if I'd use the terrain texture "compilation" method as described above, on a 1024x1024 map, this would consume around... wait... 48 Gigs of memory!? (If I am not mis-using calc here). Too much to load into VRAM (and I might eventually like to use a texture size of 256x256 and even a 2048x2048 map)
Alright, this is obviously not the right solution. Also you can always see parts of the map only, no need to load all the data into memory therefore.


Hope you get my problem (in one word: "one-texture-per-tile"). Help's much appreciated!
0 x

User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US

Post by xavier » Fri Jul 20, 2007 12:03 am

Trosan wrote:(or does Ogre split the material up into several small pieces itself perhaps?)
No.
0 x
Do you need help? What have you tried?

Image

Angels can fly because they take themselves lightly.

User avatar
HexiDave
OGRE Expert User
OGRE Expert User
Posts: 1538
Joined: Sat Jan 14, 2006 8:00 pm

Post by HexiDave » Fri Jul 20, 2007 12:18 am

Trosan wrote:
CABAListic wrote:I'm afraid that is what you'll have to do (set/modify it in-program). The texture repeat is a property of the terrain material, and there's no way to change that. TSM and older ETSM simply have one dynamic material where they manage that setting for you. ETM gives you the freedom to decide how your material looks like. You'll either have to create your material dynamically, too, or you need separate material scripts for each terrain :)
Thank you REiNDEeR and CABAListic.

I was already up to writing a program which writes the material script when I came to the conclusion that I don't need the splatting stuff and so on anyway. All I need is one texture per tile - no splatting images, no detail textures, even shadow can wait.
Unfortunately, even the smallest map I load has 64 tiles in each direction. With a texture size of 128 this map would make up a 8192x8192 texture if "compiled" into one terrain texture which I'd then load into the material. Unfortunately, this is way more than today's graphic hardware can handle (I believe the maximum texture size even on newer cards is 4096x4096), so I can't go this way. (or does Ogre split the material up into several small pieces itself perhaps?)
What other options do I have, is there need for shaders to solve this problem (which would require me to learn more about them) or does someone know another workaround?
Currently I think of the map material as one big texture - perhaps that is even wrong and there are other ways to access it.

At the moment, I have roughly 380 different terrain textures (including transitions from one ground type to another - that's why there are so many) consuming 17 MB of memory (128x128, 24bit depth).
Now if I'd use the terrain texture "compilation" method as described above, on a 1024x1024 map, this would consume around... wait... 48 Gigs of memory!? (If I am not mis-using calc here). Too much to load into VRAM (and I might eventually like to use a texture size of 256x256 and even a 2048x2048 map)
Alright, this is obviously not the right solution. Also you can always see parts of the map only, no need to load all the data into memory therefore.


Hope you get my problem (in one word: "one-texture-per-tile"). Help's much appreciated!
I skimmed over this a bit, so excuse me if I'm talking about something different:

What you want is texture splatting. You'd use an alpha map (or several, depending on how many you want to try to fit in per pass) with a channel for each texture you want to place - things like rocks, grass, sand, snow. In your shader you'd use either a normal map texture or vertex normals to do the basic lighting calculations, and do your splat-map calculations to come up with a final color. Generally in the vertex shader stage you'd multiply your input texture coordinates (the ones that cover the whole map go from 0.0 to 1.0) by a scale factor and use them to sample your detail textures. This gives you high resolution texturing without enormous memory usage.

Now, the nice thing here is that, for variation, you could multiply your final color by the base texture's color (the one you stretch over the whole terrain) - this helps the transition colors as well, so you only need a handful of textures. The next thing to do would be to build passes and enable/disable them (I can't remember if you have to add/remove them for that) if that section of terrain is entirely one type of land (i.e. your northern/southern regions might be mostly ice-type textures, so you'd pre-process the map and whenever the tiles with those come up, you'd enable only the ice-type pass while leaving transitions to have ice and whatever the neighbor would be). This would most likely be done in a pre-process stage, so it'd be very fast (I haven't used ETM, but I'm assuming their are Listener functions to tell you when tiles are built).

It's a bit of work, but that's what happens in graphics programming - you have to cheat to fit everything in and sometimes you have to cheat REALLY hard.
0 x

Trosan
Greenskin
Posts: 101
Joined: Sun Jun 10, 2007 4:57 pm

Post by Trosan » Fri Jul 20, 2007 1:16 am

Thank you HexiDave for you decent answer. It was a very interesting read and a nice summary of what I had gathered about splatting already (even some nice new information).

ETM already offers splatting ability, so I think I could use this for what you just explained.

However, it's not quite what I am looking for, I think (hope) I can make myself a bit clearer:

The point why I still don't use this technique is that I am using a given format which I need to be able to modify and write back (without altering the format). While I could read the format, convert it into a splatting map, and load it, I am still afraid that things might look different (than they should) and that "painting" the terrain and then writing back (requiring a converting back) to the format is much more difficult. I am aiming at a fast (but not so finalized/optimized) solution at the moment.

Now, if it turns out that any other solution is not much easier anyway, I need to get into the splatting stuff and do it that way.

To sum up about the format I am using (the important part):
- consists of an array of floats (one for each tile) with height map data
- consists of an array of integers (one for each tile) as reference to an entry in the file the bitmaps are stored. For example at position [3] there is int "53", that means that the tile at position (0,3) has texture number 53 (which is simply a 128x128 map texture showing either grass, or a grass->rock transition, or a different type of grass - all those transitions and so have been "pre-rendered" long time ago and are saved in the file)

Now I'd need to go through the map file and see: "Aha! Tile 0,0 uses texture 33. Tile 0,1 uses texture 31. Tile 0,2 uses texture 20, etc... and ideally, each tile should have the respective texture on it.

I am sure that with the right words one could explain this in a much easier way. Right now I am a newb in this section, tho..
0 x

REiNDEeR
Greenskin
Posts: 149
Joined: Sat Apr 14, 2007 1:27 pm

Post by REiNDEeR » Fri Jul 20, 2007 1:45 am

Hmm I think you want something like a paging solution where you can set 1 texture per tile. I'm not sure but isnt that how PLSM works? I know AdVantage works like this, you start with an enourmous texture and it has a program which can split it up into pieces.

For Ogre there are 3 options which I know of:

- PLSM, no idea about the status of it but its a bitch getting it to work. I managed to compile it for eihort but I had no idea how the splitter worked nor could I find any info about it.

- AdVantage is not an Ogre library altho someone(falagard) is trying to get it to work in Ogre, not sure if hes getting somewhere. DirectX only.

- SPT, looks promising but I have no idea if it can do what you want.

I for example use TSM and its pretty decent. With a 32kx32k terrain it takes me about 10mins to walk to the other end(with the SampleApplication default speed) and u can make it bigger if you wish. Of course id like to switch to a paging solution but Im gonna wait till something reliable comes along.
0 x

User avatar
HexiDave
OGRE Expert User
OGRE Expert User
Posts: 1538
Joined: Sat Jan 14, 2006 8:00 pm

Post by HexiDave » Fri Jul 20, 2007 1:54 am

Trosan: Are you talking about map painting like the old isometric games used where a specific tile type (i.e. concrete, red, rotation 45 degrees) would be used at a specific point and stop at the boundaries? If so, you're going to need to cut down how many textures you want to use and let pixel shaders do the transitions for you OR make a huge pre-baked texture (as if all the stuff was rendered) and let either AdVantage Terrain (Falagard's port is over in Developers section) or my new SPT (currently unstable, but I give it a week before it's solid) carve up the texture/heightmap data from the large maps to use.

There are ways to do what you want to do with tons of different textures, but it's going to be obscenely slow and an absolute memory hog. Modern games have a set of textures for each land type and use pixel shaders and splatting maps do all the transitions and colorations.
0 x

Trosan
Greenskin
Posts: 101
Joined: Sun Jun 10, 2007 4:57 pm

Post by Trosan » Fri Jul 20, 2007 2:26 pm

Thank you, that was very helpful.
Modern games have a set of textures for each land type and use pixel shaders and splatting maps do all the transitions and colorations.
I guessed so.. Well, I will write a converter then which creates splatting maps from the map file data.
0 x

Trosan
Greenskin
Posts: 101
Joined: Sun Jun 10, 2007 4:57 pm

Post by Trosan » Sat Jul 21, 2007 1:15 am

Sorry, it's me again.

Now, I've tried the suggested solution. It worked much better than I had expected. I think I can go with it.

Only problem is: FPS!

The original ETM demo comes with a material script including the shaders and 6 textures. Using these 6 textures, I got about 80 FPS on a 64x64 map I am currently working with.
Now, 6 textures weren't enough for me. Therefore, I doubled the texture size.
I did this as follows:
- in the ETTerain.material, I added a second pass with exactly the same content as the first one (which had 6 texture references already), but changing the file names for the textures and the coverage map names and using "scene_blend add"
- in the code I changed "SplattingMgr->setTextures(6)" to "SplattingMgr->setTextures(12)"

Only problem is: The framerate dropped to around 20 FPS (from about 80 FPS) and I was even planning on adding 10 more textures...

Is there anything I did wrong or that could be more optimized?
0 x

User avatar
HexiDave
OGRE Expert User
OGRE Expert User
Posts: 1538
Joined: Sat Jan 14, 2006 8:00 pm

Post by HexiDave » Sat Jul 21, 2007 1:51 am

Well, I don't know what video card you have, nor do I know how ETM's shader's look like (unless it's using FFP), but when you use more than a few textures, it starts needing more passes and that means re-drawing the terrain for each pass. So, lets say support for 4 textures per pass - if you have 12 textures, thats 3 passes plus coverage, so you render the whole terrain 4 times. It's not a limitation of ETM, it's a limitation of how graphics cards work - this is why I said to manage how your passes were setup based on land type. If you only have 2-4 splat textures for a specific land mass, ONLY USE THAT 1 PASS! You should have 1 (maybe 2) pass per land mass type, with the exception of when you have transition land types (even then if you're smart you can keep it to one pass).

So, if you need 12 textures per land type you have to target newer graphics cards because drawing all the terrain 4 times is going to eat most cards. Your next best bet is to atlas the textures - say you have 16 256x256 textures, you could put them 4x4 into a 1024x1024 texture and sample it multiple times per pass (based on texture locations) in a shader. Of course, you need to know how to program shaders, but if you're getting into graphics programming you might as well get used to that idea :D
0 x

warmi
Gnoll
Posts: 674
Joined: Sun May 27, 2007 3:56 am
Location: USA

Post by warmi » Sat Jul 21, 2007 5:12 am

you could put them 4x4 into a 1024x1024 texture and sample it multiple times per pass
Yep, that's the way to go if you want to get the best performance.

Although that introduces a slight complication - the textures within the atlas will need to be specially prepared to avoid typical problems associated with texture atlases :-)

PS.

Here is an example of "the problem".

A small (just 4 textures atlas texture)
http://www.warmi.net/tmp/test_tex.jpg

And the resulting quick render using only first texture on each tile ( with some splatting introduced just to show it is not fake)

http://www.warmi.net/tmp/test.jpg

There are ways to solve this problem, that but it is still pain in the ass.
Last edited by warmi on Sat Jul 21, 2007 5:39 am, edited 1 time in total.
0 x

User avatar
HexiDave
OGRE Expert User
OGRE Expert User
Posts: 1538
Joined: Sat Jan 14, 2006 8:00 pm

Post by HexiDave » Sat Jul 21, 2007 5:23 am

warmi wrote:
you could put them 4x4 into a 1024x1024 texture and sample it multiple times per pass
Yep, that's the way to go if you want to get the best performance.

Although that introduces a slight complication - the textures within the atlas will need to be specially prepared to avoid typical problems associated with texture atlases :-)
Ya, but everyone learns eventually that there are very few win-win situations with graphics programming. It's all about who can cheat the best and get away with it :P
0 x

Post Reply