Myrddin Landscape Plugin SDK

[oskEE]

Hi myrddinson,

just a small question, is there a way how to query the terrain manager, if all necessary geometry has been loaded already? I couldn't find any method for this on the parent SceneManager class. I need this because I had some problems with ray-scene queries (to find an intersection with terrain data), if a query is been performed early during the level initialization (practically when the camera is moved to a completely different position on the terrain, the geometry under the camera has to be loaded and if during this the query is performed, an assertion fails or the program crashes on an exception - or at least I suppose that's the reason). Also I need this because I need to tell the rest of the application that "everything is ready, let's fly"
On second thought, here's another one (but related) - can you tell the behaviour of MLP, when I perform a ray-scene query at a different place than the camera is located at (say, 10 pages away from it). Will it have to load the entire page just for this purpose (and then throw it away)? And what if this has to be performed a few tens of times per frame, on a different locations, is it feasible?
Thanks for the answers

Regards,
oskEE

[sarmadm]

Hi, great work myrddin. its probably the best terrain manager out there:D..can you perform dynamic deformation of the terrain like in etm?? if so can you please give me the starting code for that.Again amazing work and keep up the good work
EDIT*
Actually i found out how to deform the terrain:D. Can you please tell me which command line call will get me getHeightAt?? I can't seem to find it in MLP doc.

[oskEE]

Hi myrddinson,

since I have a very large terrain in my project (32 x 32 pages at 512 x 512 pixels per page) I've found a bug that most probably needs addressing - I think that the page caching policy does not work. When I start my game, it consumes about 500MB of physical memory; then when I fly over the terrain, with each new visited page the app loads about 60MB of memory, somewhere up to 1.6GB, when it crashes on an exception (bad_alloc, I think). I'm pretty sure the game itself is not leaking, because when I don't move out of the current page, the memory consumption does not grow.
Of course, I've been playing with LOD settings, I've changed PageCacheSize to various values between 20 and 0, but with no effect (more precisely, if I set it to zero, it really does not cache anything, so entering each new page forces to load it, but it doesn't deallocate the old ones). Playing with LivePageMargin and AsyncLoadPageMargin also didn't helped it.
I also think this may be related to the older bug about the plugin leaving some residual allocated memory after cleaning the scene after finishing a mission and starting a new one.

Any help is appreciated.
Best regards,
oskEE

[snoefler]

Hi folks,

I work since a few days with MLP. Big thanks to Myrddin for this great plugin!!!

I have a problem (I think only a understanding problem) with the splatting (alpha) masks.
For testing purposes I created a terrain with only one page (size=256)
I turned off the auto-splatting (AutoSlopeNormalYMax/Min = 0) and defined a set of 5 materials (one default texture, and 4 splatting textures).

Code: Select all

        <Material BaseTexture="Sample_Dirt.png"
                  DetailTexture="Sample_Dirt.png"
                  NormalMapping="true" />
        <Material BaseTexture="Sample_Stones.png"
                  DetailTexture="Sample_Stones.png"
                  SpecularColorMap="false"
                  ApplyEnhancedLightingToDetail="false"
                  UseDiffuseAsSpecularMask="true"
                  NormalMapping="true"
                  OffsetMapping="true"  />
        <Material BaseTexture="Sample_Gras.png"
                  DetailTexture="Sample_Gras.png"
                  SpecularColorMap="false"
                  ApplyEnhancedLightingToDetail="false"
                  UseDiffuseAsSpecularMask="true"
                  NormalMapping="true"
                  OffsetMapping="true"  />
        <Material BaseTexture="Sample_Rock.png"
                  DetailTexture="Sample_Rock.png"                     
                  SpecularColorMap="false"
                  ApplyEnhancedLightingToDetail="false"
                  UseDiffuseAsSpecularMask="true"
                  NormalMapping="true"
                  OffsetMapping="true"  />
        <Material BaseTexture="Sample_Cobbies.png"
                  DetailTexture="Sample_Cobbies.png"
                  SpecularColorMap="false"
                  ApplyEnhancedLightingToDetail="false"
                  UseDiffuseAsSpecularMask="true"
                  NormalMapping="true"
                  OffsetMapping="true" />

I stored my manually created alpha maps in the terrain directory named "basename_sml1-4.png". (size 512x512x8)
I have defined "PageMaterials" as follows:

Code: Select all

<PageMaterials>
        <Material x="0" z="0"
                  Layers="1 2 3 4 -1 -1 -1 -1"
                  NumLayerPerTile="5" />
</PageMaterials>

But now only third material (gras) is displayd all over my terrain. No alpha blending at all

Can anyone help me with this problem?

I think it would be a good idea to write more about Myrddin in OgreWiki... little MLP Howto´s for some use cases... the doku is complete but not always easy to understand... I will write something about the alphamaps / splatting if it would work

[snoefler]

Hi,

nobody can help me with MLP and user made splatting (alpha) masks?
A short "What ToDo" will be very helpfull... I think thats a topic many MLP users will have.

[oskEE]

Hi snoefler,
maybe I might provide an insight for you (I've seen your post earlier here but I didn't have time to reply).

At first, make sure you understand the difference between 'page' and 'tile' - a 'page' is a terrain element that has a 1:1 correspondence to your heightmap/splatting map/etc. data, while 'tile' is a part of page that corresponds to a single batch of geometry during the rendering. During the paging process, at least one page has to be loaded from HDD (i.e. one heightmap file, one splatting map file etc.), but the manager then breaks this page on tiles according to your settings. One tile has only a single LoD state at the time, but more tiles from the same page can have different LoDs, although their data come from the same heightmap file. To be concrete, I use PageSize="513" and TileSize="129" (and so one page in my terrain consists of 16 tiles).

Now, the splatting mask is, according to the documentation, a 32bpp 4-channel image file. Each channel represents a single layer, i.e. corresponds to a single material. Make sure you understand how splatting works (I recommend http://www.bundysoft.com/docs/doku.php? ... :splatting). For creating them, I use L3DT software. It's important for you to understand, how the channels are stored in the texture and then how to specify the mapping in the .xml file. For your case, if you have 5 materials, you need a single 4-channel splatting map. For the settings you have posted, I think they should be more like

Code: Select all

<PageMaterials>
        <Material x="0" z="0"
                  Layers="0 1 2 3 4 -1 -1 -1"
                  NumLayerPerTile="5" />
</PageMaterials>

If your PageSize==TileSize. But this is always a bit confusing and should be played with (meaning the order of materials in "Layers" attribute etc.). See the documentation, it's quite comprehensible (at least in this part). But this is not the main cause of your problem, I suppose - at first, ensure that you provide only one splatting map per heightmap and that the splatting map has the right format (that it's 32bpp 4-channel image) and name. For your case, it should be named exactly "basename_sm0_0_0.png" - deviation from the correct naming will cause your app to crash or its malfunction. The naming ("basename_sml1-4.png") and format ("size 512x512x8") you provided seems a bit suspitious (the name is incorrect and I don't understand your format specification - what does "x8" mean? eight channels?...). It's important that you understand the documentation if you want to do anything for youself - most importantly, become familiar with the "Terrain data files" (bottom of the documentation .html).

[snoefler]

Hi,

thanks for taking the time!

First lets clarify the 512x512x8 _smlX_ splatting mask:
- With x8 I meant a 8bit -> single Layer splatting/alpha mask. Btw. one aplpha mask per file (see below PF_L8)
- The _sml_ stands for that intermediate single layer splatting masks.

See the MLP docu:

Code: Select all

The splatting masks : splatting mask(s) are stored in 0 up to 2 PF_A8R8G8B8 image files per terrain page – see Terrain Splatting. As explained before in Terrain Splatting, if TSM_EDIT preprocessor define is set, plugin will also use intermediary alpha-blend layers masks which are PF_L8 encoded image files...
If editing is enabled, alpha-blend layer mask file name syntax is ‘basename_sml[1-7]_[N]x_[N]z.ext’: note that only the 7 upper alpha-blend layer masks need to be stored.

Perhaps its not usual to load alpha mask from this intermediary splatting masks... perhaps its only for saving not for loading alpha masks... i dont know. I will try it with the combined 4 channel alpha masks...

Important for me: Do I have to "load" the splatting masks manually, or should it work "out of the box" if the right file is present?

[oskEE]

See the MLP docu:

Code: Select all

The splatting masks : splatting mask(s) are stored in 0 up to 2 PF_A8R8G8B8 image files per terrain page – see Terrain Splatting. As explained before in Terrain Splatting, if TSM_EDIT preprocessor define is set, plugin will also use intermediary alpha-blend layers masks which are PF_L8 encoded image files...
If editing is enabled, alpha-blend layer mask file name syntax is ‘basename_sml[1-7]_[N]x_[N]z.ext’: note that only the 7 upper alpha-blend layer masks need to be stored.

Perhaps its not usual to load alpha mask from this intermediary splatting masks... perhaps its only for saving not for loading alpha masks... i dont know. I will try it with the combined 4 channel alpha masks...

Ok, you are right, but that's the thing, you don't need those intermediary layers if you don't need to edit the terrain (what you haven't said you do and I suppose that you don't, especially if you are trying to run the plugin for the first time). But even if you have MLP compiled with TSM_EDIT enabled then you don't have to provide those intermediary layers (for example, I have MLP compiled that way, because I use the "ToolGenerateMosaic()" utility for breaking the terrain onto pages - but even that way, I don't provide those intermediary layers to the plugin, and neither it does create them automatically - it would probably do that only if I used those editing functions, which I don't. I just normally provide those A8R8G8B8 images to MLP). Of course I'm not myrddinson so it's not a 100% sure thing what I'm telling you, but it works in my game

Important for me: Do I have to "load" the splatting masks manually, or should it work "out of the box" if the right file is present?

Practically the only thing to do in the code is to create an instance of MLP and then call setWorldGeometry() on it. In other words no, you don't have to load those masks manually, MLP does it automatically, based on the settings in the .xml file you specify in the setWorldGeometry() call. Just make sure you have them appropriately named, and that the path where those masks are stored is in your recource.cfg (or that you specify it in the code).

Cheers
oskEE

[EDIT]
Yea, and I've just noticed this - when you specify those splatting masks incorrectly, MLP will use just the third material to cover the entire terrain - just as you reported...

[snoefler]

Ok,

with a combined 4 channel alpha mask (basename_sm0.png) it works out of the box.
The intermediate 1 channel pngs dont work for me. The problem was (is) that my 3D terrain tool only generates 1 channel alpha masks, so I have to combine it manually to a PF_A8R8G8B8 image.

[Beauty]

The Myrddin landscape plugin is great!
So 19 months ago I created a wiki page for it.
It's very sad to see that really NOTHING was updated there, although there were several news in this and the 2 previous forum threads. Not even the minimum of updating the related forum link was done.
And this although I remind to it in February.

For newcomers and common users the wiki would be a good place to get current information in compact manner.
The long forum threads are not good for an overview.
But it seems so that nobody matters the wiki ...

[Beauty]

However: Myrddinsson haven't been around much - and I wonder if this project has dropped off of the edge of our planet?
He's the only person who can update that particular page.

That's wrong. Everybody could make updates. Myrddinsson writes changes and news here in the forum. So others could copy&past them to the wiki. So I did for many things without knowing detailed project backgrounds. I just added/updated the telled facts.


Months ago I added a section for the Myrddin Landscape plugin to the SceneManagersFAQ in the wiki. I also aked here in a Myrddin thread to fill in the open points. This wouldn't be much work or time to people with knowledge.
Also on this important newcomer page nearly nothing was been done by other people in the last 2 years.


Ogre is famous, it has many nice extensions and many people use it. But its wiki not even has an overview about the extensions. It's something like a great software, but complex and without manual. And a manual would be good for everybody, not only for newcomers.
Ok, this is not the right place to write about. I just wrote it down now. But if I write it somewhere else, I think there also would be done noting more in the wiki.
In the middle of this year I lost my wiki motivation for months. Today I lost it again. It's like a huge construction site without workers ...

[Ident]

Bad to hear about the WIki stuff,.. i personally also had a huge problem finding info about certain stuff, when i started using Ogre. Wiki is sometimes containing very old information, some information you don't get at all, not in the forums, nor the wiki, nor the manual, you've to gather them by wasting a lot of time on searches and trying stuff out and investigating the pure code or comments to the code, which is okay, but some things have to be documentated somewhere... I created 1 wiki page some time earlier, but thats quite all i did for the wiki so far, but also i got not much to contribute as i dont know much yet


so i ask a really noobish question now concerning this plugin. and i m sorry if they were answered somewhere, i didn't read thru the epic number of posts here, and couldn't find the answers by myself until now:
This plugin is built upon the terrain manager, i read, which uses heightmaps.

So however, in our project we dont use heightmaps, we have streets and a city, and that's not really something that can be done with heightmaps easily, and also it wouldnt make much sense imho

However, i liked the fog and the sky and the night with the moon in your demo a lot, and I'd like to use this plugin - so, the question i ask myself is, can i use this even without having a height map, just having static meshes ? and will this, if it's possible, make the performance drop due to the different scenemanager (atm. using the ogre default generic one) but not using the heightmaps, or will it be the same ?

Also- it says "parallax occlusion mapping" is included into this - what does it mean? that there is a shader you can use? Do we have to use it? is it also appliable for other objects in the scene, just like an additional shader available to use?

Also I'm using Ogre 1.7 unstable, will this plugin work with 1.7?

[DanielSefton]

can i use this even without having a height map, just having static meshes ?

Yep, just point it to an empty landsample folder and it won't load any terrain.

will this, if it's possible, make the performance drop due to the different scenemanager (atm. using the ogre default generic one) but not using the heightmaps, or will it be the same ?

There will be a significant performance drop. The realtime calculations are expensive. Actually, the atmosphere is okay, it's the tessellated sky that hits it the most. I get 70% FPS drop when I enable the sky (though I don't exactly have a fast card). In fact, I've personally added Caelum with this plugin; looks just as good, without the performance drop. But if you did that, you might as well not use MyRddin.

I hope Myrddinson will decouple the atmosphere from the terrain in v2.0 (along with depth scene rendering); if you want to use it without the terrain at this time, it's not worth it.

Also- it says "parallax occlusion mapping" is included into this - what does it mean? that there is a shader you can use? Do we have to use it? is it also appliable for other objects in the scene, just like an additional shader available to use?

There is a parallax shader that you can use which is compatible with the plugin, yes.

Also I'm using Ogre 1.7 unstable, will this plugin work with 1.7?

Yes, but you'll have to port it. I ported it and it didn't take me long.

[Beauty]

Also I'm using Ogre 1.7 unstable, will this plugin work with 1.7?

Yes, but you'll have to port it. I ported it and it didn't take me long.

I think it would be useful to publish your modified code. Then other people doesn't has to do the same work. It also would improve the common development of this project. (Maybe create an other tree in SVN for use with for Ogre 1.7?)

[DanielSefton]

I think it would be useful to publish your modified code. Then other people doesn't has to do the same work. It also would improve the common development of this project.

I'll post it if MyRddinson wants me to, or if anyone is desperate. I posted the diff for porting it to 1.6, but it wasn't much use. Best if it comes from the horses mouth IMO.

(Maybe create an other tree in SVN for use with for Ogre 1.7?)

MLP isn't on SVN unfortunately.

[Ident]

Oh, it's Daniel again

In fact, I've personally added Caelum with this plugin; looks just as good, without the performance drop. But if you did that, you might as well not use MyRddin.

Oh, well, then I guess I'll use Caelum. However when i looked at Caelum I didn't think it looked that great, but after what you just said I think I will recheck more exactly performance is more important anyways, can't live with much of a drop. Also it's a pity that this plugin also would have had a nice fog, and it's still in development and so on, while caelum is kind of abandoned

well thanks for your answer daniel

[Beauty]

I posted the diff for porting it to 1.6, but it wasn't much use.

Generally there are many people who search for ready solutions in forums and wiki.
When they don't find it, most people do it themself. They don't ask for a solutions, unless they have problems.
Even if less people use you published code - it's good for the public to have it.
Also most people read solutions without saying thanks. So how you want to know that only less people use it?
Even simple code snippets can help somebody to learn how to do something.

I'll post it if MyRddinson wants me to, or if anyone is desperate.

Do you think that people ask you for a solution when the people doesn't know that you have a solution?

Best if it comes from the horses mouth

Generally yes, but also the horse could be happy about improvements.
And other people can use it more early. Also maybe the horse doesn't talk about this.


update (addition)

A big problem for a project is, when the horse runs away.
The same problem happens with the Mogre wrapper. (using Ogre by .NET applications)
There was no documentation. Some people wanted to update and maintain the wrapper, but they didn't understand some internals and gave up after a while.
One year later the horse came back to do the job (update from Ogre 1.4 to 1.6).
Also he gave explanations to some important points.

An other point shows the power of the community.
An important SDK feature didn't exists. The horse didn't create it.
One user created a helper tool to add this feature. But he didn't published it and nobody did know about.
All 6 months I remind the Mogre people to this imporant and needed feature.
Not until then he posted his solution.
Now it's integrated to the SDK building process and everybody can use this enrichment. (Also some of them said thanks for integrating it.)


Just some thoughts about the horse ...

[jacmoe]

You can lead the horse to the water, but you can't force it to drink..

[DanielSefton]

What's with all the horse analogies

Okay, changes include:
- Compiles against latest Ogre trunk.
- Updated to the latest scene manager methods.
- Updated the 2 year old octree scene manager to the latest.
- All memory is now managed through Ogre. Hopefully this fixes *some* memory leaking.
- Some misc optimisations.

I call this MLP 0.94

[Vectrex]

"while caelum is kind of abandoned" that's like saying the Vector3 class is abandoned It's done what it set out to do, therefore doesn't need much maintaining.
I personally only want to use the clouds and maybe the skydome colouring parts of Myrddin. These libs really should contain the seperate parts in a reusable way. I'd love to use the 2d clouds/sky from Myrddin, the stars, time and weather of caluem and the 3d clouds from skyX. This is why Ogre is 'just a rendering engine' for the good reason that no one lib can cover everything.
Caluem does seperate it's bits decently so I'm not sure how difficult it would be to use Myddin clouds/sky.

[DanielSefton]

Yeah I agree. I think people will make much more use out of this plugin if it's various elements were decoupled. It's just unfortunate that this is released at the same time sinbad is working on the new terrain.

Even the water plane - everyone creating outdoor scenes want good looking, performant water, but that doesn't exist. There's only Hydrax and the ocean demo, nothing else.

Problem is, all of the components are currently fused tightly together. I think making it not a scene manager would be a start.

But don't forget, the shaders are commercial, they are not LGPL. You are required to pay $100 if you wish to modify them. I respect that, because he's put a huge amount of time and effort into this, and that's his choice. I'm not sure how this would translate if everything was decoupled.

It comes down to if he has the time, all of this would require a rewrite I guess. Lets hope there's a MLP v2.0, which he did mention.

[Beauty]

But don't forget, the shaders are commercial, they are not LGPL. [...] I'm not sure how this would translate if everything was decoupled.

I'm not shure with licences, but I think that people are allowed to modify the LGPL code and use it for other projects.
So theoretically somebody (or a team) could create a secondary project with the focus to one feature (e.g. cloud system). Using a SVN would support a public development.

Note - I don't want to steal money earnings of Myrddin. He should get what he deserves.
I just wanted to tell this alternative way for single features.


An other thing:
What's about a section in the Ogre add-ons forum?
It would be much more clear to have different threads.
Currently all information is modley mixed in one singe thread (ok, splitted to 3 threads, but the content is like a huge thread).

[skynet]

Yeap, i agree with Beauty, we need own thread in addons.
Hm, i got some problems when started compile Myrddin with Ogre 1.7
Before compiling i try to patch files with v.094 and half of files didnt want to be patched (i use v.093)
Its not so big problem. i change and recompile QuickGUI and Plugin_ZPK.
When i try recompile LandscapeSceneManager i got 105 errors
Than i started magic dance with tambourine around PC and Myrddin was compiled
Big thanks for Myrrdin update.
p.s^ sorry for my english

[nico008]

Congratulation!!!!
This is the best landscape plugin I have ever seen
I think that I will use it for the game I'm creating

Myrddin, do you speak French? I saw some free.fr link if I'm not mistaken

[nico008]

Does anyone here knows how to get the NoGUI version working? I was able to compile the DLL with NoGUI mode but the Game demo app does not run anymore (even if I compile the DLL without NoGUI mode)
Can someone upload a working pre compiled NoGUI version?
Actually I want to use the MIYAGI GUI (see latest svn : http://code.google.com/p/miyagi/ or forum thread : http://www.ogre3d.org/addonforums/viewt ... &start=255)

Thanks in advance!