ROAM Planet rendering

[DavlexDesign]

G'day Sovak,

with regards to ....

Out of curiosity, what's the largest planet you have made with decent frame rates?

The radius of the planet you see in the pics is 127,560.0 meters ..... here's the call I make ...

Code: Select all

		if (newPlanet->createPlanet(mCamera,"Planet01",3,3,865365,127560.0,847.5,26,7.75,0.55,1,0)) 

I could go bigger, I did go to 6378000.0m, and the frame rate did not change (well, only slightly while approaching the planet, on the surface the frame rate is pretty constant)but for testing purposes the size above is quite big enough, around the size of the moon. I can go out to whatever a double can get to , but it's gonna be hard enough populating this one, let alone one the size of Jupiter.

I do use an offset centers system for the pages/chunks, so when you get close to the surface, the accuracy is better, stopping the vertex jitters, I also enable the ...

Code: Select all

mSceneMgr->setCameraRelativeRendering(true);

seems to help. I know I have to use a scaled Model Matrix when I bring the other planets into the picture ( helps the zbuffer out), but that will come.

regards

Alex

[Praetor]

The skirts were quite hard to see in the screenshots. Actually anything was hard to see. But i finally squinted them into existence. I'm glad you got it working. So far, your system seems to have an enormous amount of promise.

[DavlexDesign]

G'day Praetor,
Thanks for the encouragement, It is showing promise, I was getting a little disheartened a couple of days ago, but I'm quite happy with it now, Getting the skirts on this thing sort of tested my nut for a few hours, the problem was trying to keep the frame rate up, so I had to really tighten up the ROAM algorithms' re-indexing system to allow me to add vertices on the fly while keeping the actual terrain vertices indexes pointing to the right places in the vertex buffer, and while doing that, keep the vertex cache intact so as the program doesn't have to update too much to the video card per frame.

I've got to leave enough room for the rest of the work the program will have to do to create the rest of the game ( flora, fauna, avatars, spaceships and the like).

I ran the program through the Nvidia Performance hud, and noticed a few things I can improve on while constructing the mesh data, and the order that I throw the polygons to the drivers to reduce the overdraw of the terrain. Also Materials play a major factor, how many times a poly gets drawn to achieve a look, I noticed that my water material uses two single textures and one anim texture for the ripples, and with each page it was changing the graphic three times per page and then go to the next page, and does it again (SLOW), so I'm going to batch the pages into a single mesh per chunk (many pages), and that should only change the graphic three times per chunk (much quicker). Maybe I need to study up on shaders a little, I know I will have to shortly, but at the moment this program runs on old cards just as good as it does on the newer cards, my work machine just has an intel(R) extreme graphics 2 chip, (NO shader support at all)
and it runs OK on it as well, that way when I have some spare time, I can mess around with it at work also.

Here's how it looks on the old hardware ...

Planet_Old_HW.jpg

Frame rate is pretty crap, but it is a slow box.
The trees aren't rendered properly for some reason, I think it's because I'm not choosing the BestTechnique or something, I'll have to look into that later.

regards

Alex

[Sovaka]

Wow I love the sound of its scalability.

I have a game idea myself that requires planetary systems and seeing as it can run on old hardware is true programming feat.
Gives us room to up the texture, shader, lighting, shadowing quality.

I came across a company called Geometrics recently that are working on bringing true Radiosity to games. Their first release title will be the new Eve expansion.
Combine that in with the likes of FarCry 2's regrowing flora system... Oh my...

*drool*

[DavlexDesign]

G'day Guys,
Been a little while, but I've been quite busy lately with a number of other projects that I'm working on for other clients DB Apps (not so much fun) , but I have made a few advances with this thing.
It's starting to look allot nicer now, the terrain is updating allot faster now, I went through my fractal algorithms and optimised them to the point where I actually picked up around 50 fps on my own private machine, that way I could increase the resolution of the terrain mesh and get allot more detail in it, while maintaining a good frame rate, around 200 to 300 fps, it's surprizing how small things can make puite a significant difference.
The pic below shows the engine running on old hardware again, sorry !, but I'm at work, I'll upload a couple of pics from my other machine when I get home tonight, anyway, it allows easier placement of terrain detail, that way Paged data can be added to the terrain easier (more compatible with PagedGeometry).

Planet_New_Old_HW.jpg

I do have one problem though ....
when I compile against the latest OgreTrunk, it loads up and then comes up with a memory heap exception and takes me to the microsoft std malloc code page, and yet it functions perfectly on Ogre_1_6_1, I've gone through the porting info doc and fixed all I could, Got me a little stumped, any ideas ???

I would love to put the meshing code into the worker thread, I've already got it ready for that stuff, the update process for the terrain is broken into nice threadable routines that can comunicate with each other and all, will help allot with the updating I reccon.

regards

Alex

[xavier]

threadable routines that can comunicate with each other

Hmm...what's the communication strategy you have in mind?

[DavlexDesign]

G'day Xavier,
With regards to what ....

Hmm...what's the communication strategy you have in mind?

With regards to my implementation of the ROAM Algorithm, When updating the terrain mesh I have to ...

A: Merge the redundant triangles that are not in view, and reduce the triangles just outside the view Frustum while letting the page the triangle is in that it has been modified, and by how much( if it's only minimal, then the update will wait a little, until the page has been modified enough to worry about).

B: Then for each Chunk that is visible or just outside the view frustum, go through the pages of that chunk and increase the detail if needed (same as above, but opposite). If the chunk is behind the horizon then remove it from consideration for the next run of Page checking, if it has just come into view, bring it into the consideration queue for the next Page run.

C: When that's all done, go through the chunks and see if they need updating to the video card.
* if there is enough change that it will make a difference to the view,
> go through the pages of this chunk and if it needs updating then update the batch for this page.
* if this page is past the horizon, remove it from the page merge split queue (freeing it's triangles from RAM).
* if this page has just come into the horizon, then put it into the page merge split queue for the next run through the loop (rebuilt by the split routine).

D: After all of that, then the water comes into play, doing pretty much the same as above, but as a layer of the terrain page, that way all the culling is already done. we go though the chunks and pages that are relevant and check the water triangle vertices depth compared to the terrain vertices and cull the triangles that are bellow a certain depth, so we don't draw water under all of the terrain, just under the low coastal shoreline.

What I was thinking is that the routines for the chunks can monitor a state mutex so when the merge for a chunk is complete, the chunk can start splitting while the other chunks are being merged, and while the chunks are being split, as a page is completed, it can start re indexing for input to the vertex buffers if needed, while the others are being split. because as I re index the page I add in skirts on the fly to the vertex buffers so I don't have to store a whole lot of data in ram as well.

It all seems to work at the moment, but in a timed incremental linear fashion, with the use of the work queue in the new Ogre, I reckon I could get this working like a champion. I might be going up the wrong track, but I would love to give it a go.

regards

Alex

[xavier]

Rather than synchronize N tasks to a single mutex...how dependent is each chunk/page on any other chunk/page? Are there any places where the entire terrain has to wait for all of the pages/chunks to be processed before any more processing can be done?

The reason I ask is that on the surface, it seems like there is an incredible amount of data parallelism here, and it would be a shame to introduce an unnecessary serialization with a shared piece of memory (to which access must be serialized). There is probably some task parallelism available here too, but if you set up the processing in a pipeline then the task parallelism will fall out naturally. Unfortunately, I am not familiar enough with your particular flavor of the algorithm to be of much help in suggesting any particular way of going about that. I just wanted to mention my concerns about cross-task communication.

[DavlexDesign]

G'day again Xavier,

I see what you mean,
The only real point to which the the data is dependent on a completed task, is between the merge and the split, at the moment I merge the whole visible and partially visible terrain before attempting the splits if needed on a per chunk and per Page basis, what I was thinking was to have an independent state mutex per Page inside the parent chunk, that way the page only has to monitor it's relevant state mutex to tell the page what to do, and the chunking algorithm just has to change the state of the relevant pages mutex to fire it up, that way there shouldn't be a problem with serialization, and that way pages with a higher priority can be fired earlier than pages out in the distance that are not as important. I already do a sort of the pages in a chunk to bring the closest to the front of the queue to get processed sooner rather than later.

The other part that needs to be completed is the chunks pages need to finish splitting and re indexing before they can be thrown at the video card in the chunks batch. This is one part that needs to be done outside of the interrupt because it actually uses Ogre specific calls (I think that's right). I have got that part down to a fine art now, so that shouldn't be a problem. If I can get the rest of the code that builds the mesh inside the workqueue and only do the vertex and index buffers in the main process, I think this thing should work quite effectively.

I could just put the current updateTerrain loop in the workqueue and do the buffers when the mesh is ready like I'm doing now, even that will make this system run allot better because as Ogre does its stuff at the end of the ExampleFrameListener, the other cpu can be churning away at the terrain data, so by the time I get to the Update buffers routine again, the terrain data might even be ready for updating, it may take a couple of frames between updates, but it will help immensely I think.

regards

Alex

[DavlexDesign]

G'day guys,
Here is the app running on my home machine,

New_Hardware.jpg

The frame rate is a little more respectable

more updates to come.

regards

Alex

[freegamerblog]

This is a very cool project. I hope maybe an established game like Vega Strike can pick it up and run with it. (I know they are planning to migrate to Ogre3D at some point.)

[DavlexDesign]

G'day guys,
About that problem I'm having with the new OgreTrunk and migrating this thing over to it,

I've found that there is some kind of change to the Hardware buffer handling or something, because I'm getting an error that tells me the Index buffer is being corrupted for some reason, It's in the re indexing procedure that I determine the size of the new buffer for a page, and with that dynamically re size the buffer if needed (the code for the dynamic re sizing is based on the Code Snippets example).

I'm just wondering if anyone else has had this problem and could maybe give me a clue on a fix for this.

The code works perfectly on the Ogre 1_6_1 build, no memory leaks or anything like that.

I'll keep looking at it, but it would be nice to hear from some one about it.

regards

Alex

[DanielSefton]

I had the same problem where it crashed on hardware buffers (and other random crashes) - it might sound really obvious (I don't know if you did this), but I found out I had forgot to build the INSTALL project after BUILD_ALL. This created an sdk folder (in my case ogre/vcbuild/sdk), did something magic, and using the libs, dlls and includes generated in that folder seemed to fix it.

If not, then I honestly don't know, but that exact same thing with hardware buffers had me scratching my head for a couple of weeks and doing that fixed it for me

[DavlexDesign]

G'day DanielSefton,

Thank's for the quick reply, I'll try that when I get home tonight, I'm at work at the moment and I don't have the space on my work computer for the DX Development Kit stuff, so I can't compile the trunk here.

Thank's again, and I'll let you know how it goes this evening

regards

Alex

[xadhoom]

The issue Daniel addresses is based on the fact that you have to include(/compile against) the correct OgreConfig.h where all the memory settings are defined. The libs compiled when compiling Ogre follow the OgreConfig.h which can be found in the include directory of your directory created with cmake, while including the header files from just the svn folder will pick another OgreConfig.h, which might be different. Compiling your project with the wrong config will lead to inconsistent memory management between the Ogre lib and your project.

Guess who had the same problem

[chuck_starchaser]

This indeed looks like a very cool project. One that I might use in PU if it works.
I wouldn't be too concerned with the planet generation procedurals if I were you. There's many
of those around, and different projects might have different needs, in that regard. If anything,
the procedural algorithm should be a plug-in, IMO. A simple interface, perhaps, that gives the
longitude and latitude, and asks for a height, perhaps. Let mathematicians compete with each
other for who can come up with the best.
The most important aspect of this project, IMO, is the management of the LOD's and height
data, and matching of geometries at the boundaries between subdivision levels.
This is what's hard to get right, and what the world's been waiting for.

[DavlexDesign]

G'day chuck,
With regards to the Height generation data needing to be a plugin, That's what I'm doing already, all I give the current routine is a vector offset from the center of the planet (normalized or not) and the height routine gives you the height from the center of the planet, or you can ask for a Height offset and it will give you the height above or below the radius of the planet (depending on the algorithms result being above sea level or below), pretty simple actually. It can also integrate DEMS, height maps or raw data (floats) into the pages if need be (for special areas like towns or cities or roads or anything else you like), the engine ties the data together as if it was just another part of the height call and sends back the result to the mesh generator (it works quite well but still needs a little tweaking).

As for the paging and LOD management, this system is pretty different to the normal type of static data terrain pages, as each page could actually be in a couple of different states of LOD while out where the algorithm is generating the pages of the planet, the parts that are manually built (Height maps and the like) are handled like the normal stuff out there, loaded when needed etc, they still get tied into the same height calls and send the same results back to the mesh generator as if they where part of the original height algorithm that way it really simplifies the mesh generation, the beauty of this thing is that for a whole planet, you may only have one base on it, and that's the only part that needs stored data, the rest of the planet can be built by an algorithm, you can still land anywhere you like, it's just a matter of telling the planet class what geometry you would like on the planet as flora and at what height, slope etc you would like it, and the terrain is generated for you (for people that enjoy wondering aimlessly around looking for stuff).

With regards to the seams between different lods on pages, well the pages are sort of stitched, they are always the same lod at the seams, and reduce out to the desired lod for the bulk of the page (this also works in reverse as well), I build skirts on the fly, so the resolution of the skirt follows precisely with the resolution of the terrain page (the skirts are needed for when morphing is enabled on the terrain, I did get some gaps because one page got built before the neighbor, and the vertex got to its desired location before the neighbor did, the skirt fixed that).

I'm still working on the flora and fauna part, it's coming along slowly, I'm pretty busy with other commitments at the moment, but should get back on track in a week or so.

regards

Alex

[chuck_starchaser]

Ahhh, so you implemented *continuos* LOD? Very interesting!

I'm still working on the flora and fauna part, ...

This is the kind of thing, I would insist, you don't need to work on; unless you're doing that for
yourself, or for a particular application. If the LOD management is working well, then I'd suggest it's
time to release it to the world. I'd be interested in the package as it is now; NO flora or fauna; thank
you very much; --our planets would be pre-defined to agree with Wing Commander canon, anyways,
so the rough height of terrain would come from a texture, and so would the colors. Unless you're
talking about actual geometry for trees... In any case, we don't need THAT much detail. Just to be
able to fly from space down to a landing strip is sufficient.

Sorry, forgot to ask: is this open-source?
Asking because I can't find a link to the sources. If you need server space I got tons of space AND
bandwidth at wcjunction.com; dedicated server, and we have SVN installed; I could set up a repo
for your project in minutes.

[DavlexDesign]

G'day chuck,
If you are only after that low a detail, why don't you check out http://sponeil.net/, his work on planetary terrain rendering should be quite sufficient for what you want. This engine will become a major part of a bigger project I'm working on for my son. It will be open source, but it's not ready for the masses yet, I want to polish it up like sinbad does, and get it more integrated into Ogre, so everybody can use it like a snap in, no need to be a scientist to use it.

regards

Alex

[DavlexDesign]

G'day guys,
Got a bit board today at work, so I played around with getting a few more trees on the terrain, this is just a test for the placement code I'm working on, nothing fancy yet.

There is only around 880 trees in these shots, but they are randomly offset from the house, so if you move the house, the trees follow and spread out to around 1.5km in all directions, dodging the water and going to their selective height limits. There is also no batching or billboarding of the trees yet (just straight 3D geometry with no instancing either), that will come, and that should help the GPU a little.

screenshot_trees.jpg

and

screenshot_Trees2.jpg

The water also animates now, looks a little nicer than just a plane blue transparent color.

regards

Alex

[Sovaka]

Very nice work Davlex, coming along quite nicely.

Do you have plans to incorporate Xavyiy's plugins like SkyX and Hydrax?

[xadhoom]

Wow, the trees definitely increase the look of the planet. Have you take a look in PLSM and PG for efficent concepts?

xad

[DavlexDesign]

G'day xadhoom and Sovaka,

Thanks for the comments.
Gives you a better idea of the scale of the planet also I think,
(in the top picture the hut from those previous images is almost in the center of the image).
(in the next picture the hut from those previous images is in the top left quadrant of the image).

I've been looking at SkyX and Hydrax, they are nice but, I'm working on my own ones at the moment, I might go back to those two options depending on how frustrated I get trying to re invent the wheel, but working with a spherical terrain is a little different to the normal flat type that these two libs are designed for, for me to use them in their entirety I would have to modify them a fair bit, and I don't think the original designers would appreciate that too much.
As great as they are, I would love to use some of their concepts, if it's OK with them.

At the moment I might just persevere with my Fluid dynamic atmosphere (clouds) and Fluid Dynamic Oceans (for waves on Coastlines, and currents) as these have been designed around the spherical terrain concept from the start, and I am quite familiar with these pieces of code because I wrote these a number of years ago, not for Ogre though, so the porting is taking a little while. If I tried to port the other two libs to the spherical terrain system, it would probably take me longer to figure out what they are doing, and modify them to suit (still an option though).

With regards to PG and PLSM, those too are designed for flat terrain once again (flat meaning ... One direction for the up vector) I will use some of PG's stuff though, as I can see the most promise with that one, JohnJ knows the inner workings of Ogre allot better than I do, although I am getting better with time.

If you have any ideas or features you would like to see in this thing, please don't hesitate to ask, as I tend to get tunnel vision while working on a project, and getting external input is something I really thrive on.

regards

Alex

[Sovaka]

Cool

Your fluid cloud and ocean sounds intriguing. Would definitely like to see the ocean in action.

[steven]

If you have any ideas or features you would like to see in this thing,

• General Fog
• Localised fog would be great: for example fog in valley but not on high mountains or in swamps.

Reading Sinbad terrain progress it seemed that fog was a difficult one that needed to be tackled sooner than later.