ROAM Planet rendering

[DavlexDesign]

G'day jtarbox,
I'm no veteran Ogre Coder by any means, I've only been working with Ogre for around 3-6 months now, so you can see, If you
put some time in, Ogre has allot to offer the startup coder. It took me a little while to get my head around Ogre, but now I'm realy
liking it, I'm still finding things in the API that I didn't know was in there, and those things make some of my old code redundant,
but I don't mind, I sort of like it, seeing how someone else implements a similar process, sometimes it's better, and sometimes not.

Anyway, allot of my inspiration for this addon came from Sean O'neil's work also, I liked his old ROAMing spherical mapping system and
based some of my system on his concepts, obviously his engine was a concept only approach, so it didn't have all the niceties,
but his implementation was sound. (Thanks for the concept Sean).

As for the scaling down of movement speed, all I do is take the distance to surface and ... well here is the call

Code: Select all

this->mscaleFactor = Clamp(0.6f,1000.0f,newPlanet->GetAltitude() * 0.01f);

of course this figure is multiplied by the time slice factor passed into the framelistener by the SceneManager.

I know it's not pretty but for the demos' purposes it works, this gives me a speed of around 100km per second in outer space
and when in the atmosphere it starts scaling down the speed to about 60 meters per second on the ground. I can of course make this
more to scale, but when I'm checking how the new Paging systems' seams are working, and the Hybrid DEM terrain system is shaping up,
could you imagine traversing a large part of the surface at real walking pace (not much fun), even at 60 mps it get's tedious.

As for the objects (buildings etc) the paging system takes care of that for me, each page has it's own mini Scene Manager as such,
all you do is plug in terrain layers, terrain Physics mesh, static objects, movable objects or pretty much anything you like, and those
objects have their own lod / distance to camera handling routines. What I'm trying to do is make this system as generic as possible,
so that users like yourself can still use routines and methods that you are use too, Last thing I want to do is make this thing change
the way you think about coding, I want this thing to be used by you guys, so it has to just plug in and go.

hope this helps,

Alex

[jtarbox]

I'm no veteran Ogre Coder by any means, I've only been working with Ogre for around 3-6 months now, so you can see, If you
put some time in, Ogre has allot to offer the startup coder. It took me a little while to get my head around Ogre, but now I'm realy
liking it, I'm still finding things in the API that I didn't know was in there, and those things make some of my old code redundant,
but I don't mind, I sort of like it, seeing how someone else implements a similar process, sometimes it's better, and sometimes not.

Anyway, allot of my inspiration for this addon came from Sean O'neil's work also, I liked his old ROAMing spherical mapping system and
based some of my system on his concepts, obviously his engine was a concept only approach, so it didn't have all the niceties,
but his implementation was sound. (Thanks for the concept Sean).

I've toyed with Ogre in the past but hadn't gotten much farther then getting used to the basics. I even saw some posts from me on this forum about planet rendering in 2005.. It was back then that I delved into Sean's work. I had a semi functional mesh factory made up for CrystalSpace, but I never released completed code for it for various reasons (bugs, etc.)

As for the scaling down of movement speed, all I do is take the distance to surface and ... well here is the call

Code: Select all

this->mscaleFactor = Clamp(0.6f,1000.0f,newPlanet->GetAltitude() * 0.01f);

of course this figure is multiplied by the time slice factor passed into the framelistener by the SceneManager.

I know it's not pretty but for the demos' purposes it works, this gives me a speed of around 100km per second in outer space
and when in the atmosphere it starts scaling down the speed to about 60 meters per second on the ground. I can of course make this
more to scale, but when I'm checking how the new Paging systems' seams are working, and the Hybrid DEM terrain system is shaping up,
could you imagine traversing a large part of the surface at real walking pace (not much fun), even at 60 mps it get's tedious.

Hmm.. What is PlanetObject::GetAltitude() specifically returning there, the distance from the surface to the camera? I haven't looked at more then the samples, but if it's returning that, shall I presume it's getting the camera from the sceneManager/view it's attached to? It probably is, silly question. So, in a larger scale where there's multiple planetary objects in a scene, one essentially has to calculate the mfactor for each planetary object and use the lower one for that camera/view's movement. Given that, if mfactor is below 1000.0, one could presume that the view is entering planetary object's atmosphere and thus trigger all sorts of stuff (crashing, landing, screen shaking commonly attached to entering an atmosphere, etc.) Many thoughts are bouncing in my head now, thank you!

As for the objects (buildings etc) the paging system takes care of that for me, each page has it's own mini Scene Manager as such,
all you do is plug in terrain layers, terrain Physics mesh, static objects, movable objects or pretty much anything you like, and those
objects have their own lod / distance to camera handling routines. What I'm trying to do is make this system as generic as possible,
so that users like yourself can still use routines and methods that you are use too, Last thing I want to do is make this thing change
the way you think about coding, I want this thing to be used by you guys, so it has to just plug in and go.

I was hoping it would be something similar to this as that should make it simple to work with. I really cannot wait to work with this. Is there anything I could do to assist you? Testing, coding, bouncing thoughts, whatever. I may be new to Ogre, but I know my way around C++.

Ah well, enough rambling. Talk to you later..
-jtarbox

[JohnJ]

Nice work so far on the renderer

I have a question about your implementation. I've always stayed away from ROAM because I assumed you had to upload the entire scene to the video card every frame as the LODs change so dynamically. I'm wondering how much data you have to upload on average, and what kind of grid resolution you're using.

[DavlexDesign]

G'day JohnJ,

With regards to using the ROAM algorithm, my implementation is not the traditional ROAM algorithm of updating the whole map
and then pumping the whole dataset up to the video card (it was, but not now), I do have a global vertex buffer in CPU RAM which
ranges from 5000 to around 45000 vertices, but that's for the whole planet, that is broken down into 6 faces of a cube
(Each Face = around 12800 km square) and each face of the cube is broken up into 1024 chunks (32x32 grid), now each of those chunks
are around 400 km square, and those are further broken up into as many pages as needed for the res of the map. Currently the resolution
of the mesh goes down to around 0.5 of a meter when the camera is 1.85 meters from the ground.

This current implementation actually only ever pumps around 1 to 5 (sometimes more) chunks / pages into a batch (depending on the height
the camera is from the planet) remembering that as you move away from a page, its vertex count reduces expotentially, so the data pumped
up to the video card is quite minimal (around 200 - 5000 vertices per batch) and as I found out recently, I have to keep the batch count down
to a minimum. (apparently you should strive for no more than 1000 batches IN TOTAL, that's for terrain, water,trees, buildings, the lot !!!).
So I drop the chunks that are beyond the horizon, and if the chunk is partially visible, I drop the pages that are not, I keep a lower resolution
version of the chunks and or pages that are just outside the view frustum in CPU RAM (drop them from the video card RAM), and lower res
again for the chunks behind, so when you turn the camera, the chunks don't have to be created from scratch, and the ones that go out of view
get reduced and dropped, that way the total memory usage is kept to a minimum. I can't give you exact figures, but when I run the program
Windows Task Manager reckons that the RAM reduces by 115640k in total and fluctuates by around 1000-3000k, I think that's OK.

I should put some useful info on screen while it's running I suppose, but I'll get to that later I think, at the moment it is not a priority for me.

regards

Alex

[JohnJ]

Hmm, that sounds kind of like a geo-mipmap technique combined with ROAM. I didn't even know you could implement ROAM this way. Very interesting

(apparently you should strive for no more than 1000 batches IN TOTAL, that's for terrain, water,trees, buildings, the lot !!!)

Yeah, with how fast video cards are getting compared to CPUs, it's becoming important to move more and more processing to the GPU rather than the CPU. This is why the old ROAM implementations don't perform nearly as well with high polygon counts as more static batch-oriented approaches.

[DavlexDesign]

G'day JohnJ,

Just thought I'd show you how the data is in RAM when close to the planet,

paging2.jpg

The red dot and arrow on the left shows the position and direction I was standing on the planet when I stopped
the map from updating, then pulled back so you can see the structure of the map, as you can see I have got the
chunks culling OK, but the page culling is not working yet, that will come in the next couple of days I hope.

The picture on the right shows the actual stucture of the mesh, it's pretty opimised, maybe not as good as I like
but it's still a work in progress, at least with this setup; when you spin on the spot, you dont get holes in the map
as the ROAM tries to catch up with the view. I'm going to shift some of the code into its own thread soon, but only
when I'm satisfied with the results, so I don't have to try and debug a thread outside of the main thread.

Anyways, it's comming along nicely at the moment, isn't far from being a usable demo for you guys to try out.
It's running at around 120 - 230 fps on average when on the ground, not as good as I'd like, but I'm yet to do the
propper batching, that's what I'm working on right now.

regards

Alex

[SpannerMan]

Your work looks seriously fantastic so far, DavlexDesign - best of luck with the page culling, and looking forward to more screenies and/or vids!

[DavlexDesign]

G'day guys,
I've done some tests on this thing with regards to JohnJ's question ...

I'm wondering how much data you have to upload on average, and what kind of grid resolution you're using.

The whole planet class including all Lists, pages, chunks ... the lot takes up around 27 mb, this fluctuates by 300k to 1.3 mb while running on the planets surface, When in outer space the memory footprint is around 18 - 19 mb, fluctuating by 100k to 300k (a little less work for the map).

The fluctuations are what I assume are the data being thrown at the card, because the figures coincide with the buffer counters I have, that show the number of buffers that need updating on the video card while the program is running.

As for the resolution, I worked out that the resolution goes from 32 x 32 vertices's per cube face in outer space,
and goes down to 255,120 x 255,120 vertices's per cube face when on the ground at this test planets radius of 63780 m.

I'm currently doing a rewrite of the main Chunking / Paging system, because as I've been going through the code, I found allot of patching code in there that really isn't needed, and allot of ineficiencies in there that really need tying into the main planet class nicer. such as tying in the layering system so it'll be easier to customize the terrain system with what ever you like, and allowing me to render a complete Solar system without too many changes later on. I'm starting to look at the bigger picture with this thing, so that you guys can have some fun with it later.

regards

Alex

[xadhoom]

I'm starting to look at the bigger picture with this thing, so that you guys can have some fun with it later.

Hi Alex!

Does it include the upcoming Ogre paging system? Or is your own implementation to far off?

xad

[JohnJ]

Interesting screenshots. I think I uderstand now how you've integrated the ROAM.

Anyways, it's comming along nicely at the moment, isn't far from being a usable demo for you guys to try out.
It's running at around 120 - 230 fps on average when on the ground, not as good as I'd like, but I'm yet to do the
propper batching, that's what I'm working on right now.

I hate to say this, but there is a possibility that this is due to ROAM, and while batching might help, if the ROAM algorithm is creating a data bandwidth bottleneck there may be nothing you can do about it. Not that there's anything wrong with what you've got, it looks like it's good enough for most uses.

The reason ROAM is problematic on modern cards is modern video cards as so fast they can render triangles many times faster than they can receive them dynamically. 5-10 years ago, when things were reversed (you could upload triangles faster than the video card could draw them), ROAM was optimal because it reduces the number of triangles very efficiently. But today, you can achieve performance 2-5x faster at least, even with a terrain containing 2x the triangles, if you can avoid dynamically upload the terrain every frame.

But you have a nice ROAM system and it runs far over 60FPS, so I wouldn't worry about it too much

[DavlexDesign]

G'day JohnJ,

I know that dynamically uploading triangles to the video card is problematic, but if you look at the video http://www.youtube.com/watch?v=I3_pb6Gq5hs you will see that the frame rate was not an issue when I had only 6 pages to upload, with my new system, I will be uploading less data than what the old program was, my implementation of the "ROAM" algorithm is a little more radical, allowing the map to be broken up into so many smaller chunks, and then batching only what is visible to the camera, and only changing the batch at strategic angles, sort of like a Japanese dancers fan that can fold up or out when needed, that way the data changes are kept to a minimum, reducing the upload of new data to the card, I also implement a cache of sorts, so the reproduction of data is kept to a minimum also.

My current problem is like what was mentioned to me previously, too many batches, and this should be solved shortly, I've got the paging running perfectly, now I'm getting the chunking sorted out, so the chunks will become the parent to the pages, so a chunk can contain a variable number of pages, and those pages (visible ones) will become one batch for that chunk, and that chunk is part of a cube face, and the cube face will assemble the chunks into their relative batches, merging and splitting where necessary to keep in the theme of that fan I was talking about, while on the ground and close to the surface you can have up to three faces visible at the same time, those will be dealt with accordingly, in outer space, the the chunks will reduce to one per cube face, and batch the pages contained within at a reduced resolution, then I should be able to get back to the frame rates I had previously, and hopefully better.

Xadhoom asked: Does it include the upcoming Ogre paging system? Or is your own implementation to far off

I'm going to try and make it close to compatible "call wise" for pages inside the chunks, so you can sort of think of the chunks as whole paged terrains strung together like with the strategies mechanism (I'm not sure if I've got Sinbads' methods clear in my head yet, I've looked at early versions of the code, and will en devour to make mine similar in approach, so as not to make it too different.

I'll try my best

Alex

[JohnJ]

You're right, that video shows 100k triangles and it runs at 500+FPS, so I don't think you'll have any issues . The main thing to worry about is usually creating a bottleneck where the bandwidth is limiting the frame rate, in which case a faster video card usually doesn't make it run faster (which is something to avoid) and creates issues when other types of paged geometry data is added - but it seems your paging system, etc. works very effectively .

[tretle]

any chance of an updated youtube video?

[DavlexDesign]

G'day all,
I've been playing around with mesh optimization on the ground and decided to show you what I meant about the Japanese dancers fan routine ...

MeshOpt_1km.jpg

With this system, the terrain doesn't have the problem of looking like rubber when you spin around on the spot when close to the ground,
as you can see in the picture, it only shows about 1km but the same pattern goes out to the horizon, on this planet around 56 km, and when you pull out
from the planet, it goes to whatever the horizon is, down to its lowest res, which is at around 3 x 6378 km. It works beautifully !!!

Rendered.jpg

I'm still getting the final page class together to enable various different terrain strategies at the same time, and it's not far off, then I've got to
come up with some sort of file saving function to enable you to save out your planet.

regards

Alex

[DavlexDesign]

G'day again,
Here are a couple of images to give you an idea of the scale of the planet ....

TreesClose.jpg

I'm standing at the base of the trees here ...

TreesView.jpg

The tree on the right is around 20m high ... (Thanks JohnJ for the trees ( Forest Demo ))

TreesFar.jpg

My next thing is getting a forest happening, that is also going to be part of my paging system, ( another layer of detail )
I'm going to modify Paged Geometry plugin for this, so it can deal with my terrain and paging / chunking system.

regards

Alex

[JohnJ]

Nice

[mariusz.plaskowicki]

I can't wait to see planets full of trees. And to get my hands on this

[DavlexDesign]

I can't wait to see planets full of trees. And to get my hands on this

It won't be long, just getting it user friendly for all of you who want it

Alex

[Sovaka]

Very nice work there.

Starting to look very habitable

[Trancient]

I have to agree, this is looking great. I'm really going to have to find some more time to get coding!

Are you planning on a project of your own which will be using this?

[DavlexDesign]

G'day Trancient,

With reference to ...

Are you planning on a project of your own which will be using this?

YES! My son Matthew realy enjoys online games like WOW and the like, and he asked me one day to write him one (just a small ask hay!!! ), anyway, he was looking at Eve Online at the time, and commented that "wouldn't it bee realy kool if you could have WOW and Eve in the same game", well that got me thinking about some old stuff that I had been playing around with, fluid dynamic clouds and ocean waves and a terrain engine, well! I thought to myself, that will be a challenge, so here is the start of it. I have allot of the server stuff, and networking code already to go, as that is what I do, but I'd rather write game software.

To make a long story short, the plan is to write a MMOG that allows users to fly in between worlds and solar systems, each with its own unique identity, characteristics, flora and fauna and civilizations. The planet is easy, one unsigned integer creates a completely unique planetary terrain, the flora and fauna will hopefully be just as easy. The rest is a little nore involved, but I'll figure it out.

stay tuned, I'll be releasing an executable demo shortly.

Alex

[Trancient]

Well, you're certainly off to a good start then!
I think putting all the parts together is very doable, especially if you've got the Networking side down already.
I''d be interested to hear about any updates you might post about it.

Nice work, looking forward to the Demo.

[DavlexDesign]

G'day guys,

Ever tried putting skirts on a dynamic terrain system like this ? (A NIGHTMARE !!!)
I've found that when on approach to the planet now, I've been seeing seams opening up between the chunks/pages while the vertices morph into their final position (so as to keep the popping to a bare minimum). I've got it working to a point, but it's been driving me crazy. I'm seriously thinking of moving from ROAM to another approach (static chunks) like Sinbads', don't know though, I'll tackle it again tomorrow.

wish me luck

Alex

[DavlexDesign]

G'day again guys,

SUCCESS!!! I'ts the next morning and I've done it, without dropping the frame rate .....

MeshSkirts.jpg

Now to concentrate on prettying this thing up a little for the demo

regards

Alex

[Sovaka]

Awesome work Alex!

Out of curiosity, what's the largest planet you have made with decent frame rates?