Simple Paged Terrain [v2.00 Released! Page 21]

HexiDave · Post by **HexiDave** » Tue May 22, 2007 1:57 am

Quick post:

This is probably going to be a hellish week, so I'll post now while I can.

I noticed a flaw in my thinking about my floating origin setup that would basically defeat the purpose eventually. I have a new method I'm working on that will work properly and give you the ability to travel endlessly without precision issues as well as get accurate placement information to send across networks.

As for the trees/Caelum demo framerates - I don't know what to tell you, really. I just added them for the demo to show what can be done and what I added was directly from their respective samples.

As for Nimble/SpeedTree - I'm pretty poor, so integrating either of these commercial items into something I'm working on in my spare time (well, attempting is a better word) would be impossible. However, the terrain system will be setup in such a way that you can get positions and height values easily, so if YOU get either of them then you're welcome to integrate them for your own project.

As for Next-Gen type terrain systems, I don't know how well clip-maps would integrate, but Continuous LOD will be added eventually.

Again, sorry for the delays - releasing anything now would be a waste as what I have is very little improvement over the first release as far as usability goes. It has settings and file-loading now, but it's still un-documented and hard-edged, not to mention the fact that the new floating origin system won't be compatible with the old one...

For now, use TSM, Editable TSM or PLSM2 - they're all great options and your data will work here as well when it's all good to go. Tuan has PLSM3 on the horizon, don't forget, so you'll have another terrain option in the future.

nemro · Post by **nemro** » Tue May 22, 2007 10:32 am

As for the trees/Caelum demo framerates - I don't know what to tell you, really. I just added them for the demo to show what can be done and what I added was directly from their respective samples.

As it seems, particularly the trees drop the fps.
At first sight, your terrain engine looks very much it could be used productively very soon.
But if it is a problem with the engine architecture, it would not
useful to take into account using the engine productively.
Therefore I ask these question.

As for Nimble/SpeedTree - I'm pretty poor, so integrating either of these commercial items into something I'm working on in my spare time (well, attempting is a better word) would be impossible.

Maybe they let you integrate their libs in the demo without a fee.
Such a demo would be a great advertising for all products.

However, the terrain system will be setup in such a way that you can get positions and height values easily, so if YOU get either of them then you're welcome to integrate them for your own project.

Yes, thats right.

For now, use TSM, Editable TSM or PLSM2 - they're all great options and your data will work here as well when it's all good to go. Tuan has PLSM3 on the horizon, don't forget, so you'll have another terrain option in the future.

Then the first impression deceived me. But just for working on spare time
your terrain engine looks promisingly.

nemro

Alexander · Post by **Alexander** » Tue May 22, 2007 2:01 pm

nemro wrote:Maybe they let you integrate their libs in the demo without a fee.
Such a demo would be a great advertising for all products.

I'd say there's about zero chance of that.

Alexander · Post by **Alexander** » Tue May 22, 2007 2:10 pm

HexiDave wrote:Ah, well if it's a recognized format, then I can probably do something with it.

It might be easier to just support a generic image file format since almost all DEM data needs to be processed to fill in gaps (3DEM is commonly used) and would be exported anyway. There's plenty of workflows to get DEM data into Tifs or anything really.

Post by **jacmoe** » Tue May 22, 2007 11:23 pm

HexiDave wrote:I have to push the release back a bit - more work came up (stupid life!).

Again, sorry for the delays (please don't kill me )

That's called 'Real Life creep' - I wish there was a function like RealLife::normalise(). But there isn't.

HexiDave · Post by **HexiDave** » Wed May 23, 2007 1:35 am

jacmoe wrote:
HexiDave wrote:I have to push the release back a bit - more work came up (stupid life!).

Again, sorry for the delays (please don't kill me )
That's called 'Real Life creep' - I wish there was a function like RealLife::normalise(). But there isn't.

I'd much rather have RealLife::destroy() at this point.

Anyways, Alexander: I already support filling the heightmap on your own in a few ways - the only thing I'll have to add in the future is pulling that extra border info to build normals. The way I do it is to have an extra 1 pixel border (i.e. 513x513 heightmaps are instead 515x515 with neighbor info) such that I don't have to rebuild normal info between pages.

I also agree that I don't think Nimble or SpeedTree would care to let me integrate their stuff for free - I'm fairly sure SpeedTree has a demo library, but I'm fairly certain that a few people in the Ogre community are building replacements to SpeedTree's library system. Besides, that tree system (PagedGeometry by JohnJ) does essentially the same thing as SpeedTree - I just said I hadn't tweaked it to play more nicely with the demo (it was extremely dense forest, so no surprise on a slowdown).

I've got some time to work on this thing tonight, so I'm going to try to implement the new floating-origin/wrap-around system which should solve all my problems, get rid of precision issues and still work with networked position information.

The basic premise is that it keeps track of which Page is it's "home" position (i.e. 0,0 would be home to start with for most cases, but it doesn't matter which it's set to) and will float the origin in regards to it's top-left corner (again, for Page 0,0 the top-left corner will be 0,0 in X,Z Real coordinates) and then adjust that after you cross a certain number of Pages too far from the current origin. For instance: Your current home Page is 0,0 and you travel past the camera's "stretch" point, so now it will shift everything in the scene back that distance, so the camera is at 0,0 in X,Z coordinates. Once that total shift accumulates to, say, 2 Pages in either X or Z, it will shift the Pages such that the new home would be Page 2, 0 if you went too far in the +X direction and the total origin shift is reduced 2 Page widths, so it's small again.

It might seem like a gross over-complication, but think about it this way: If you were to have such a large world that would have 65536 Pages at 513 vertices each, that alone is (accounting for overlapping vertices) 33,554,433 integer positions. That might not seem bad, but if you scale it by 200.0 in X and Z, that's an enormous number. Now even if you're shifting around the origin with doubles, you're still going to get some problems. My setup keeps returning the origin to 0.0 with a shift that is reset as well after you move several Pages. No precision problems even with Real's set to "float" (i.e. no Double Precision mode) this way, and you still retain "real world" coordinates based on Page number and offset (i.e. your tank is at Vector3(500.0,200.0,300.0) from Page 900,900's top-left corner).

I'll certainly include ways to turn these features on and off at compile-time, so you don't HAVE to use these features, but you'll start noticing problems with large worlds, so they'll be needed there.

Lord LoriK · Post by **Lord LoriK** » Wed May 23, 2007 4:16 am

HexiDave, I have a new suggestion: add a way to get a height from any point of the hightmap. This would avoid unnecesary ray scene queries (you can interpolate between four known vertices to get an exact height).

What do you think?

HexiDave · Post by **HexiDave** » Wed May 23, 2007 4:21 am

Already in

You can get it as a height value (i.e. "give me height value at (0,0) of the heightmap") or as a height value in scaled, real space through interpolation (that is, I have a function that interpolates/scales for you).

Lord LoriK · Post by **Lord LoriK** » Wed May 23, 2007 4:31 am

HexiDave wrote:Already in You can get it as a height value (i.e. "give me height value at (0,0) of the heightmap") or as a height value in scaled, real space through interpolation (that is, I have a function that interpolates/scales for you).

Hell, that was fast!

deficite · Post by **deficite** » Wed May 23, 2007 6:25 pm

HexiDave is definitely my hero. I await your terrain manager, good sir!

nemro · Post by **nemro** » Thu May 24, 2007 10:20 am

Maybe this site could be interesting for you.

nemro

nemro · Post by **nemro** » Thu May 24, 2007 10:31 am

Alexander wrote:
nemro wrote:Maybe they let you integrate their libs in the demo without a fee.
Such a demo would be a great advertising for all products.
I'd say there's about zero chance of that.

Maybe the chance is somewhat higher then zero http://www.ogre3d.org/phpBB2/viewtopic.php?t=32369

nemro

HexiDave · Post by **HexiDave** » Thu May 24, 2007 10:33 am

I remember reading that over when working on my planet engine. I use 3D vectors for positions whereas they mostly use spherical coordinates - it's faster for what I use in my system. This terrain system is flat, so it's a rather different approach.

Edit: In response to Nimble - that's all GPL stuff, so it is possible, but not a focus for right now. I'm focusing on the terrain aspect itself - I've got limited working time and I'm just going through the motions of trying to build a bullet-proof system for the terrain use. It's working well so far, but I just need to make sure it's applicable in just about all situations without needing massive adjustments.

Alexander · Post by **Alexander** » Thu May 24, 2007 1:20 pm

nemro wrote:
Alexander wrote:
nemro wrote:Maybe they let you integrate their libs in the demo without a fee.
Such a demo would be a great advertising for all products.
I'd say there's about zero chance of that.
Maybe the chance is somewhat higher then zero http://www.ogre3d.org/phpBB2/viewtopic.php?t=32369

nemro

You see? One comment from me and they cut this deal.

Would be interesting to see what they're going to do with the technologies.

Lord LoriK · Post by **Lord LoriK** » Sat May 26, 2007 4:31 pm

This silence is a bit disturbing. Please, HexiDave, say something!

deficite · Post by **deficite** » Sat May 26, 2007 5:27 pm

This is probably going to be a hellish week, so I'll post now while I can.

HexiDave · Post by **HexiDave** » Sat May 26, 2007 6:22 pm

I'm not dead, just taking a breather. I've had a little time this week to work on the project and I'm taking it one step at a time. Before when I had a lot of time to work on it I could blow through whole sections because I didn't lose the rhythm of work, but it's kind of stop-go, stop-go right now.

Yesterday I was drained pretty badly and wasn't doing too well (we had a nice temperature spike and I wasn't aware of it until I woke up... in my 95 degree room...), but today's looking up.

I didn't make any update posts because I hadn't made any major progress. I'm not gonna release anything until I get this new floating-origin system finalized and functional, at which point just about everything else falls into place (it's essential to precision and it is in a lot of different systems).

I'm going to do my damnedest to get some work done today, so I might make a pot of coffee and just GO. I haven't had caffeine in about a year, so if I never post again, that means I died.

Thanks for keeping an interest in the project - sorry it's going so slowly after showing a lot of progress early on. All that early stuff was cake, but the wrap-around system vs. the precision systems REALLY slowed it down and I had a lot of unexpected work crop up to boot, so it all went downhill from there.

Edit: WOO! 500th post!

ajs15822 · Post by **ajs15822** » Sat May 26, 2007 6:40 pm

HexiDave wrote:I'm not dead, just taking a breather. I've had a little time this week to work on the project and I'm taking it one step at a time. Before when I had a lot of time to work on it I could blow through whole sections because I didn't lose the rhythm of work, but it's kind of stop-go, stop-go right now.

Yesterday I was drained pretty badly and wasn't doing too well (we had a nice temperature spike and I wasn't aware of it until I woke up... in my 95 degree room...), but today's looking up.

I didn't make any update posts because I hadn't made any major progress. I'm not gonna release anything until I get this new floating-origin system finalized and functional, at which point just about everything else falls into place (it's essential to precision and it is in a lot of different systems).

I'm going to do my damnedest to get some work done today, so I might make a pot of coffee and just GO. I haven't had caffeine in about a year, so if I never post again, that means I died.

Thanks for keeping an interest in the project - sorry it's going so slowly after showing a lot of progress early on. All that early stuff was cake, but the wrap-around system vs. the precision systems REALLY slowed it down and I had a lot of unexpected work crop up to boot, so it all went downhill from there.

Edit: WOO! 500th post!

Woot, congrats!

We've all had days (weeks, months) where things just don't go the way we planned and entirely new obstacles appear without warning. Don't sweat it HexiDave, you've already done so much for this community (for free!), anything else you can contribute is totally just icing on the already-delicious cake.

HexiDave · Post by **HexiDave** » Sat May 26, 2007 7:09 pm

Thanks muchly for the reassurances. I've had this happen before and it usually leaves me bug-eyed for a few days, but I'm already feeling better and making a bit of progress (got the world coordinates functioning with all sorts of precision-friendly things now), so as long as the day cools off a bit I think I'll be able to get a lot done.

I think the reason why I made so much progress in the beginning was because I was so pissed about the major glitch in my planet rendering project and had a lot of energy to burn on terrain

deficite · Post by **deficite** » Sat May 26, 2007 9:32 pm

sorry it's going so slowly after showing a lot of progress early on.

No need to apologize, man. (Most of us) understand how life goes, and I mean you're doing this in your spare time. I'm grateful for what you've done already, and I'm really excited about getting to play with this thing. This is the most exciting project in OGRE since I've started using OGRE IMHO.

So, good luck, and get yourself an AC, lol.

Jerky · Post by **Jerky** » Sun May 27, 2007 7:31 am

ajs15822 wrote:We've all had days (weeks, months) where things just don't go the way we planned and entirely new obstacles appear without warning.

Heh, thats like everyday of the 2.3 years of PW's life.

HexiDave, you're doing a great job.

HexiDave · Post by **HexiDave** » Sun May 27, 2007 7:51 am

Thanks

I'll give you an example of what I'm talking about, though:

Yesterday was my off-day. Everything was smooth - I even had time left to go see Pirates of the Caribbean with my friend. So what happens?

Pipe in the basement busts. 3 hours out the window. Then my friend calls and tells me he wants to go a different day instead. I had just gotten ready for the movie, so that's another 45 mins to an hour out the window.

I think "ok, now that's all done I'll go sit down and program." Sure, ok, I sit down, load up my IDE... and the light in my lamp blows out.

What.
The.
Hell.

I got some stuff done - the floating origin stuff is looking good now and seems to be happy with wrap-around or not, too. I also created a test-tile system with colored (excuse me, "coloured") ManualObjects that's easier to see where the Pages meet up and is nice and fast.

So, progress is happening - not at the speed I like, but that's life, eh?

kcbanner · Post by **kcbanner** » Sun May 27, 2007 9:04 pm

Wow talk about bad luck

Don't worry about us man, the work your doing is great and we're not paying you so take your time

HexiDave · Post by **HexiDave** » Tue May 29, 2007 10:44 am

Quick update: floating origin seems to be working flawlessly now - totally precision-problem-free. I haven't seen any bugs and I've let it just run (i.e. go really fast in random direction, then repeat for another random direction... for 6 hours, unattended), so that's a good sign. I'm going to connect it to the real terrain again shortly and clean up what code I have and re-integrate the Config system that Ogre was nice enough to supply...

The screenshots don't make any sense, so I'm not posting them - I need to make a diagram or two for demonstrations as the system is a bit weird. It works, though, so that's good enough for now

It's all just random colored squares for Pages with MovableText above it to display it's world position and Page number.

I'm going to re-build the Queue system (I HATE this thing) to allow threading instead of my build-queue system, should you want to use it that way (for multi-core systems). I need to re-build it anyways because it's a bit screwy when dealing with the actual construction phase - if you're moving around and causing the origin to jump around mid-build, the Tiles end up all over the place (that screwed me up for a while). I know how I need to fix it, but I'll have to do some unit-testing with it since it's abstractable to work with anything that needs Queuing (i.e. Heightmaps, Tiles, etc).

Anyways, still not dead and making some progress. I also read about Ambient Aperture Lighting, which is great for terrains, so I'm going to look at implementing that in the future. Cheap horizon mapping? Yes, please...

HexiDave · Post by **HexiDave** » Tue May 29, 2007 2:49 pm

Woo, found my copy of Fraps (was on a disc which was in a stack being used to prop up a cabinet in my room!) and I made a quick recording to show what I'm screwing around with:

http://www.fileden.com/files/2007/5/28/ ... 20Test.avi

Pretty sure I set it to use Xvid codec, so if it bugs out on you, try Media Player Classic or VLC Player.

If the video isn't too small for you, you can see at the bottom (when the debug overlay is up) some numbers:

First two numbers are the actual Page number that you're over (i.e. Page 0,0)
Next are the two World Page numbers - as you get further from the starting point, it builds up to give you a real-space position in Page numbers. The reason for this becomes apparent when you start working with wrapping around a number-line. Try it - write 0 - 3 on a paper and keep marking it over and over again. Now pick a 0 and tell me how far away it is from another 0; now tell me how you know which direction you're going

Next number isn't of use for this video, it was just a helper number for Height off the fake ground.
Lastly is the actual camera position. You'll notice that it never really gets too big - that's the Floating Origin system hard at work.

I left out the automatic recycling for Pages that get too far way since I wanted you to see how far I'm getting while traveling. You can see the individual Page's number and World Position in red text (courtesy of MovableText in the wiki) as I zip along.

I tried posting this on Google video but it came out looking like crap on there - if it looks really bad for you, let me know and I'll make a better copy (my monitor is really wacky, so my gamma/brightness are a bit off).