Planet Rendering Engine preview (released Jul/20/2009)

[tomshackell]

A planet is basically a cube mapped onto a sphere, so you can apply 6 heightmaps currently (the planet in the screenshots above uses the same heightmap+texture for all 6 faces of the cube).

That's interesting, I was thinking about doing planetary terrain in OGRE and had the same thought. However I couldn't see anyway to do it without creating visible seams.

Consider where two sides of the cube join. Along that edge the heightmaps for both sides would have to be zero. Anything else and you'd get a crack in the terrain since one set of heights goes in one direction, and another set of heights goes in the other.

A perfect sphere wouldn't have these lines of equal height, so I couldn't see how you'd end up with an actual sphere as opposed to just a "dumpy" cube.

[JohnJ]

Anything else and you'd get a crack in the terrain since one set of heights goes in one direction, and another set of heights goes in the other.

No, it doesn't work like that. "Up" is always directly out from the center of the cube, so the seams will always match up perfectly as long as the height values are the same.

A perfect sphere wouldn't have these lines of equal height, so I couldn't see how you'd end up with an actual sphere as opposed to just a "dumpy" cube.

I don't think you're understanding the concept. While the data internally could be called "cubic", when it's rendered, the whole thing is "warped" like rubber and mathematically molded into a perfect, seamless sphere.

For example:


The 6 cube faces are folded in to form a sphere - they're not just 6 run-of-the-mill flat heightmaps hacked to look sphere-like. If the heightmap was nothing but 0 everywhere, you'd get a perfectly flat sphere. When you start to add mountains, etc., they're raised up out of the sphere.

Here's how it works internally: First, the vertex's position on the pseudo-cube is calculated (relative to the center of the planet, so it's essentially a vector). This vector is fed into an equation which warps it into sphere-space. Normally, this would easily be achievable by normalizing the vector, but if you think about it this will cause the grid tiles around the seams to become more and more tightly packed, which isn't good. The formula I use is specially designed to eliminate this and ensures that each tile has a near-uniform surface area. Now at this point, you have a cube that's been converted into a sphere, and all you need now are the mountains. This part is easy - you don't push the mountains out relative to the normals of the original cube faces as you thought, but simply multiply the current vertex's vector by a height scalar, which achieves the effect of moving them "up", or away from the planet's center. The result is a perfectly spherical, seamless, undistorted planet.

Believe me, it works, beautifully (and it looks great with dynamic lighting as the planets and their moons orbit around the sun )

[tomshackell]

Ah I see, you're warping the cube to a sphere on rendering Very cunning. I'm guessing using something like the algorithm described here:

http://mathproofs.blogspot.com/2005/07/ ... phere.html

As far as I know you can only change the height on ogre terrains, not the positions of the vertices. So this technique would presumably require the use of a vertex shader? If so I guess that would add some overhead since it would have to run for every vertex in the mesh?

[JohnJ]

Yeah, if I remember correctly that's exactly the equation I use.

There's no need for a vertex shader except for cross-LOD morphing. When generating the vertex data for the planet, feed it through the equations before saving it into the mesh.

[tomshackell]

Oh that's very interesting, I was not aware that OGRE terrains could be anything other than flat rectangular grids with uniform spacing. If you can morph the individual vertices that does indeed allow a cube of such terrains to formed into a sphere

[JohnJ]

You're correct that Ogre terrains only work with uniform grid sizes and must be flat - this would not work at all for a planet renderer. My planet renderer does not use Ogre terrains - I wrote a custom MovableObject class called "Planet" from scratch to render planets.

This is at the same level as Ogre's Mesh or BillboardSet classes, so it's very efficient and integrates seamlessly with Ogre (for example, you simply attach the Planet object to a SceneNode like you would a Mesh, and voila).

My Planet class is basically a dynamic hierarchical quad-tree of terrain chunk nodes. Each terrain chunk has a fixed size (although this isn't necessary). When the MovableObject is rendered, it starts by considering the 6 cube faces for rendering. If the resolution is too low considering how close they are to the camera (if their screen-space error value is above a certain threshold), they are subdivided into 4 smaller chunks, which doubles the resolution. This process is repeated recursively until there is an adequate level of detail. Of course there are also bounding box frustum checks and some horizon culling so off-screen mesh data isn't rendered. Chunk meshes are generated manually by populating a vertex and index buffer, which are supplied to Ogre to be rendered.

Of course there's a lot more to it, but that's the idea. Basically I don't use Ogre's or any other terrain renderer, my planet engine is the terrain renderer. It's based on Thatcher Ulrich's terrain rendering technique described here.

[booljayj]

A lot of you keep talking about using this for space games, but what about just regular old single-planet games? Could this be used for something similar to Oblivion on a global scale?

I propose a true test of your engine. NASA has detailed textures of the Earth's surface, including height maps, textures, normal maps, and specular maps. These maps are already made for you, so all you have to do is download them. How well do you think your engine would take to a planet on this scale? From what you've shown already, I don't doubt that it would run, but how well is the question.

[JohnJ]

what about just regular old single-planet games? Could this be used for something similar to Oblivion on a global scale?

Of course that would be possible, though if you go through the trouble to make a spherical world, it almost seems a waste to not see it from space

I propose a true test of your engine. NASA has detailed textures of the Earth's surface, including height maps, textures, normal maps, and specular maps. These maps are already made for you, so all you have to do is download them. How well do you think your engine would take to a planet on this scale? From what you've shown already, I don't doubt that it would run, but how well is the question.

Yeah, that would be an interesting test. Currently the planets I'm working with are 4096x4096x6 after expansion, and they still run very well from the surface (~800 FPS), and that's with several other fully visitable planets / moons orbiting in real-time as well.

It would be interesting to see how the engine does with really really really high-res data, but it's not that important anyway (to me at least) because I don't intend to use gigabytes of data per planet. Unfortunately there would be a little more work than just downloading the data to test it though - with large volumes of height data that won't fit in RAM so easily, a special loading method would need to be used. I could do it, but I'd rather not waste the time since it wouldn't be useful for most games, and certainly wouldn't be useful for the one I'm working on.

[syedhs]

NASA has detailed textures of the Earth's surface, including height maps, textures, normal maps, and specular maps. These maps are already made for you, so all you have to do is download them.

Where can I download this? Is there any license that I should be aware?

Thank you.

[JohnJ]

Yeah, I'd like to know too - now that I think about it, high res or not, something like that would be very useful when we want to add Earth in the game . The only problem might be that it's probably not perfectly suited to a cubemap approach, and might need readjusting.

[ajs15822]

I couldn't find a unified compendium for this stuff-- I suppose you'll have to compile it from multiple sources yourself. For the heightmap data, you could use data from the recent Shuttle Radar Topography Mission and for the textures, you could use data from The Landsat Program.

[booljayj]

Here is the link to the NASA textures: http://planetpixelemporium.com/earth.html

Looks like the free ones are 1k versions, I thought they were a little better than that. Oh well, at least there they are.

@JohnJ - Well, although the test might not be suited for your game, I think that your planet engine is just what I am looking for to realize my own concept game. You are correct that visualizing a semi-real-scale planet would overload the ram. However, if only the visible terrain were loaded into memory, I think that would solve the problem. Overall, I do not have the skills to do any of this anytime soon. Therefore, all I really want to do is support you as you improve this engine with features that will rival some commercial games. Good luck!

[JohnJ]

However, if only the visible terrain were loaded into memory, I think that would solve the problem. Overall, I do not have the skills to do any of this anytime soon. Therefore, all I really want to do is support you as you improve this engine with features that will rival some commercial games. Good luck!

Thanks . I kind of wish I could try some really huge planets out now, but given the amount of work it would take to write a system that handles that amount of data, it'll probably have to wait until there's a more practical need for it.

But that link to Earth and Moon maps is definitely useful. It's not very much resolution, but it's a nice free reference at least. The only problem is that it uses a different (distorted) mapping - as you can see near the top and bottom the textures are more squashed than the rest.

Actually the resolution of those planet textures are low enough that they could very easily fit in memory, so it wouldn't be too hard to load them in the current game, once I can map them to the correct format. And I'll probably do this some time too, because Earth is planned to be in the game in some form at least.

[JamesKilton]

Looking good. I'm yet another one who wants to put together a space game. Thankfully, there's an ever growing amount of information for the details required for planet rendering. The best I've seen (and most of you have, I think as well) is Inifinity: The Quest for Earth:

http://www.infinity-universe.com/

Just look at what he's been able to do with procedural generation

http://www.infinity-universe.com/Infini ... php?t=7385

Check the "Development Journal" forum for a ton of information on what and how he does what he does. It's phenomenal.

[Kojack]

$8 to download a nasa map which is higher res than 1024?
I'd rather go to nasa's website and download the 21600x10800 images for free.
http://www.nasa.gov/vision/earth/featur ... arble.html

Or for 31768x16384 images (normal map, cloud layer, specular map, etc) with 6 detail levels, go to: http://celestia.h-schmidt.net/earth-vt
Although those ones are split into thousands of 512x512 dds textures for use in Celestia.

[Falagard]

Slightly off topic, but I'm thinking about tackling porting Thatcher Ulrich's Chunked LOD system to Ogre as well - but just for flat terrains.

No specific questions at the moment except were there any major problems getting it to work? Apparently someone made a port to OpenSceneGraph with some nice additions to the functionality but I can't find it - which is really bizarre because this is the Internet for god sakes - someone must have made a backup of it before the author's site when down

[JohnJ]

The best I've seen (and most of you have, I think as well) is Inifinity: The Quest for Earth:

Yeah, I've read a lot of his development journal. Very interesting stuff - the procedural generation especially is very impressive, in addition to the rendering technology.

Falagard: There weren't any major problems that I can remember. The base algorithm is fairly straightforward. A few minor issues I had are: if you use the algorithm cull out entire nodes from being rendered, be careful that this doesn't keep those nodes from loading properly also. It's usually not a good thing when the game lags when you just try to look behind you. Another minor issue I had was that my node expiration system was only renewing child nodes that were directly being rendered from being deleted, while their parents weren't, and therefore every few seconds the entire planet would flicker as it was regenerated since deleting a parent node in my system also deletes the children.

Once you finish the chunked LOD management system, the next big job will of course be constructing the terrain meshes for each chunk. Depending on how you want to do this, it can be complex or simple. I've found that simply using a fixed grid resolution works very well (as opposed to doing full mesh polygon reduction as a preprocessing step), but depending on the uses of the terrain (editable/procedural vs. pregenerated), you could try both methods.

[chuck_starchaser]

Perhaps you would like to look at the possibility to merge your techniques / rending with their gameplay engine. If it's at all feasible, it would be an incredibly powerful combination.

I'm sure this engine would be a great combination, and I'll be glad to open-source it when the time comes . Unfortunately, now just isn't the time for me to open-source it for various reasons (incompleteness, many technical difficulties yet to be overcome, lack of tools, in addition to the reasons mentioned previously).

These are arguments for open-sourcing; rather than against it.
What's the point of waiting to open the sources for development until all the problems are solved?

BTW, I think the greatest thing would be to open-source the framework, WITHOUT the procedurals. There have been and there will be gazillions of procedural algorithms for terrain generation; and the mistake everybody in the planet-modelling business always seem to make is embedding an algorithm for terrain generation. That should be a plug-in. If you can restrict this project to the LOD-ing framework, you'll make your life easier, AND this piece of code a million times more popular.

Heck, there's people out there who will want ultra-fast and lightweight planets; and others who will want to sacrifice nothing to realism. You won't make a hell of a lot of people happy by forcing a generative algorithm down their throats. And that's the part everybody likes to tweak, anyways. The hard part to do, and what everybody needs, and not now, but ***yesterday***, is a functional framework --i.e.: something that hopefully works, for a welcome change.

Damn, there must have been hundreds of cracks at this very problem, already, by hundreds of programmers, in the past 10 or 20 years; and none of them ever got finished. None that are open-source anyways. And the world keeps waiting...

Please think of terrain as "data"; and leave that problem to us, the users; we'll be glad you did, and you'll be glad you did too. You can ignore the whole dilemma of whether to use stored data, or going purely procedural, or something in-between, or whether (or how) to allow edits. None of it is germain to what the world needs. What we've been asking for, for the past 20 years, is a planetary LOD framework we can actually use in space games, as opposed to yet another storm of sweet pie in the sky.
http://vegastrike.sourceforge.net/forum ... php?t=4597
http://vegastrike.sourceforge.net/forum ... hp?t=10520

I'm a PU developer, btw. Rest assured if this code happens, it WILL be used. But your saying that "it's not ready to open-source because it's not finished yet" is NOT encouraging... Sounds like yet another doomed start... Please release the code now, so that others can help you when you get stuck, or run with the ball if you get bored. Even if you did nothing else, everyone will remember your name as the guy who got it started. Keeping the code closed is in nobody's interest.

And please leave terrain generation as an external plug-in. There will never be an algorithm that is good for every kind of planet, and there will never be an algorithm that pleases everybody for any given planet type. And those algorithms grow on trees. Just call a call-back float height( float longitude, float latitude ) function; and let us worry about how to write it.

What's missing and badly needed is just the LOD framework.

[JohnJ]

First of all, I agree with you that procedural generation should be optional and at the plug-in level, in fact that's exactly how I already have it working. The code that loads the terrain is separate from the code that actually renders it.

Rest assured if this code happens, it WILL be used. But your saying that "it's not ready to open-source because it's not finished yet" is NOT encouraging... Sounds like yet another doomed start...

Sorry if I sound rude, but that's not exactly the best way to ask someone to release commercial code under open-source. FYI the project is moving along very well and the planet terrain rendering code is almost done, so no, it's not doomed.

I don't disagree with you that this would benefit many to be open sourced. I can probably think of 100 reasons to open-source it, but unfortunately I also have to think of 1 very good reason not to: this technology is going into what will most likely be a commercial game, and understandably it's usually not a good idea to open-source a commercial game's technology at least until after the release.

Here's a screenshot though

[The FPS is down to ~600 here I think because it was running at 1280x1024 with 4xAA. Also it's an old screenshot, as you can see the star is just a white ball (this was later replaced with a nice procedurally animated sun shader)]

BTW, it's really not all that difficult to achieve something like this - just read up on Chunked LOD (a very easy and effective terrain LOD algorithm) and "spherical-izing" cubic coordinates, and start working on it. Hopefully you should be able to get something up and running without too much trouble. I think it only took me a week to code the engine that rendered the first screenshots on this thread.

[2asueekim]

Hi, I was wondering if you could maybe help vegastrike with its planets?
Vegastrike is a free and opensource space sim that's been around nearly 10 years now, maybe more. There has been talk of "seamless planetary flight" for 4 years or so but no one on the dev team has been able to pull it off (they aren't really capable of it). You've made here just what VS needs (though vs would need larger scale planets). I was wondering if maybe you could help them out with your talent? They have a site and forum at http://vegastrike.sf.net

[JohnJ]

I'd be happy to help anyone who's trying to implement a planet renderer. Just ask what you need to know, and I'll try to answer as well as I can (short of providing my source code, which I can't do).

[db123]

You are genius!
By a beginner,I wan't to know how you can make it!
I think it will be very excellent!

[2asueekim]

I'd be happy to help anyone who's trying to implement a planet renderer. Just ask what you need to know, and I'll try to answer as well as I can (short of providing my source code, which I can't do).

How is the level of detail stuff done, how do you load it. Like say you had generated your planet texture and heightmap (to some level of detail, maybe down to a kilometer, or 10 kilometers) and you have your salt for the planet saved so the generator will create that same planet from now on. How do you work it so as you get closer and closer to a sector more detail is generated from that sector. This seems to be the big hang up. Libnoise can generate planet maps for instance but I and others don't know how to generate (probably using fractuals) and load higer areas of local detail as you get closer and closer and are finnaly 1 meter from the planets surface.

Kinda what charlieg is saying I suppose.

[JohnJ]

I probably described this before, but here's how it works:

There are two basic "modules" to the planet renderer - the base LOD manager, and the planet geometry loader. The base LOD system is a quadtree style scene manager except it's not a scene manager, but implemented as a MovableObject. This way I can bypass using SceneNode's for each terrain chunk and therefore gain a lot of speed. The system loads terrain chunks using the algorithm described in detail in the Chunked LOD paper, except there are actually 6 terrain planes, one for each side of the virtual cube. I say "virtual" cube because it's only a cube at one level of thinking - the resulting planet is 100% spherical after the cube-space vertex positions are fed into the mapCubeToSphere() function, which uses a special equation to convert coordinates of a subdivided grid into a undistorted sphere. Height values are applied after the cube-to-sphere conversion level, by multiplying the vertex vector (and it is a vector since the planet is centered around (0,0,0) local) by a scalar.

Terrain chunks can be deleted by setting an expiration time and deleting ones that haven't been in view for a while. However be careful not to let chunks just out of the view frustum expire unless this is desired, and usually it's not because it causes lag when just rotating the camera around. There are several ways to delete terrain chunks, depending on if you need it thread-safe or not. It's possible to make a thread-safe deletion function without using (inefficient) queues, but I'll leave that as an exercise to the reader, as they say

Now as to actually loading the heightmap data, this is the one area where the engine is not fully complete. There are still lots of optimizations to be done (although it runs pretty well already), and some features to add. Basically my plan is procedurally increase the detail of the fixed-res heightmap as needed, while still maintaining control over the terrain's features through use of "property maps", which provides a finer degree of control over the procedural generation. I'm in the process of working on this, so I couldn't help much here, but just read up on procedural heightmap generation, which is what I did.

I'm probably not being very specific though, so I can elaborate a little on some areas if you want.

[Lord Alexion]

Can you elaborate further on the cube-to-sphere process, more specifically the equation used to warp the vertices?