Planet Rendering Engine preview (released Jul/20/2009)

[Shadow007]

Not sure it's exactly the same equations, but I seem to remember somthing that could be related at Iñigo Quilez homepage

[JohnJ]

Can you elaborate further on the cube-to-sphere process, more specifically the equation used to warp the vertices?

I think the general equation is posted in several places around the internet, but here it is in Ogre-compatible format for your convenience :

Code: Select all

		Vector3 mapCubeToUnitSphere(const Vector3 &cubeCoord)
		{
			Real x = cubeCoord.x;
			Real y = cubeCoord.y;
			Real z = cubeCoord.z;

			assert(x >= -1 && x <= 1 && y >= -1 && y <= 1 && z >= -1 && z <= 1);

			Vector3 sphereCoord;
			const Real div3 = 1.0f / 3.0f;
			sphereCoord.x = x * Math::Sqrt(1.0f - y * y * 0.5f - z * z * 0.5f + y * y * z * z * div3);
			sphereCoord.y = y * Math::Sqrt(1.0f - z * z * 0.5f - x * x * 0.5f + z * z * x * x * div3);
			sphereCoord.z = z * Math::Sqrt(1.0f - x * x * 0.5f - y * y * 0.5f + x * x * y * y * div3);

			return sphereCoord;
		}

P.S. Sorry I missed your post and took so long to reply.

[TheOtherGuy]

Frame rates are indeed impressive

I've made an attempt at a "planetary rendering" implementation but am only getting around 30fps with 100K visible triangles and a batch size of 2K triangles. Admittedly my platform is a ATI Radeon 9550 on a P4D 3Ghz DDR400 mobo, but this graphics hardware isn't too many generations behind at 7800GT. I am doing a lot of occlusion and frustum culling on the CPU and with a quad tree that gets subdivided up to four times, the ATI GPUPerfStudio tool tells me I am CPU limited (haven't gone multi-threaded, so only using one CPU core).

Are you able to reveal how are you colouring the vertex here?
It looks from the early screen shots like you are using the same height map on each face. Have you gotten to the point where you are using different height maps on each face? I'm currently trying to tackle texture mapping and my fragile little mind is having some difficulty with the mapping to each face of the cube

[JohnJ]

It looks from the early screen shots like you are using the same height map on each face. Have you gotten to the point where you are using different height maps on each face?

Yeah.

I'm currently trying to tackle texture mapping and my fragile little mind is having some difficulty with the mapping to each face of the cube

It's really not all that different from texturing a single flat terrain, except you're applying your terrain material to all 6 faces of the terrain cube. As long as you apply useful UV values to the vertexes (so that each face is mapped from (0,0) to (1,1)), it works pretty well.

the ATI GPUPerfStudio tool tells me I am CPU limited (haven't gone multi-threaded, so only using one CPU core).

Yeah I haven't gone multi-threaded yet either (although it's thread-ready) - this way I can more easily identify parts of code that cause lag and eliminate them for a more efficient multi-threaded system later

I think the reason I'm getting such high FPS is because I optimized the core planet manager system right from the start to be extremely efficient from the lowest level, and carefully designed the high-level code to be near optimal, algorithmically. But really, the chunked LOD algorithm isn't that complex, so with a little work, it can have almost no CPU (and memory) overhead at all.

Are you able to reveal how are you colouring the vertex here?

I'm not quite sure what you mean by coloring. Do you mean texturing? Lighting?

I can reveal the basics of the planet renderer's operation, like how I adapted chunked LOD to work, etc., but I just can't give you code (other than the equation above) or too specific implementation details (like how specifically I got it to run as fast as it does).

[TheOtherGuy]

Thanks for the feedback John – I appreciate this is a commercial venture.

I think the reason I'm getting such high FPS is because I optimized the core planet manager system right from the start to be extremely efficient from the lowest level, and carefully designed the high-level code to be near optimal, algorithmically. But really, the chunked LOD algorithm isn't that complex, so with a little work, it can have almost no CPU (and memory) overhead at all.

I have done a reasonable amount of tweaking with regard to when I switch to software culling and when I just send everything to graphics hardware but always end up maxing out either my graphics card or CPU with the same resultant frame rate. Sitting down and doing some in-depth profiling would probably go a long way however, looking at the frame rates you are getting my current approach is flawed from the outset.

I'm not quite sure what you mean by coloring. Do you mean texturing? Lighting?

My question really relates to texture mapping and scalability without using too much video RAM. I may be encroaching on proprietary intellectual property here – looks like I need to spend some more time sitting in a dark corner with pencil and paper


Keep up the good work!

[JohnJ]

My question really relates to texture mapping and scalability without using too much video RAM. I may be encroaching on proprietary intellectual property here – looks like I need to spend some more time sitting in a dark corner with pencil and paper

Well it's pretty much like all the other terrain texturing algorithms - not much is changed here by planets being spherical. I guess the biggest challenge is texturing such a huge world with a little memory. But since we haven't even finalized our texturing technique yet, I really can't give much advise here (at least with confidence).

[Temaruk]

Impressive work! How about Megatexturing for terrain?

[Brocan]

I'd be happy to help anyone who's trying to implement a planet renderer. Just ask what you need to know, and I'll try to answer as well as I can (short of providing my source code, which I can't do).

I'm one of these people . I have just read the whole post, but i'm still having a ton of doubts. And my implementation is a big disaster

Hi have a terrain chunk entity that is the quadtree. Then I have Quad nodes with Renderables. This renderables represents each chunk, and derives from renderable and movable object.

Well, a lot of questions and doubts:

A) How do you manage the geometry? Do you have a simple mesh for each lod, and share it for the renderables of this lod level? (putting the correct indices for each renderable, by overriding the getRenderOp method?). Or you have a vertex and index data for each chunk?

B) How manage the geometry? i have read that you use some kind of dynamic loading; Dynamic loading / unloading can't be too slow?

C) How do you manage the position of chunks? one scene node for all the tree, and local positions for the chunks, or a scene node for each chunk? (I have understood that you use the second, i have tried both, and with the second, i have the problem that when you move the camera, a gap between chunks appear)

D) How do you manage the updating?, because now, i have a update function that is called each frame if the entity is visible. It watch all the tree updating the visibilities of each node. I think that if I have 20 planets in a solar system, this is going to be inefficient.

E) And the last, is ogre related, i have doing a renderable per chunk. Do you write that chunk lod is very well for batching, but with a renderable per chunk, i have a big batch count when i'm near to the planet.

Feel free to answer the questions what you want

Thanks in advance

[JohnJ]

A) How do you manage the geometry? Do you have a simple mesh for each lod, and share it for the renderables of this lod level? (putting the correct indices for each renderable, by overriding the getRenderOp method?). Or you have a vertex and index data for each chunk?

Each "chunk" of terrain could be thought of as a mesh, although it's not represented by Ogre's "Mesh" class.in the implementation. A chunk is basically a vertex buffer / index buffer (implemented as a Renderable). In my implementation I reuse the same index buffer for all chunks because I use a fixed chunk resolution (this wouldn't work for polygonal optimized chunks, for example).

B) How manage the geometry? i have read that you use some kind of dynamic loading; Dynamic loading / unloading can't be too slow?

I think this is explained in the ChunkedLOD papers - if a chunk is in view, start loading it's children (this can be threaded). If out of range, the chunk can be unloaded. Dynamic loading / unloading can be very fast if done right, since terrain mesh data isn't that heavy (as long as you keep your chunk resolution reasonable).

C) How do you manage the position of chunks? one scene node for all the tree, and local positions for the chunks, or a scene node for each chunk? (I have understood that you use the second, i have tried both, and with the second, i have the problem that when you move the camera, a gap between chunks appear)

Chunks are identified with a face ID (which face of the cube they belong to), and a 2D bounds value indicating the location (0,0 - 1,1) on the cube face. From this, the chunk's vertexes, bounding box, center, etc. is calculated.

D) How do you manage the updating?, because now, i have a update function that is called each frame if the entity is visible. It watch all the tree updating the visibilities of each node. I think that if I have 20 planets in a solar system, this is going to be inefficient.

When the planet is rendered, all the visibility calculations are performed. For loading / unloading, this is done in a update function or separate thread to avoid stalling the render process.

E) And the last, is ogre related, i have doing a renderable per chunk. Do you write that chunk lod is very well for batching, but with a renderable per chunk, i have a big batch count when i'm near to the planet.

If your batch count is getting too high, then you should probably increase the resolution of your chunks. By balancing the size of each chunk, you can achieve a perfect balance of batching / culling. The reason CLOD is good for batching is because each "chunk" is basically a batch, and all the chunks have roughly the same number of vertexes (or exactly the same, depending on the implementation), so by setting the chunk size to a good batch size, everything performs really well.

[Brocan]

Thank you

[JohnJ]

I forgot to mention:

I think that if I have 20 planets in a solar system, this is going to be inefficient.

For multiple planets and solar systems, you're probably going to want to implement an additional LOD system, where for example you can render planets as a simple normal mapped sphere when distant, and as a normal mapped sprite when even farther away.

[Bakerman]

This is looking brilliant! I'm another hopeful aspirant to making a space game, and despite what you say, the amount of work to come this far looks staggering . Though I have other projects on my plate right now - I'll come back in a few years and see how things stand

I'd be most interested to know where I should start to implement something like this. I guess, obviously, I'd need to read up on the algorithms necessary for the planet's LOD. HOw much OGRE-specific knowledge is needed - and how much of the gritty, low-level rendering stuff? I've got to admit that these two are my weak points right now .

I think what I'll do is focus on the actual gameplay first - controlling a ship in a huge area, physics/collision, etc. And see if it's any fun...

[JohnJ]

I'd be most interested to know where I should start to implement something like this. I guess, obviously, I'd need to read up on the algorithms necessary for the planet's LOD. HOw much OGRE-specific knowledge is needed - and how much of the gritty, low-level rendering stuff? I've got to admit that these two are my weak points right now .

Optimally, you'd want to be somewhat familiar with writing extensions of Ogre::MovableObject and Ogre::Renderable to do custom rendering. Otherwise, it probably won't run fast enough. To do this you'll have to have construct vertex and index buffers manually too, so it's definitely more advanced than some other methods, but it's also the fastest.

[dkx187]

you should integrate hydrax into it but it is looking pretty cool.

[Moohasha]

Hopefuly this thread isn't dead (the last post from JohnJ is back in August). First of all, this is amazing and I'm really impressed with what you've done. Kudos.
I'm trying to make a Google Earth like application (not game), so this is very interesting to me. The three main differences I see between what you have done and what I need are as follows:

• 1) Your planet is a cube mapped to a sphere, where as I'm working with a "uv sphere" (lat/long lines). While not completely accurate, you can think of each initial chunk as a 1x1 degree area of the planet, so they taper off toward the poles. Point is, this is different from the geometry you have.
  
  2) It's impossible for me to load all of my elevation data at startup because I'm using DTED (Digital Terrain Elevation Data), which is about 1.5 Gb for the entire world. So I need to be able to load the elevation at run time for the relevant areas when the camera is close enough to the ground.
  
  3) It is also impossible for me to load all of my imagery at startup, because I'm using Google Earth like imagery that itself is "subdivided" into higher resolutions. So just like you may start with one chunk and divide it into 4 smaller ones, I start with one image and when the chunk is subdivided I load 4 higher resolution images to texture each one with so that the imagery is never blurry (except where higher-resolution imagery isn't available. So essentially, when the resolution of the geometry increase/decreases, the imagery also increases/decreases.

Now, I also have a few questions about your implementation:

• A) Do you load all resources (heightmaps, textures, etc...) at startup, or does some loading occur at runtime. ie, it makes more sense to load the elevation for a planet as you get close to it rather than at startup if the planet is very far away.
  
  B) What is your current load time for all resources for a single planet? (Please include system specs to put into perspective).

I have no experience with paging or landscape generation, but I have to say after a few days of tackling it on my own I have an engine that is at best "sort of okay". If you can provide any assistance with how to go about building an engine like what you have with my requirements, I would be eternally greatful.

[JohnJ]

A) Do you load all resources (heightmaps, textures, etc...) at startup, or does some loading occur at runtime. ie, it makes more sense to load the elevation for a planet as you get close to it rather than at startup if the planet is very far away.

In my implementation, each planet has a set of maps (height map, normal map, splatting map, etc.) which are all loaded when you approach a planet. Even though my planets are big, they aren't full scale, so with a little dynamic procedural enhancement, this approach works fairly well for me (so far).

However, the Chunked LOD algorithm (if you read the paper on it) was actually designed for exactly what you're saying: it can load the heightmap and texture maps, etc. dynamically only as needed. It's fairly easy to do.

B) What is your current load time for all resources for a single planet? (Please include system specs to put into perspective).

With threaded resource loading, it took about 1 or 2 seconds to load all the maps needed when you approach a planet on my old computer (2.2 GHz single core, 1 GB RAM, GeForce 7800 GT).

If you can provide any assistance with how to go about building an engine like what you have with my requirements, I would be eternally greatful.

My main suggestion would be to read all you can about Chunked LOD and maybe other terrain LOD techniques, and try to figure out something that will fit your needs. Most importantly (to begin), find the best way to graph the hierarchical LOD structure in memory. For a cube-based solution like mine, it's as simple as a quad-tree subdivided cube face system, but for your UV-sphere type planet, it might be more difficult. Once you can subdivide chunks as you like, the rest will probably be a little more straightforward (but not easy, for sure ).

[Moohasha]

I read the paper on Chunked LOD and it was very helpful. Thank you. But I'm completely inexperienced with terrain rendering, and despite having dabbled with it for a few years I'm still relatively inexperienced with 3D graphics in general, so I was hoping you (or anyone else) could explain a few terms/concepts that the paper describes in not-so-technical terms to help clarify them.

• A)

  delta represents the maximum geometric deviation of a chunk (in object space) from the portion of the underlying full-detail mesh it represents.

  (found on page 6 just above equation 1). What does that mean? Is that simply how many sub-meshes can make up that chunk when subdivided?
  
  B)

  We start by choosing a value for our maximum tolerable screen-space error, tau.

  (found on page 7 just above the pseudo-code). Is this describing the maximum desireable distance in pixels between each vertex before the chunk needs to be subdivided?

And one last question about your implementation. For extremely large planets (ie, modeling Earth), have you run into issues when you get close to the surface with floating-point error? If so, how have you resolved them?

Thanks again for the help!

[JohnJ]

delta represents the maximum geometric deviation of a chunk (in object space) from the portion of the underlying full-detail mesh it represents.

Hmm.. it's hard to explain what "maximum geometric deviation" is without diagrams, but it's basically the greatest distance at any point that the surface of the high res mesh "deviates" spatially from the lower res mesh. For example, if you used a cube as a LOD for a sphere, there would be a large geometric deviation where the cube's surface is farthest from the sphere's surface.

You actually don't need to have a perfectly accurate geometric deviation though. I use an approximation method. Basically, assume "geometric deviation" to mean: the length (in world units) of the maximum geometric "error" that this LOD level has. If it's a really low-res terrain chunk for example, it might omit a hilly peak 1.5 units high. This would mean your LOD has a 1.5 geometric deviation at least.

We start by choosing a value for our maximum tolerable screen-space error, tau.

This value simply represents the maximum "pixel error" (in screen space) you would like. For example, in the above example where a low LOD omits a 1.5-unit peak, it shouldn't be drawn unless 1.5-units at it's current distance from the camera is less than or equal to the maximum tolerable screen-space pixel error. For example if the 1.5-unit tall peak is calculated to take up 6 pixels on-screen, and you don't want a pixel error more than 5 pixels, this means that lower LOD won't be drawn until it moves far enough away that it takes 5 pixels or less on-screen.

And one last question about your implementation. For extremely large planets (ie, modeling Earth), have you run into issues when you get close to the surface with floating-point error? If so, how have you resolved them?

So far the planets I'm using are relatively small enough that I don't have any real floating-point errors, as long as they're centered at (0,0,0), but for very very large planets, you'll probably have to make a dynamic floating origin system where rather than moving the camera around in a scene, you move the scene around the camera (although it should be implemented somewhat transparently if possible).

[Moohasha]

Okay, I think I get it. Thanks.

One last question (for now ). The paper on Chunked LOD assumed that you can pass specific geometry that you want rendered to the rendering pipeline, but I don't know how to do that in Ogre. How are you rendering only the correct LOD and not the others? Are you doing a recursive setVisible() call on them (which doesn't seem efficient), or something else?

Again, thank you for all the help.

[JohnJ]

One last question (for now ). The paper on Chunked LOD assumed that you can pass specific geometry that you want rendered to the rendering pipeline, but I don't know how to do that in Ogre. How are you rendering only the correct LOD and not the others? Are you doing a recursive setVisible() call on them (which doesn't seem efficient), or something else?

There are actually many different levels you can implement this in Ogre, but the easiest (less hacky, like setVisible() calls would be) and most effecient way to do is to manually add chunks to the render queue as desired.

Here's how I did it: I created a Planet class which derived from Ogre::MovableObject. This allows my Planet to be attached to a scene node and to be rendered by Ogre. When Ogre wants to render my MovableObject, it calls the _updateRenderQueue(queue) (passing a RenderQueue object). All the Planet class then needs to do is add Renderable objects as desired to the RenderQueue, and they'll be drawn by Ogre.

These "Renderable's" are my "ChunkNode" objects, which derive from Ogre::Renderable, therefore allowing them to be added to Ogre render queues and rendered. A Renderable basically contains a vertex/index buffer (that defines the polygonal structure) and a material. It's Ogre's most basic renderable element, and gives you absolute and full control.

Of course, this system means you have to do culling, etc. yourself, since only scene nodes are culled by Ogre (in which case only the whole planet would be culled), but it also allows you to potentially have a really super-efficient rendering "engine" behind your planets because you do have control how everything is rendered.

[Moohasha]

Ah, an excellent idea. I'll see if I can pull off something similar. You have been extremely helpful, and I thank you very much!!

[Moohasha]

Ok, nope. I'm still lost. I looked at the Ogre::Renderable object and I didn't see anything about vertex/index buffers, and the documentation didn't explain too much about it, so I made each of my chunks ManualObjects instead. The problem I'm actually having is figuring out how to move each chunk. If I attach each one to the SceneNode, they all get rendered. If I don't attach them, but only add the ones I want to render to the RenderQueue, I lose the ability to transform them since transformations are done through SceneNodes. So how can I transform my chunks so that they act like they're attached to the planet's SceneNode, but still have the flexibility of adding only the chunks I want to the RenderQueue?
Sorry for being such a n00b, but I'm still learning all of this.

[JohnJ]

I looked at the Ogre::Renderable object and I didn't see anything about vertex/index buffers

There should be a virtual function "getRenderOperation()", which you override to supply the vertex and index data.

so I made each of my chunks ManualObjects instead

That'll work, but you should know that of all the methods you can use to achieve this in Ogre, this is the slowest.

I lose the ability to transform them since transformations are done through SceneNodes

Actually Ogre's Renderable class has "getWorldOrientation()" and "getWorldPosition()" functions which you can override. But I don't even use them - when the terrain chunk meshes are generated, there's no need to transform anything because the vertexes simply go where they're supposed to. As to moving/rotating planets, that's handled pretty much automatically through the scene node I attach the MovableObject to.

P.S. I think I learned most of this by looking at Ogre source code for things like Entity and StaticGeoemtry. I can understand that it would be nearly impossible to learn this from the documentation. Since there aren't any tutorials (that I know of), learning by example is the next best thing.

[Moohasha]

Sorry to keep bothering you, and if you'd like I'll create a new thread for my problems so I don't hijack yours, but I'm still having issues.

Actually Ogre's Renderable class has "getWorldOrientation()" and "getWorldPosition()" functions which you can override. But I don't even use them - when the terrain chunk meshes are generated, there's no need to transform anything because the vertexes simply go where they're supposed to. As to moving/rotating planets, that's handled pretty much automatically through the scene node I attach the MovableObject to.

I'm still missing this. I rewrote my chunks to derive from Ogre::Renderable and created/filled the vertex buffers and all that, and it renders fine as long as I don't move anything, but I still don't see how to move/rotate the Renderables with the MoveableObject. You can't attach Renderables to a SceneNode, so when the SceneNode gets transformed, how do those transformations get passed along to the Renderables? Again, sorry for being such a pain, but I really do appreciate the help.

[Moohasha]

Ok, I got it to work by adding the following code:

Code: Select all

Ogre::Matrix4 xform;
getParentSceneNode()->getWorldTransforms(&xform);
chunk->setWorldTransform(xform);

Where chunk is an instance of LodChunk which derives from Renderable. Not sure if that's the best way to go, but it works and I'm happy. Thanks again for the help, and hope your game works out!!