New vertex data types and Improved terrain plugin

[dvdjg]

I wrote a lot of changes to the Ogre core:

http://atc1.aut.uah.es/~infind/Programm ... hanges.pdf

I've submit a patch. It is the first time, so I hope everything goes right.

Here you can download a complete FAT package og 58 MB, with a copy of my development Ogre core (only if you are very very curious).

http://atc1.aut.uah.es/~infind/Programm ... .6.djg.rar

I've just submit other patches to the DevIL project, because I found some bugs:

http://sourceforge.net/tracker/index.ph ... tid=304470
http://sourceforge.net/tracker/index.ph ... tid=104470

Some DevIL needed changes:
http://atc1.aut.uah.es/~infind/Programm ... dified.zip
http://atc1.aut.uah.es/~infind/Programm ... hanges.pdf

[Falagard]

Wow! I read through most of the doc and it sounds like you have some great optimizations and features that you've added.

I'm sure Sinbad or someone else from the core team will respond and either agree with your changes and apply the patch, or disagree and explain why, but I just wanted to say nice work either way! Seems to me that most or all of your changes are for the better of Ogre in general.

Clay

[WhiteKnight]

Wow. You've been quite busy . Thank you for taking the time to document your changes. I'm very interested to see what the response will be.

[Game_Ender]

Hopefully all the changes being against 1.0.6 (as far as I can tell won't be a problem). Have you compared you changes vs the current Dagon change list?

[sinbad]

Thanks for the sizeable patch. Depending on how much time we have, and whether the patches are quick to adapt to Dagon (they won't get applied to Azathoth since it's maintenance-only), this will either go into Dagon or Eihort (the next dev version once Dagon is released). It looks like useful stuff so I'd like to say it'll go into Dagon, but we already have a heap of work for that so at some point we have to draw a line and get the thing out the door.

[tuan kuranes]

Nice work there.
About how to reduce terrain VBO, here's what's done in PLSM2 ograddon :
- Position is one VET_SHORT2 (converted to a single float in a vertex shader), representing y only (height). x and z are computed using texture coordinates in a vertex shader.
- Texture coordinates are shared between pages. So only one set for all pages.
So each pages added would just add a float per vertex. (plus normal/morph if needed.)
Using new vertex types, if they do work on both ATI and NVidia, on both D3D and OpenGL, will sure would help reduce it even more.

[genva]

Thanks contribute the patch. I'm look through the patch, some of them conflict with Dagon heavily (e.g. named auto constant of GpuProgramParameters), we'll very happy if you can adapt to Dagon .

Some code dealing with very generalise vertext data type, I'm afraid about the performance problem, since the data need to convert from one type to Real, and then convert back again, especially there are plan to support SSE/SSE2 optimise in Eihort.

[dvdjg]

Thanks for your attention.

Have you compared you changes vs the current Dagon change list?

Hummmm, this issssss, thaaattt. . . no :-$
But I’m going to do it

Nice work there.
About how to reduce terrain VBO, here's what's done in PLSM2 ograddon :
- Position is one VET_SHORT2 (converted to a single float in a vertex shader), representing y only (height). x and z are computed using texture coordinates in a vertex shader.
- Texture coordinates are shared between pages. So only one set for all pages.
So each pages added would just add a float per vertex. (plus normal/morph if needed.)
Using new vertex types, if they do work on both ATI and NVidia, on both D3D and OpenGL, will sure would help reduce it even more.

I only use one VET_SHORT3N (or VET_USHORT3N if supported) with the semantic of VES_POSITION. The y coord stores the height data, and the x and z coords stores the texture coordinates. So there is no need of an additional vertex element for texture coordinates. Even if you force the situation and run under DX9 with a Radeon 9500+, you can store the position (including textures coords) in only one VET_DEC3N (or VET_UDEC3) witch means that you only need 4 bytes per vertex!!! It is very difficult to give a better situation from the geometry compression viewpoint. But it is only of interest in the one page terrain world.
Everyway I was thinking of complementing this solution in a similar way as they use for when using multiple pages.

some of them conflict with Dagon heavily (e.g. named auto constant of GpuProgramParameters)

Oh please forget the named auto constants changes I did. I saw that GpuProgramParameters needs a complete redesign. It was only a workaround for using parameters in the nVidia fp30 and fp40 profiles.
GpuProgramParameters was designed with the [erroneous] supposition of that all the named parameters can be converted to indexed parameters before the compilation, and that after compiling you'll have an API that allows you to access them indexing. The first is not true with GLSL and the second isn't with nVidia fragment programs. So I didn't continue implementing them. There are also some other changes I didn't document because them were experimental for me.
A thing I really forgot to document (and is more or less definitive) was the support for automatic texture format conversion as I post in other thread.

I'm afraid about the performance problem, since the data need to convert from one type to Real, and then convert back again

I'm conscientious of the performance problem, but don’t think that "data need to convert from one type to Real, and then convert back again". Actually data must be converted from a float to a compact representation or from a compact representation to float but not both things.
It is probably you say that because you have seen the VertexData::readElementArray and the VertexData::writeElementArray member functions. These are only the highest level interface I did, but I recommend using them only if you don’t like the HardwareVertexBuffer::ElementArray witch is the recommended way. HardwareVertexBuffer::ElementArray is the easiest and more versatile way of accessing vertex elements. It will work as fast as the conversion is, and I think it is a better solution than directly locking and unlocking vertex buffers.
With HardwareVertexBuffer::ElementArray you don’t have to worry about internal issues, and you don’t need to be care of locking unlocking Hardware Buffers. It will lock and unlock as necessary. Even if an exception is thrown this class will unlock all the locked buffers.
Everyway there is no need of removing the older interface. The three can coexist everyone with its advantages and its troubles. (Everyway I think HardwareVertexBuffer::ElementArray is the better .)
In the other hand Ogre uses the Real type in places where it mustn’t. It is a thing that should be corrected in a future version.
I’m continuing changing things:
Yesterday I was creasy trying to debug an OpenGL error with an ATI 9000. I populate the opengl routines with traces to see what the function that caused the error was, and I finally found:

Code: Select all

    void GLRenderSystem::setStencilBufferParams(CompareFunction func, 
        uint32 refValue, uint32 mask, StencilOperation stencilFailOp, 
        StencilOperation depthFailOp, StencilOperation passOp, 
        bool twoSidedOperation)
    {
        bool flip;

        if (twoSidedOperation)
        {
            if (!mCapabilities->hasCapability(RSC_TWO_SIDED_STENCIL))
                OGRE_EXCEPT(Exception::ERR_INVALIDPARAMS, "2-sided stencils are not supported",
                    "GLRenderSystem::setStencilBufferParams");
            glEnable(GL_STENCIL_TEST_TWO_SIDE_EXT);
            // NB: We should always treat CCW as front face for consistent with default
            // culling mode. Therefore, we must take care with two-sided stencil settings.
            flip = (mInvertVertexWinding && !mActiveRenderTarget->requiresTextureFlipping()) ||
                   (!mInvertVertexWinding && mActiveRenderTarget->requiresTextureFlipping());

            // Set alternative versions of ops
            glActiveStencilFaceEXT_ptr(GL_BACK);
            glStencilMask(mask);
            glStencilFunc(convertCompareFunction(func), refValue, mask);
            glStencilOp(
                convertStencilOp(stencilFailOp, !flip), 
                convertStencilOp(depthFailOp, !flip), 
                convertStencilOp(passOp, !flip));
            // reset
            glActiveStencilFaceEXT_ptr(GL_FRONT);
        }
        else
        {
		[color=red]if (mCapabilities->hasCapability(RSC_TWO_SIDED_STENCIL))[/color]
			glDisable(GL_STENCIL_TEST_TWO_SIDE_EXT);
            flip = false;
        }

        glStencilMask(mask);
        glStencilFunc(convertCompareFunction(func), refValue, mask);

        glStencilOp(
            convertStencilOp(stencilFailOp, flip),
            convertStencilOp(depthFailOp, flip), 
            convertStencilOp(passOp, flip));

    }

Please add the red marked line because it generates a quiet error in those cards that don’t support it.


For the spotlights to be completely usable the GL_SPOT_EXPONENT parameter must be set, isn’t it? (I’m not sure if it is not used with any intention).

Code: Select all

    void GLRenderSystem::setGLLight(size_t index, Light* lt)
    {
        GLenum gl_index = GL_LIGHT0 + index;

        if (!lt)
        {
            // Disable in the scene
            glDisable(gl_index);
        }
        else
        {
            switch (lt->getType())
            {
            case Light::LT_SPOTLIGHT:
                glLightf( gl_index, GL_SPOT_CUTOFF, 0.5f * lt->getSpotlightOuterAngle().valueDegrees() );
				// djg 1 In OGL there no exist the inner angle concept ;(
	            [color=red]glLightf( gl_index, GL_SPOT_EXPONENT, lt->getSpotlightFalloff()); [/color]
                break;
            default:
                glLightf( gl_index, GL_SPOT_CUTOFF, 180.0 );
            }
	. . .

For a future HDR support I added 3 new capabilities:

Code: Select all

        /// Supports source 2D and cube float textures, and single float render targets
        RSC_TEXTURE_FLOAT           = CAPABILITY(0, 10),
        /// Supports G16R16F multiple (2) render targets
        RSC_TEXTURE_FLOAT_MRT       = CAPABILITY(0, 11),
        /// Supports floating point texture blending
        RSC_TEXTURE_FLOAT_BLENDING	= CAPABILITY(0, 12), 
        /// Supports floating point texture filtering (other than point)
        RSC_TEXTURE_FLOAT_FILTERING	= CAPABILITY(0, 13),

In RenderSystemCapabilities::log:

Code: Select all

        pLog->logMessage(
            " * Arbitrary floating point texture filtering: " 
            + StringConverter::toString(hasCapability(RSC_TEXTURE_FLOAT_FILTERING), true));
        pLog->logMessage( 
            " * Floating point texture blending: " 
            + StringConverter::toString(hasCapability(RSC_TEXTURE_FLOAT_BLENDING), true));
        pLog->logMessage(
            " * Multiple render target floating point textures: " 
            + StringConverter::toString(hasCapability(RSC_TEXTURE_FLOAT_MRT), true));

When I have HDR code enough I’ll send other patch.

The code I put in the patch for supporting the new data types in OGL should be into the GL_ARB_fragment_program block. So only Radeon 9500+ support them. (Really the 8500+ have hardware to support these data types, but it is not enabled in the drivers).

Code: Select all

if (mGLSupport->checkExtension("GL_ARB_fragment_program"))
{

  if (mGLSupport->checkExtension("GL_ATI_fragment_shader"))
  {
	// At least ATI Radeon 9500+ supports these types
	mCapabilities->setCapability(RSC_VERTEX_FORMAT_BYTE4); // Or not?
	mCapabilities->setCapability(RSC_VERTEX_FORMAT_USHORTN);

	// It could be good if ATI supports the DirectX 3-tuple vertex types UDEC3 and DEC3N
	//  in their OpenGL drivers using a UNSIGNED_INT_10_10_10_2 vertex type or similar, 
	//  but they doesn't.
  } 
}

PD: I've just seen that MRT is to be included in Dagon. I'll stop working in a HDR app till then. (MRT is not necessary for HDR but if you use it the performance boost can grow up 30%).

[sinbad]

MRT is already in Dagon, and wumpus has done a deferred shading demo which uses it.