[GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellation

[holocronweaver]

Are you using my GSoC BitBucket repo? As masterfalcon mentioned, my work has not yet been merged into mainline so for now if you want to use the latest GL3+ you will need to obtain OGRE from my GSoC repo. All my work is in the v2-0 branch of that repo. This coming weekend I will be merging all my completed work into that repo, so hopefully allot of bugs will be fixed then.

[Ident]

I wasn't using it. I didn't know what masterfalcon meant because i m not familiar with the term "mainline" i assumed it was the "default" branch. I will clone your repo and build it in a bit and report back.

[Ident]

The tesselation demo works fine now in GL3 Renderer.

I noticed the previously described lockup in the application still exists, but it only occured to me after i loaded the Volume terrain demo.
If it matters - That same demo is bugged for me btw. it shows the surfaces in a pure white when on a certain distance to it. I dont know if it is related to your branch, I havent tested this demo in other branches.


I m planning to start integrating the Ogre version containing your GL3 Renderer work into my application once you are done finishing it and merged it into the official ogre3d repo - I subscribed this thread so please post here when u are finished, i dunno where else to check for when it is done.
From then on it might take some days until i got everything set up, but I will inform you about my progress or any issues i might encounter. My tesselation shader usage is pretty simple anyways, i just need to tesselate polygons depending on their estimated NDC space size.

[uzik]

Did khr_debug make it into your project by any chance?

Not yet, but it is on my TODO list. Getting the error logs will be easy, but how to expose the markers and groups to the user via OGRE is not yet obvious to me because I am not sure how this compares to DirectX.

I did some checking and found that none of the cards I own support it. Some do however support the previous versions of logging.
So the ideal solution might be to support those as well.

Here's some code that might help speed things along. It will need some small tweeks:

definitions ( you probably don't need these with gl3w ) :

Code: Select all

      // define an ARB callback
      typedef void ( APIENTRY* GLDEBUGPROCARB )( GLenum source, GLenum type, GLuint id, GLenum severity, GLsizei length, const GLchar* message, GLvoid* userParam );
      // function to set callback
      typedef void ( APIENTRY* GLDEBUGMESSAGECALLBACKARB ) ( GLDEBUGPROCARB callback, void* userParam );
      GLDEBUGMESSAGECALLBACKARB glDebugMessageCallbackARB;

      // define an AMD callback
      typedef void ( APIENTRY* GLDEBUGPROCAMD )( GLuint id, GLenum category, GLenum severity, GLsizei length, const GLchar* message, GLvoid* userParam );
      // function to set callback
      typedef void ( APIENTRY* GLDEBUGMESSAGECALLBACKAMD ) ( GLDEBUGPROCAMD callback, void* userParam );
      GLDEBUGMESSAGECALLBACKAMD glDebugMessageCallbackAMD;

      static void CALLBACK AMDDebugCallback( GLuint id, GLenum category, GLenum severity, GLsizei length, const GLchar* message, GLvoid* userParam );
      static void CALLBACK ARBDebugCallback( GLenum source, GLenum type, GLuint id, GLenum severity, GLsizei length, const GLchar* message, GLvoid* userParam );

Setting up the callbacks:

Code: Select all

         // See if the AMD version is available (the 2010 version)
         if ( mContext->hasExtension( QByteArrayLiteral( "GL_AMD_debug_output" ) ) )
         {
            glDebugMessageCallbackAMD  = (GLDEBUGMESSAGECALLBACKAMD) wglGetProcAddress( "glDebugMessageCallbackAMD" );
            if ( glDebugMessageCallbackAMD )
               // set callback function to receive messages
               glDebugMessageCallbackAMD( AMDDebugCallback, 0 );
            else
               LogManager::getSingleton().logMessage( "glDebugMessageCallbackAMD not found. OpenGL AMD debug will not function" );
         }
         else
            // See if the ARB version is available (the pre 2010 version)
            if ( mContext->hasExtension( QByteArrayLiteral( "GL_ARB_debug_output" ) ) )
            {
               glDebugMessageCallbackARB  = (GLDEBUGMESSAGECALLBACKARB) wglGetProcAddress( "glDebugMessageCallbackARB" );
               if ( glDebugMessageCallbackARB )
                  // set callback function to receive messages
                  glDebugMessageCallbackARB( ARBDebugCallback, 0 );
               else
                  LogManager::getSingleton().logMessage( "glDebugMessageCallbackARB not found. OpenGL ARB debug will not function" );
            }

and the callback functions:

Code: Select all

   void ARBDebugCallback( GLenum source, GLenum type, GLuint id, GLenum severity, GLsizei length, const GLchar* message, GLvoid* userParam )
   {
      String msg( "OpenGL message: " );

      // todo: determine if this is needed
      msg.append( " source: " );
      const uint32 DEBUG_SOURCE_API_ARB = 0x8246;
      const uint32 DEBUG_SOURCE_WINDOW_SYSTEM_ARB = 0x8247;
      const uint32 DEBUG_SOURCE_SHADER_COMPILER_ARB = 0x8248;
      const uint32 DEBUG_SOURCE_THIRD_PARTY_ARB = 0x8249;
      const uint32 DEBUG_SOURCE_APPLICATION_ARB = 0x824A;
      const uint32 DEBUG_SOURCE_OTHER_ARB = 0x824B;
      switch ( source )
      {
         case DEBUG_SOURCE_API_ARB:
            msg.append( "API" );
            break;
         case DEBUG_SOURCE_WINDOW_SYSTEM_ARB:
            msg.append( "window system" );
            break;
         case DEBUG_SOURCE_SHADER_COMPILER_ARB:
            msg.append( "shader compiler" );
            break;
         case DEBUG_SOURCE_THIRD_PARTY_ARB:
            msg.append( "third party" );
            break;
         case DEBUG_SOURCE_APPLICATION_ARB:
            msg.append( "application" );
            break;
         case DEBUG_SOURCE_OTHER_ARB:
            msg.append( "other" );
            break;
         default:
            msg.append( "UNKNOWN" );
            break;
      }

      // todo: determine if this is needed
      msg.append( " type: " );
      const uint32 DEBUG_TYPE_ERROR_ARB = 0x824C;
      const uint32 DEBUG_TYPE_DEPRECATED_BEHAVIOR_ARB = 0x824D;
      const uint32 DEBUG_TYPE_UNDEFINED_BEHAVIOR_ARB = 0x824E;
      const uint32 DEBUG_TYPE_PORTABILITY_ARB = 0x824F;
      const uint32 DEBUG_TYPE_PERFORMANCE_ARB = 0x8250;
      const uint32 DEBUG_TYPE_OTHER_ARB = 0x8251;
      switch ( type )
      {
         case DEBUG_TYPE_ERROR_ARB:
            msg.append( "ERROR" );
            break;
         case DEBUG_TYPE_DEPRECATED_BEHAVIOR_ARB:
            msg.append( "deprecated" );
            break;
         case DEBUG_TYPE_UNDEFINED_BEHAVIOR_ARB:
            msg.append( "undefined behavior" );
            break;
         case DEBUG_TYPE_PORTABILITY_ARB:
            msg.append( "portability" );
            break;
         case DEBUG_TYPE_PERFORMANCE_ARB:
            msg.append( "performance" );
            break;
         case DEBUG_TYPE_OTHER_ARB:
            msg.append( "other" );
            break;
         default:
            msg.append( " UNKNOWN" );
            break;
      }

      // todo: determine if this is needed
      msg.append( " Id: " );
      msg.append( StringConverter::toString( id ) );

      // todo: determine if this is needed
      msg.append( " severity: " );
      const uint32 DEBUG_SEVERITY_HIGH_ARB = 0x9146;
      const uint32 DEBUG_SEVERITY_MEDIUM_ARB = 0x9147;
      const uint32 DEBUG_SEVERITY_LOW_ARB = 0x9148;
      switch ( severity )
      {
         case DEBUG_SEVERITY_HIGH_ARB:
            msg.append( "HIGH" );
            break;
         case DEBUG_SEVERITY_MEDIUM_ARB:
            msg.append( "Medium" );
            break;
         case DEBUG_SEVERITY_LOW_ARB:
            msg.append( "Low" );
            break;
         default:
            msg.append( "UNKNOWN" );
            break;
      }

      msg.append( " Message: " );
      msg.append( message, length );

      LogManager::getSingleton().logMessage( msg.c_str() );
   }

   void AMDDebugCallback( GLuint id, GLenum category, GLenum severity, GLsizei length, const GLchar* message, GLvoid* userParam )
   {
      String msg( "OpenGL message: " );

      // todo: determine if this is needed
      msg.append( " Category: " );
      msg.append( StringConverter::toString( category ) );

      // todo: determine if this is needed
      msg.append( " severity: " );
      const uint32 DEBUG_SEVERITY_HIGH_AMD = 0x9146;
      const uint32 DEBUG_SEVERITY_MEDIUM_AMD = 0x9147;
      const uint32 DEBUG_SEVERITY_LOW_AMD = 0x9148;
      switch ( severity )
      {
         case DEBUG_SEVERITY_HIGH_AMD:
            msg.append( " HIGH" );
            break;
         case DEBUG_SEVERITY_MEDIUM_AMD:
            msg.append( " Medium" );
            break;
         case DEBUG_SEVERITY_LOW_AMD:
            msg.append( " Low" );
            break;
         default:
            msg.append( " UNKNOWN" );
            break;
      }

      msg.append( " Message: " );
      msg.append( message, length );

      LogManager::getSingleton().logMessage( msg.c_str() );
   }

[uzik]

Did khr_debug make it into your project by any chance?

Not yet, but it is on my TODO list. Getting the error logs will be easy, but how to expose the markers and groups to the user via OGRE is not yet obvious to me because I am not sure how this compares to DirectX.

I was thinking the filter options could be added to the configuration so users could turn it on/off, set severity levels, etc.
Just having it might be of great value while debugging code though. (If it's as nice as it sounds)

Thanks for all the good work

[Ident]

Are you using my GSoC BitBucket repo? As masterfalcon mentioned, my work has not yet been merged into mainline so for now if you want to use the latest GL3+ you will need to obtain OGRE from my GSoC repo. All my work is in the v2-0 branch of that repo. This coming weekend I will be merging all my completed work into that repo, so hopefully allot of bugs will be fixed then.

I just checked the commits to the main repo, the merge has not happened yet, has it?

[holocronweaver]

No, I have not merged yet. I mentioned this on Twitter, but I forget that not everyone lives in the Twitterverse. I pushed the merge back to the coming weekend because work unexpectedly ate up most of this past weekend. This should not happen again for a few weeks, so the next few weekends will be OGRE GL3+ from alpha to beta.

My ETA timeline:

• GL3+ RS alpha - this weekend
• GL3+ RS beta (95% feature complete, including material script access to new features) - last weekend of November
• Samples complete - mid Decemeber

If the beta compiles and runs well, I will make a merge request for OGRE main. While all the samples should be done by the end of December, two are very close to complete and should be included with the November beta. Of course, my goal is to get all of this done ASAP, so things may come sooner. I will do everything in my power to ensure these deadlines do not slip.

(Warning: I did a small push last night by accident. Not sure if the repo compiles in its current state. Will try to fix this tonight and will report back.)

[Ident]

Okay cool, thanks for the info.

[holocronweaver]

My GL3+ repo is in a good state after all. Using MinGW to build a fresh clone on Windows 7 x64 + Catalyst 13.11 beta 6 drivers, I was able to run everything on my laptop just as well as on my Linux desktop.

[holocronweaver]

Anyone out there know the RTSS well enough to help me get it working with GL3+? Has any work been done on this recently upstream? This is the only thing holding me back from fixing the visual unit tests, but I am not familiar with the RTSS internals so I cannot explain why all RTSS dependent tests produce black output. If someone could point me in the right direction I should be able to do the rest. (I am guessing it is a lighting calculation issue in the RTSS generated fragment shader? Or perhaps a projection matrix issue in the vertex shader?)

[masterfalcon]

Yeah, what exactly is the problem? Maybe I need to update and try it out.

[holocronweaver]

Yeah, what exactly is the problem? Maybe I need to update and try it out.

Essentially all visual tests which depend on RTSS render as pure black in the GL3+ RS, but they render fine in the GL RS. Since no GL errors are reported, I am guessing this is a problem with how RTSS is creating the shaders.

Right now I am focusing on getting PlayPen_BasicPlane to display something other than pure black. It uses the Examples/RustySteel material, which is just a pass with a texture unit (no shaders). If I switch the material to BaseWhiteNoLighting or even BaseWhite I can see the plane, so perhaps the issue is how RTSS does texturing?

[masterfalcon]

I swear I had this working at one point. Even for ES 2. But I'll keep looking to see what's going on. The good news is that I can also reproduce the issue.

[holocronweaver]

I swear I had this working at one point. Even for ES 2. But I'll keep looking to see what's going on. The good news is that I can also reproduce the issue.

While you take a look at that, I will try to figure why even the non-RTSS tests have problems in GL3+ but not GL.

Update: I fixed PlayPen_16Textures simply by adding the missing "in vec4 ambientColour;" to the fragment shader written in PlayPenTests.cpp. Perhaps not all vertex output parameters are being inputted in the RTSS fragment shaders? No idea.

[masterfalcon]

Fixed and changes pushed! Let me know how it goes.

[holocronweaver]

Thanks! I am still getting the same black visual test results on Linux as before though. Are you getting reasonable results on Mac? Also, are visual tests integrated into the automated build system?

Update: I am pushing back the GL3+ alpha a day or two while we finish sorting out the visual test / RTSS issues. Might even squeeze in an extra GL3+ feature while I am at it.

[masterfalcon]

I haven't done a full pass of the tests yet. But they are looking somewhat good so far. Even PlayPen_BasicPlane doesn't work for you? Visual tests are not yet integrated into the build system.

[holocronweaver]

Nope, even PlayPen_BasicPlane is pure black. I am guessing this is a GLSL strictness difference between our drivers. These Linux AMD drivers can be pretty anal retentive.

[masterfalcon]

That's mighty strange. Especially that no GLSL errors are emitted. I should have noted too that I am running only at GL 3.3(my macbook pro is kinda old). I'll also try on my iMac tonight, I think it'll do GL 4.1.

[holocronweaver]

It is worth noting that when I fixed PlayPen_16Textures it was not giving any GLSL errors, but it was indeed the shaders that needed to be modified. Could something be muting warnings?

Update: Ok, TransparencyTest fails, but its setup is almost identical to the equivalent sample in Sample Browser, which works fine. So this appears to simply be an RTSS initialization issue.

[masterfalcon]

Ok. In my case, USE_RTSHADER_SYSTEM or whatever, wasn't defined soon enough so addSceneManager for the current test wasn't being called. I thought I'd solved it but I guess not. Also, OS X ends up using a slightly different run loop than linux, and I know that everything gets initialized right when using OS X.

[holocronweaver]

Can I trust the RTSS initialization used in the Ogre wiki to guide me in the right direction?

[masterfalcon]

Yeah you probably can. The initialization that's in the test app is actually a clone of what is being used in the sample browser. So it *should* work. Uncomment the write shaders to disk define to make sure that the shaders are actually being created and I'd put in a #error in Sample.h createSceneManager before the mShaderGenerator->addSceneManager(mSceneMgr); line to make sure that it's being compiled right.

[Ident]

In "Tesselation.material" if i change the name of the variable "g_fTessellationFactor" to an arbitrary other name that i also use in the "TesselationTd.glsl" file, the triangle isn't rendered anymore. If i leave the variable and remove the whole code that sets the tesselation factors, it will still render as if the code was there. What the heck! Likewise, if i completely remove the shader program file, the demo won't run - so it is definitely using this file and I am not in the wrong directory or something! At the same time it doesn't care what changes i apply to the shader program file - it only cares about changes to the variable's name inside the material OR shader file, but as long as the variable is there it always runs the same way. Changing the value for the variable works as expected. However replacing it with a fixed value in the shader program doesn't show any effect. I'm baffled. Is my driver caching something? I couldnt find references to this variable in Ogre and didn't expect any to exist - I mean the only logical explanation to me now is that Ogre somehow accesses this variable under its given name.

[scrawl]

Check that it's not using compiled shader cache. Do you have a ~/.ogre/<Version>/cache.bin file?