[GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellation

Threads related to Google Summer of Code
Post Reply
User avatar
masterfalcon
OGRE Team Member
OGRE Team Member
Posts: 4270
Joined: Sun Feb 25, 2007 4:56 am
Location: Bloomington, MN
x 126
Contact:

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by masterfalcon »

holocronweaver wrote:What is the status of the new shaders (tessellation, geometry, and compute)? Do any of these shaders have a functioning example? From what I can tell only the tessellation shader has been used in Samples, but no vertices are visible. I am currently looking into this.
There are two samples that use tessellation shaders, the other is the PNTriangles sample. But I have not worked on GLSL support for it at all. I'm not positive that I ported the shaders to GLSL properly either so tessellation may be working if we have the shaders correct.

The ParticleGS sample uses geometry shaders AND transform feedback. Transform feedback is definitely not working and needs some major work and I'm not sure if geometry shaders are working though again, they could be.

Compute shaders aren't highlighted anywhere in the samples at the moment.
holocronweaver wrote:What is the best mercurial workflow? I want to avoid any messy VCS fiascos if possible. Is the Ogre developers guide still best practice, with rebasing upstream pulls to keep a single commit timeline?
The dev guide is still the best way to go.
holocronweaver wrote:What is the best means of debugging GLSL shaders and OpenGL render system code across platforms? I typically rely on a mix of valgrind, GNU debugger and print statements for debugging C++, but I have never found a good multi-platform solution for debugging GLSL shaders. I am considering trying AMD CodeXL. I primarily develop on Linux using an AMD card, but am willing to try other systems if it will make debugging easier.
CodeXL looks great. It's the latest incarnation of gDEBugger after AMD bought it. NVIDIA PerfHUD may be helpful as well. CodeXL will at least let you edit and recompile shaders at runtime which is extremely helpful.

I've found apitrace https://github.com/apitrace/apitrace to be helpful sometimes too.
TheSHEEEP
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 972
Joined: Mon Jun 02, 2008 6:52 pm
Location: Berlin
x 65

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by TheSHEEEP »

If you encounter problems, I can recommend glIntercept.
It can basically log everything OpenGL does and will tell about any encountered errors. Also, it an be configured quite a bit.
My site! - Have a look :)
Also on Twitter - extra fluffy
User avatar
Lee04
Minaton
Posts: 945
Joined: Mon Jul 05, 2004 4:06 pm
Location: Sweden
x 1

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by Lee04 »

inmutable sate objects and casting in shaders are two DirectX11 features that OpenGL has ONE extension for.

At GTC a couple a years ago, I asked for this openGL extension, because these two are needed to make a real wrapper between DirectX11 and OpenGL3++.

I asked that we would get this extension and we got it. I asked because I had Ogre in mind.

So DirectX11/OpenGL3++ wrapper would always use inmutable state objects and casting in shaders. To run at proper speeds on DirectX11 class hardware,

Without it It will run slower than a DirectX9 version.
Ph.D. student in game development
User avatar
masterfalcon
OGRE Team Member
OGRE Team Member
Posts: 4270
Joined: Sun Feb 25, 2007 4:56 am
Location: Bloomington, MN
x 126
Contact:

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by masterfalcon »

Which extension is that?
User avatar
holocronweaver
Google Summer of Code Student
Google Summer of Code Student
Posts: 273
Joined: Mon Oct 29, 2012 8:52 pm
Location: Princeton, NJ
x 47

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by holocronweaver »

I have pushed my first commits to the GL3+ GSoC fork. I have been focusing mostly on debugging the tessellation shaders. I made a pure C++ & OpenGL framework for comparison and the shaders ran fine, but they failed in Ogre. After some fiddling, I discovered that the gl_Position being passed to the tessellation shaders was somehow becoming malformed. If I applied the WVP matrix twice, once in the vertex shader and once in the tessellation evaluation shader, I could see a mirror image of the tessellated Ogre head which stayed forever in front of the camera no matter where I turned. It was like having an Ogre head flashlight.

Figuring out why gl_Position was malformed took me on a little journey through the GL3+ render system and ultimately led me to suspect compilation and linking to be the cause. I tried reproducing the same error in my pure OpenGL framework by copying the exact same OpenGL shader compilation and linking calls made by OGRE. This was slow and tedious; I made little progress.

Ultimately the 'solution' came when Dave (masterfalcon) pushed a commit today which switched me to using separable shader objects via OgreGLSLProgramPipeline. For reasons I do not yet understand, this corrected the gl_Position and the tessellated Ogre head finally appeared in the right location and stayed there (though he does not yet have point/wireframe/solid modes yet, only solid, so I have not verified the effects of tessellation yet). So my suspicion that the gl_Position was being malformed by compilation/linking seems correct, though I still do not know why it was occurring for non-separable shader objects. Tracing GL calls did not reveal anything out of the ordinary. (BTW, several of the samples, such as Deferred Lighting, are now almost fully functional for me on Linux thanks to the switch to separable programs)

So my first progress update comes to you thanks to serendipity. :oops:

Oddly I still cannot see the manually created triangle in the tessellation sample, but I can see a full Ogre head mesh. I will investigate this tomorrow, along with hopefully wrapping up my work on getting the geometry shader to play nice. I think I will be able to catch up with my schedule by the end of the week.

As for debugging tools, I tried out CodeXL and was disappointed to learn that it does not support GL4. This forced me to use GPU PerfStudio 2 on Windows for frame debugging, which has been a mixed blessing. The program is fantastic...when it works. The ability to decompile and edit shaders in real time is awesome. I can fully analyze almost any aspect of OpenGL or DirectX I could desire, and can even do some pretty nifty profiling. Unfortunately the client/server model PerfStudio relies on is buggy and often fails to connect and/or causes SampleBrowser to crash. That being said, PerfStudio is still the most powerful tool I have in my arsenal.
User avatar
masterfalcon
OGRE Team Member
OGRE Team Member
Posts: 4270
Joined: Sun Feb 25, 2007 4:56 am
Location: Bloomington, MN
x 126
Contact:

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by masterfalcon »

Awesome. I played around with it and have a couple shader compilation issues.

In TesselationTd.glsl line 12 should be:

Code: Select all

    vec3 finalPos = (gl_TessCoord.x * gl_in[0].gl_Position.xyz) + (gl_TessCoord.y * gl_in[1].gl_Position.xyz) + (gl_TessCoord.z * gl_in[2].gl_Position.xyz);
That one is my fault. I should have fixed that error long ago.

Then I get the following error from TesselationTh.glsl:

ERROR: 0:8: Declared size (32) of output interface block array in the tessellation control shader is incompatible with earlier declarations (expected 3)
User avatar
holocronweaver
Google Summer of Code Student
Google Summer of Code Student
Posts: 273
Joined: Mon Oct 29, 2012 8:52 pm
Location: Princeton, NJ
x 47

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by holocronweaver »

Thanks for pointing out the errors. They should be very easy to fix. Time to become familiar with how various OpenGL implementations differ from the opengl.org wiki. :) What system are you using to test (GPU + OS)? I just tested my changes on Windows and Linux on two different machines, but both have AMD cards.
User avatar
masterfalcon
OGRE Team Member
OGRE Team Member
Posts: 4270
Joined: Sun Feb 25, 2007 4:56 am
Location: Bloomington, MN
x 126
Contact:

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by masterfalcon »

In this case it was OS X Mavericks on NVIDIA.
User avatar
holocronweaver
Google Summer of Code Student
Google Summer of Code Student
Posts: 273
Joined: Mon Oct 29, 2012 8:52 pm
Location: Princeton, NJ
x 47

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by holocronweaver »

Wow, that might be my ultimate blind spot system. I should be getting my hands on a functioning Mavericks install today or tomorrow. Anything particular I should know about getting Ogre set up on Mavericks? Are the samples generally working?
User avatar
masterfalcon
OGRE Team Member
OGRE Team Member
Posts: 4270
Joined: Sun Feb 25, 2007 4:56 am
Location: Bloomington, MN
x 126
Contact:

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by masterfalcon »

Yeah at the moment. You might need to set the OpenGL header path in cmake manually. /System/Library/Frameworks/OpenGL.framework/Headers
User avatar
holocronweaver
Google Summer of Code Student
Google Summer of Code Student
Posts: 273
Joined: Mon Oct 29, 2012 8:52 pm
Location: Princeton, NJ
x 47

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by holocronweaver »

The sample browser crashes when switching from GL to GL3+ in Linux, but not the other way around. In my attempts to debug this, I have stumbled upon what may be the first quantum bug ever.

The crash occurs in RenderSystem::shutdown() when it is called by GLRenderSystem::shutdown(). When I step through RenderSystem::shutdown() using gdb, the program goes through GLRenderSystem::shutdown() twice and then successfully relaunches the sample browser for GL3+. At first I thought I had solved the bug, but when I try to run the sample browser normally and switch from GL to GL3+, GLRenderySystem::shutdown() is called once and crashes on GLRenderSystem::shutdown() (which I can confirm via littered print statements).

Thus when I look, the program behaves (almost) normally and all is well. When I don't look, bug city. It's a quantum bug whose interfering wave function collapses upon inspection!

Has anyone ever experienced this strange behavior before?

EDIT:
Figured out that the error is occurring via the old Mesa headers used by the GL render system. Currently GL on Linux never cleanly releases its context on shutdown because it crashes while attempting to call glXDestroyContext(). Not sure what is the best remedy for this, though updating headers probably will help.
User avatar
robert_sasu
Google Summer of Code Student
Google Summer of Code Student
Posts: 237
Joined: Mon Apr 02, 2012 11:07 am
x 42

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by robert_sasu »

I have had a similar problem with DirectX 11 Render system last year. The problem was that not all the resources were freed before starting up the other render system. You must be sure to free every used resource before restarting the SampleBrowser.
Google Summer of Code 2013 Student
Topic: "DirectX 11 & Tessellation samples"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Assaf Raman
User avatar
masterfalcon
OGRE Team Member
OGRE Team Member
Posts: 4270
Joined: Sun Feb 25, 2007 4:56 am
Location: Bloomington, MN
x 126
Contact:

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by masterfalcon »

How are things going right now? What are you currently working on? Don't forget to give everyone status updates here in the forum.
User avatar
holocronweaver
Google Summer of Code Student
Google Summer of Code Student
Posts: 273
Joined: Mon Oct 29, 2012 8:52 pm
Location: Princeton, NJ
x 47

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by holocronweaver »

I have managed to get the geometry and tessellation shader stages working. I also fixed the isosurface sample and basic triangle tessellation sample which show off each of the respective stages. I am not sure I will be able to get the remaining two geometry shader samples working because they do not appear to be functional in any rendering system. I will give them a shot near the end of my GSoC, but I cannot promise they will work.

I was sick most of last week and a few days, which slowed my progress to a crawl, though I did manage to fix visual tests so that they are running again in GL and GL3+. (I decided to fix the tests earlier than scheduled because I need a way to quickly test RS changes so I do not pile on bugs.) I also worked on fixing a few of the samples, such as the Grass sample, which were something my disease addled brain could handle. I spent much of my remaining time trying to better understand the rather complex flow of shader processing in Ogre. Talk about levels of indirection! Calling parent class methods which call child class methods which call parent class methods which call methods of another class which calls parent class methods which call... :shock: Confusing is an understatement. This is OOP gone wild. I am grateful that multiple inheritance appears to be rarely used.

I have about half of my work on compute shaders completed, though I decided to set it aside and focus on something simpler, returning once I get a better understanding of the Ogre rendering system in general. In the mean time I have been working on atomic counters, transform feedback, texture multisampling, and image load/store. Any time I hit a wall while coding, I would move on to another project and hope it would give some hints for solving the one I temporarily abandoned. The result of this strategy has been so-so. I do indeed occasionally find commonalities among topics that help me understand why each is not working, but I realize eventually I have to face each problem to completion. Thus I am now committed to completing the listed features in order, one after another, hell or high water, until I am closer be being back on schedule.

Atomic counters are 90% finished, though I need some advisement to proceed. Ogre does an interesting dance to handle GL uniforms. First it manually parses the shader to extract a list of uniforms via GLSLProgram::buildConstantDefinitions and, particularly, GLSLProgramManagerCommon::extractConstantDefs. Later, after shader program compilation and just prior to rendering, uniforms are directly extracted from the program via OpenGL calls in GLSLProgramManagerCommon::extractUniforms and compared to the prior parsed list of uniforms. If they match, Ogre will add the uniform reference to a list so that its values can subsequently be set by Ogre via GLSLProgramPipeline::updateUniforms.

(please correct me if this description is somehow wrong)

During the first part, the manual parsing of the shader, the GLSL variable types are related to Ogre 'constant' variable types which begin with GCT (GpuConstantType). Nearly all variable types introduced in GL3+ have no direct correspondent in GCT, so they fail to be parsed. For instance, the GLSL type uint (from GL3) has no corresponding GCT type, though GCT_INT is close and has the same size. In cases like this, should I simply use the closest existing GCT type, or should I introduce new types? Currently I am reusing existing types, though GCT_INT is a little more of a stretch for atomic counters since they cannot be set using glProgramUniform and hence a special case has to be mode during binding in GLSLProgramPipeline::updateUniforms. Am I taking the right direction?

Perhaps I could answer this myself if I knew what the GCT types were meant for, besides helping with binding. To be honest I am not sure why they are used even there. Why not just directly use OpenGL types? It seems cleaner and more straightforward.


Once I get my build working again and complete at least one of the half-finished features I will push my changes.

EDIT:

Just noticed that there is a TODO comment someone left in the spot causing me uncertainly which says "Move these to a new type??" So I am not alone!
User avatar
masterfalcon
OGRE Team Member
OGRE Team Member
Posts: 4270
Joined: Sun Feb 25, 2007 4:56 am
Location: Bloomington, MN
x 126
Contact:

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by masterfalcon »

Sad to hear that you were sick. Your workflow sounds very similar to how I usually work on Ogre.

Your analysis of the flow for shaders is correct. The GCT types are just for identifying types of constants inside the engine in a platform-agnostic way. I would go ahead and add UINT types. DX11 can add support for them as well. Just make sure to add them at the end of the list but before GCT_UNKNOWN. What other types don't have their corresponding GCT entries? Also, you may also need to add code to GpuProgramParams and MaterialSerializer.
User avatar
holocronweaver
Google Summer of Code Student
Google Summer of Code Student
Posts: 273
Joined: Mon Oct 29, 2012 8:52 pm
Location: Princeton, NJ
x 47

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by holocronweaver »

I have spent three days now tracking down a few bugs which seem to be related to mishandled main memory. After much frustration, I decided to switch from Debug to Release builds and all the bugs disappeared. I replicated this across three separate OS (Windows, Ubuntu, & Fedora).

Why would a debug build type cause bugs? In my experience it is optimizations performed during release builds that causes problems, so I am flabbergasted. Anyone have experience with this?
User avatar
masterfalcon
OGRE Team Member
OGRE Team Member
Posts: 4270
Joined: Sun Feb 25, 2007 4:56 am
Location: Bloomington, MN
x 126
Contact:

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by masterfalcon »

Where are these bugs happening?
User avatar
holocronweaver
Google Summer of Code Student
Google Summer of Code Student
Posts: 273
Joined: Mon Oct 29, 2012 8:52 pm
Location: Princeton, NJ
x 47

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by holocronweaver »

The most annoying bug is:

OgreMain/src/OgreAnimationTrack.cpp:958: void Ogre::VertexAnimationTrack::applyPoseToVertexData(const Ogre::Pose*, Ogre::VertexData*, Ogre::Real): Assertion `!data->hwAnimationDataList.empty() && "Haven't set up hardware vertex animation elements!"' failed.

which occurs during the PlayPen_PoseAnimNoNormals visual test. Thus the animation data list was not successfully copied. This bug only appears when I use a debug build type. Even RelWithDebInfo does not exhibit it.

EDIT: BTW, there is also a bug in Zip which prevents it from handling relative paths and so I have to convert all the default relative paths in my resources.cfg to absolute paths. Have not had time to fix it yet. This only causes me trouble in Windows where the default resources.cfg uses relative paths.

EDIT2: Scratch that, handling relative paths seems to be generally broken in config file parsing.
Last edited by holocronweaver on Mon Jul 22, 2013 10:40 pm, edited 1 time in total.
User avatar
masterfalcon
OGRE Team Member
OGRE Team Member
Posts: 4270
Joined: Sun Feb 25, 2007 4:56 am
Location: Bloomington, MN
x 126
Contact:

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by masterfalcon »

Ah yes, I've seen that bug too but haven't tracked it down. If you're stuck on that I would just disable it for now.
User avatar
holocronweaver
Google Summer of Code Student
Google Summer of Code Student
Posts: 273
Joined: Mon Oct 29, 2012 8:52 pm
Location: Princeton, NJ
x 47

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by holocronweaver »

For now I will stick to testing my changes using RelWithDebInfo builds and only resort to Debug builds when something is really...bugging me. 8)
User avatar
Klaim
Old One
Posts: 2565
Joined: Sun Sep 11, 2005 1:04 am
Location: Paris, France
x 56
Contact:

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by Klaim »

There are basically three ways debug build can make more bugs than release builds:

1. code relying on macro depending on build mode;
2. code that is inlined (implicitely or explicitely) - including copy removal;
3. debug mode, at least for VS, will make all newly allocated raw memory set to values 0, while in other modes that memory will not be set at all - obviously most of the time this cause errors in release mode, but it might be because the release build does something with pointers which can't work with 0 but works with other values in the related memory accesses;

I can't say which one is related to your problem but maybe 3?

Hope it helps.
User avatar
holocronweaver
Google Summer of Code Student
Google Summer of Code Student
Posts: 273
Joined: Mon Oct 29, 2012 8:52 pm
Location: Princeton, NJ
x 47

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by holocronweaver »

Klaim wrote:I can't say which one is related to your problem but maybe 3?
Yep, it was 3. :) Thanks allot for the pointer. My head was getting a rosy bruise from all the slamming.
User avatar
holocronweaver
Google Summer of Code Student
Google Summer of Code Student
Posts: 273
Joined: Mon Oct 29, 2012 8:52 pm
Location: Princeton, NJ
x 47

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by holocronweaver »

In GLSLProgramManagerCommon::parseIndividualConstant, in the "if(sharedParams.isNull())" block, why is every uniform either float or int? The current algorithm counts sampler2D as part of int buffer. Either I do not understand what the buffers are for, or they are being reused to do something they were not intended for.

EDIT: :idea: I think I understand. sampler2D stores a pointer to an array, and the pointer is an int. Is this the idea?
User avatar
masterfalcon
OGRE Team Member
OGRE Team Member
Posts: 4270
Joined: Sun Feb 25, 2007 4:56 am
Location: Bloomington, MN
x 126
Contact:

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by masterfalcon »

Yeah that's right.
User avatar
Klaim
Old One
Posts: 2565
Joined: Sun Sep 11, 2005 1:04 am
Location: Paris, France
x 56
Contact:

Re: [GSoC 2013 - accepted] OpenGL 3+; Geometry & Tessellatio

Post by Klaim »

holocronweaver wrote:
Klaim wrote:I can't say which one is related to your problem but maybe 3?
Yep, it was 3. :) Thanks allot for the pointer. My head was getting a rosy bruise from all the slamming.

8)
Post Reply