Does anyone have any suggestions about how one would go about debugging a CG shader? I really want to get rid of all the separation between platforms (HLSL and GLSL) but I cannot for the life of me find a single program that can debug CG shaders in a helpful manner. I tried to use the DirectX PIX debugger but all it shows is the assembly code, which, for somebody like me that knows nothing about assembly, is impossible to debug.
I've been looking into converting all of the shaders to CGFX files and using nvidia FX Composer, which has very nice shader debugging, but that seems overly complicated since I would basically have to rewrite the shaders to convert back and forth between CG and CGFX.
I mostly perform manual CG debugging (e.g. display intermediary result instead of final value, Ogre's compile errors ) but it's slow and not always that helpful for complex shaders.
I may be wrong, but aren't CG shaders transformed into either GLSL or HLSL when they are used?
If so, the shader that actually runs is not the one you wrote, but the transformed one for which you have no code as it likely exists in binary format only (in memory).
I guess this makes "real" debugging nearly impossible.
When fiddling with cg shaders, I use the cgc commandline compiler to verify that I have no syntax errors. Of course debugging is a step further.
Like TheSheep said, you can specify in the cg compiler what your output has to be. You can make it output either assembly for a certain graphics card brand's dialect, or make it output HLSL or GLSL.
duststorm wrote:
Just specify a HLSL or GLSL profile and you can use your old tool to debug the shader.
I thought I had already tried that actually and It didn't change anything, at least when using PIX to debug the shader it still only showed assembly and no source. I could try again I suppose. I'll let you know what I find out in a few minutes.
EDIT: Well I just finished testing it. I changed all fragment shaders to use hlslf and vertex shaders to use hlslv and the shaders don't even work anymore. The Ogre log is spitting out lots of stuff like "Pass 0: Vertex program Basic/AmbientOneTexture cannot be used - not supported.". Is this a graphics card issue? It seems a little strange that this wouldn't be working if normal HLSL shaders work just fine...
Hmm, I don't know what's happening there.
Did you change your material and program files correctly to use HLSL? You might need to manually compile your hlsl shaders with cg first and copy the generated hlsl output files in your shaders folder. I don't know if you can make it compile in HLSL within ogre dynamically.
In your vertex_program and fragment_program declaration, make sure the type is set to "hlsl" and point source to your .hlsl file.
Also choose appropriate syntax (probably something like ps_... or vs_...).
Have a look in the manual for available options: http://www.ogre3d.org/docs/manual/manual_18.html#SEC104
hlslv and hlslf do not exist in the ogre material setup as far as I know.
Just make sure source points to the .hlsl file you manually compiled with cg.
I have no idea whether Ogre supports alternative parameters to cg to compile from .cg to hlsl and load it as hlsl shader on the fly, but you probably first want to try it like this before doing more fancy stuff.
Ah that explains it. I guess I'll give that a shot. It's not the most ideal way to debug since it requires so many changes, but if it's the only thing that works, oh well.
Been super busy and not sure I have the time to test the first option, but I'm trying to get to it. What, if you don't mind me asking, would the "fancy stuff" be? Some modifications to the engine code or something?
EDIT: Well I just gave it a shot and there are some issues with the generated HLSL files. Cg is adding global variables and modifying them, which is illegal in newer versions HLSL since all global variables are const. This looks like a bug in Cg or something, unless you know anything about a way to make it not use globals or something. I manually cleaned up the files though and it seems to be somewhat working. The lines seem to be off a bit since the current line marker jumps around all over the place, but the assembly code has a bit more comments which is helpful.
With fancy stuff I meant putting a CG shader source file in your materials folder, but somehow setting up Ogre so it uses the CG compiler to compile and output a .hlsl shader, and pass that to the DirectX API.
Yes, it's probably not supported and you would probably need to hack the Ogre source to pull off such a thing. Whether that would be a useful thing to do.. I don't know.
Ah, that's what I was afraid of...Most likely not going to go that route. I guess It's going to have to be either learn assembly and have a hard time debugging it or to have maintainability issues having to write both HLSL and GLSL shaders. Either way kinda sucks, oh well
Hmm apparently there is already an ogre fork that does shader language translation.
It's called hlsl2glslfork. (http://www.ogre3d.org/forums/viewtopic.php?f=21&t=62256)
It's intended for another purpose though, but if you would like to do some modding it might be a good starting point.
Wow...that is the most confusing code I've ever seen. I don't think I could ever be able to touch that without getting lost. I really don't have the time to take on something like that. If it's not a somewhat simple solution I can't really afford to work on it.
