[2.1][GL3Plus] Reloading shaders

Design / architecture / roadmap discussions related to future of Ogre3D (version 2.0 and above)
Post Reply
zxz
Greenskin
Posts: 139
Joined: Sat Apr 16, 2016 9:25 pm
x 5

[2.1][GL3Plus] Reloading shaders

Post by zxz » Wed May 23, 2018 11:28 am

Hello,

I am having issues reloading a shader in Ogre 2.1, with the GL3Plus rendering system. For reloading, I am using the same procedure as in HdrUtils and MiscUtils from the samples.

Something along these lines:

Code: Select all

          Ogre::MaterialPtr material = Ogre::MaterialManager::getSingleton().load(
                      "HDR/Resolve_4xFP32_HDR_Box",
                      Ogre::ResourceGroupManager::AUTODETECT_RESOURCE_GROUP_NAME ).
                  staticCast<Ogre::Material>();
  
          Ogre::Pass *pass = material->getTechnique(0)->getPass(0);

          Ogre::GpuProgram *shader = 0;
          Ogre::GpuProgramParametersSharedPtr oldParams;
    
          //Save old manual & auto params
          oldParams = pass->getFragmentProgramParameters();
          //Retrieve the HLSL/GLSL/Metal shader and rebuild it with the right settings.
          shader = pass->getFragmentProgram()->_getBindingDelegate();
          shader->setParameter( "preprocessor_defines", preprocessorDefines );
          pass->getFragmentProgram()->reload();
          //Restore manual & auto params to the newly compiled shader
          pass->getFragmentProgramParameters()->copyConstantsFrom( *oldParams );
If I do this any later than during initial loading, no rebuild is made (preprocessor defines not applied), and parameter (uniform) changes stop working for the specified shader.

Is this a known issue, or am I just doing something wrong?

I found this issue on the subject:
https://github.com/OGRECave/ogre/issues/75
which was fixed in
https://github.com/OGRECave/ogre/pull/577
last November.

Unfortunately these changes don't apply very cleanly at all on 2.1 in isolation. Are there any other commits that would help apply these fixes more cleanly?

paroj?
0 x

Hrenli
Halfling
Posts: 45
Joined: Tue Jun 14, 2016 12:26 pm
x 8

Re: [2.1][GL3Plus] Reloading shaders

Post by Hrenli » Wed May 23, 2018 12:31 pm

After reading your post I've tried it with 2.1 samples (PBS materials to be precise). Run the sample, left it running, went to media folder, added one line in pixel shader template (making outColour.xyz white no matter what), switched back to the sample, pressed ctrl-F1 - it works. Shaders were recompiled and all PBS materials went white. All under GL3Plus renderer. So I think it has to be somewhere in your code...
0 x

zxz
Greenskin
Posts: 139
Joined: Sat Apr 16, 2016 9:25 pm
x 5

Re: [2.1][GL3Plus] Reloading shaders

Post by zxz » Wed May 23, 2018 1:37 pm

Hello,

Thanks for the test results. However, reloading the HLMS isn't necessarily the same as recompiling a single GPUProgram. I took a quick look at the code run when the HLMS is reloaded, and it's doesn't really seem applicable to the use case above.
0 x

paroj
OGRE Team Member
OGRE Team Member
Posts: 594
Joined: Sun Mar 30, 2014 2:51 pm
x 83
Contact:

Re: [2.1][GL3Plus] Reloading shaders

Post by paroj » Wed May 23, 2018 2:04 pm

it will not be as easy as cherry-picking patches - GL3Plus has heavily diverged between 1.x and 2.1. You will have to manually port the changes. For this look at the following commit:
https://github.com/OGRECave/ogre/pull/5 ... fde424a8ad

the fix is using "destroyAllByProgram".

Additionally in 1.11 the reloading code was changed further by this commit:
https://github.com/OGRECave/ogre/commit ... eb1db30119

Now GL shaders are compiled on load and use the core mCompileError flag instead of the GL specific lifecycle handling.
With this GL and D3D shaders should behave the same in Ogre.
0 x

zxz
Greenskin
Posts: 139
Joined: Sat Apr 16, 2016 9:25 pm
x 5

Re: [2.1][GL3Plus] Reloading shaders

Post by zxz » Wed May 23, 2018 3:42 pm

What are the high level differences in the Rendersystems between 1.11 and 2.1? The question can be asked for GL3Plus in particular, and all rendersystems in general.

- 1.11 has a lot of bugfixes, cleanup
- 2.1 a bunch of new files when it comes to windowing and VAOs.
- .. and more, I only briefly eyed the changes

Shouldn't they in principle be the same thing, only with divergence due to accumulating changes over time that never ended up being merged?

Are there any big conceptual things that would block the two branches from using the "same" rendersystems? For example, starting from 1.11 and adding the new things from 2.1.
0 x

xrgo
OGRE Expert User
OGRE Expert User
Posts: 900
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 50

Re: [2.1][GL3Plus] Reloading shaders

Post by xrgo » Wed May 23, 2018 6:24 pm

Haven't looked at 1.11, but 2.1 uses azdo
0 x

paroj
OGRE Team Member
OGRE Team Member
Posts: 594
Joined: Sun Mar 30, 2014 2:51 pm
x 83
Contact:

Re: [2.1][GL3Plus] Reloading shaders

Post by paroj » Wed May 23, 2018 11:20 pm

xrgo wrote:
Wed May 23, 2018 6:24 pm
Haven't looked at 1.11, but 2.1 uses azdo
yes, in 2.1 the RenderSystems are using the new CommandBuffer API[1], which in turn requires HLMS in core (i.e. the HLMS backport is not sufficient).

Additionally some 1.x classes have been put in the v1 namespace [2] instead of using a v2 namespace for the new stuff, which breaks backwards compatibility.

[1] https://github.com/OGRECave/ogre/tree/v ... mandBuffer
[2] https://github.com/OGRECave/ogre/blob/v ... tem.h#L126
0 x

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 3740
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 130
Contact:

Re: [2.1][GL3Plus] Reloading shaders

Post by dark_sylinc » Fri May 25, 2018 5:03 am

I shall chime in:
  1. Although 2.1 is AZDO, what zxz is asking is a much simpler level, where AZDO is not yet involved
  2. Most of the fixes paroj made to GpuProgramPtr & GL3PlusGpu*Program families should be possible to port them, but as paroj indicated, the branches have diverged a lot by now.
  3. One major source of pain is that originally GL3+ RenderSystem placed VAOs inside the shader, which makes no sense. 2.1 fixed it with VertexArrayObject class and PSOs, paroj solved it differently. However to keep the legacy paths working 2.1 still has some of this old Vao pointers hanging around (that apparently do nothing except when used inside ::_render(const v1::RenderOperation& op) ). It should be removed, but touching this is a minefield
I do not know why zxz's snippet isn't reloading the shader but it is for me with the HDR & Smaa samples. I suspect it could be the microcode cache. Or perhaps the reload only works once.
0 x

Post Reply