[2.1] Shader compilation very slow or too many shaders being generated

Discussion area about developing with Ogre-Next (2.1, 2.2 and beyond)


Post Reply
xissburg
Halfling
Posts: 83
Joined: Sun Feb 21, 2010 4:58 pm
x 28

[2.1] Shader compilation very slow or too many shaders being generated

Post by xissburg »

I am using HLMS PBS for my materials and it seems it's generating one individual shader for each material even though they all just have different textures, colors, samplers, etc. So it takes a long time for my game to start up (about a minute). I would expect it to use only one or two shaders and pass different parameters. I even tried deleting all my materials actually and it still seems to compile 10 shaders (I see "GLSL link result" about 10 times in the output). What am I missing?

Thanks
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: [2.1] Shader compilation very slow or too many shaders being generated

Post by dark_sylinc »

10 shaders sounds about right, unless you have 10 materials; in which case the best explanation is that your materials are too different (the textures don't have the same resolution / format, they use different combinations e.g. one uses diffuse another uses diffuse + normal maps).

For improving shader compilation (by a lot) I suggest you read Starting my app takes forever! (particularly Direct3D11) from the 2.1 FAQ.
xissburg
Halfling
Posts: 83
Joined: Sun Feb 21, 2010 4:58 pm
x 28

Re: [2.1] Shader compilation very slow or too many shaders being generated

Post by xissburg »

Hi, thanks for pointing out the FAQ entry. I have implemented that code and encountered a few problems.

1. In OgreGL3PlusRenderSystem.cpp:542 it gets GL_NUM_PROGRAM_BINARY_FORMATS which is returning 0 so that really means my GPU does not support microcode? Obviously enabling caching didn't speed up the app start up.

2. It looks like the wrong parameter is being passed to getMicrocodeFromCache. In here OgreGLSLProgram.cpp-235 and here OgreGLSLMonolithicProgram.cpp-121 getCombinedName() is used whereas I think it should be getCombinedSource() since that's what's used as key when adding a new entry in OgreGLSLMonolithicProgram.cpp-247.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: [2.1] Shader compilation very slow or too many shaders being generated

Post by dark_sylinc »

xissburg wrote:Hi, thanks for pointing out the FAQ entry. I have implemented that code and encountered a few problems.

1. In OgreGL3PlusRenderSystem.cpp:542 it gets GL_NUM_PROGRAM_BINARY_FORMATS which is returning 0 so that really means my GPU does not support microcode? Obviously enabling caching didn't speed up the app start up.
I'm afraid that is correct. It is the driver the one not supporting this.
However often this is because the driver already performs an internal cache of its own (e.g. Mesa on Linux is the only one that does this AFAIK).
Perhaps your Mesa build is too old or compiled without the internal cache?
If this is OS X, I'm afraid you're out of luck. AFAIK OS X is the only OpenGL implementation that does not offer any sort of shader cahing.
Metal does have an internal cache though (like Mesa).
xissburg wrote:2. It looks like the wrong parameter is being passed to getMicrocodeFromCache. In here OgreGLSLProgram.cpp-235 and here OgreGLSLMonolithicProgram.cpp-121 getCombinedName() is used whereas I think it should be getCombinedSource() since that's what's used as key when adding a new entry in OgreGLSLMonolithicProgram.cpp-247.
:shock:
You're right. Can't believe that went hidden for this long.
Fixing...
xissburg
Halfling
Posts: 83
Joined: Sun Feb 21, 2010 4:58 pm
x 28

Re: [2.1] Shader compilation very slow or too many shaders being generated

Post by xissburg »

I am on macOS so yeah, drivers suck :|

I'll try to make my materials follow the same layout to avoid generating too many shaders.
Post Reply