[SOLVED] [1.11.2] Instancing fails on OpenGL Topic is solved

Problems building or running the engine, queries about how to use features etc.
rpgplayerrobin
Bugbear
Posts: 804
Joined: Wed Mar 18, 2009 3:03 am
x 463

[SOLVED] [1.11.2] Instancing fails on OpenGL

Post by rpgplayerrobin »

Ogre Version: 1.11.2
Operating System: Windows 10
Render System: OpenGL

Hello!

I have started to experiment with the instancing (the NewInstancing sample in 1.11.2), and it seems to be working great.
I am using the HW Basic instancing.

The only problem is that it does not work for CG shaders for OpenGL. It works for Direct3D9 and Direct3D11 perfectly.
It seems that the TEXCOORD1-3 data seems to be completely scrambled when using CG for OpenGL.

If I convert it by hand to a glsl shader, it works, but I really want to avoid having to do that.
After a couple of hours trying different things in the shader to try to fix it for CG, I realized I could look on the example in the SampleBrowser.

The SampleBrowser has the EXACT same issue as I have been having with OpenGL:
Image
It should look like this:
Image

Has anyone any idea what is happening here?
Has anyone any workaround to make it work?
Last edited by rpgplayerrobin on Mon Feb 25, 2019 11:38 pm, edited 1 time in total.
User avatar
saintnick
Halfling
Posts: 51
Joined: Tue Nov 27, 2018 1:41 am
x 5

Re: [1.11.2] Instancing fails on OpenGL

Post by saintnick »

I can reproduce this behavior. I have not worked with instancing or shaders however and can't solve your problem.

Ogre 1.12
Windows 10
OpenGL

Same behavior as OP in sample browser.
rpgplayerrobin
Bugbear
Posts: 804
Joined: Wed Mar 18, 2009 3:03 am
x 463

Re: [1.11.2] Instancing fails on OpenGL

Post by rpgplayerrobin »

It is good to hear that its not just my computer! :D
I wonder if someone has a possible fix for this.
paroj
OGRE Team Member
OGRE Team Member
Posts: 2278
Joined: Sun Mar 30, 2014 2:51 pm
x 1241

Re: [1.11.2] Instancing fails on OpenGL

Post by paroj »

try using the glslf/ glslv cg profiles and inspect the generated source code that will be in the log - it will be you only chance getting this working on non-nvidia hardware anyway.
rpgplayerrobin
Bugbear
Posts: 804
Joined: Wed Mar 18, 2009 3:03 am
x 463

Re: [1.11.2] Instancing fails on OpenGL

Post by rpgplayerrobin »

paroj wrote: Sun Feb 24, 2019 7:30 pm try using the glslf/ glslv cg profiles and inspect the generated source code that will be in the log - it will be you only chance getting this working on non-nvidia hardware anyway.
Thank you! This helped A LOT to debug why the shaders do not work.

The generated glsl code used "dot" instead of just doing "v * m" of matrices, which I was able to fix by doing the matrix multiplication manually in the shader (which translates later to glsl perfectly).

Though there is still one obstacle that I cannot seem to fix, gl_MultiTexCoordX does not seem to work, at all.

This is now my adjusted vertex hlsl shader (which I use for CG):

Code: Select all

float4x4 viewProjMatrix;

struct VS_OUTPUT
{
	float2 oUV : TEXCOORD0;
};

VS_OUTPUT main_vs( float4 position : POSITION,
				   out float4 oPosition : POSITION,
				   float2 iUV : TEXCOORD0,
				   float4 mat14 : TEXCOORD1,
				   float4 mat24 : TEXCOORD2,
				   float4 mat34 : TEXCOORD3 )
{
	VS_OUTPUT Out;

	float4 worldPos;
	worldPos.x = dot(mat14, position);
	worldPos.y = dot(mat24, position);
	worldPos.z = dot(mat34, position);
	worldPos.w = 1;

	oPosition.x = dot(float4(viewProjMatrix[0].x, viewProjMatrix[1].x, viewProjMatrix[2].x, viewProjMatrix[3].x), worldPos);
	oPosition.y = dot(float4(viewProjMatrix[0].y, viewProjMatrix[1].y, viewProjMatrix[2].y, viewProjMatrix[3].y), worldPos);
	oPosition.z = dot(float4(viewProjMatrix[0].z, viewProjMatrix[1].z, viewProjMatrix[2].z, viewProjMatrix[3].z), worldPos);
	oPosition.w = dot(float4(viewProjMatrix[0].w, viewProjMatrix[1].w, viewProjMatrix[2].w, viewProjMatrix[3].w), worldPos);

	Out.oUV = iUV;

	return Out;
}
That gets turned into this (seen from the log file as you mentioned):

Code: Select all

vec4 _oPosition1;
vec4 _a0008;
vec4 _a0010;
vec4 _a0012;
vec4 _a0014;
uniform mat4 viewProjMatrix;

void main()
{
    vec4 _worldPos;

    _worldPos.x = dot(gl_MultiTexCoord1, gl_Vertex);
    _worldPos.y = dot(gl_MultiTexCoord2, gl_Vertex);
    _worldPos.z = dot(gl_MultiTexCoord3, gl_Vertex);
    _worldPos.w = 1.00000000E+000;
    _a0008 = vec4(viewProjMatrix[0].x, viewProjMatrix[1].x, viewProjMatrix[2].x, viewProjMatrix[3].x);
    _oPosition1.x = dot(_a0008, _worldPos);
    _a0010 = vec4(viewProjMatrix[0].y, viewProjMatrix[1].y, viewProjMatrix[2].y, viewProjMatrix[3].y);
    _oPosition1.y = dot(_a0010, _worldPos);
    _a0012 = vec4(viewProjMatrix[0].z, viewProjMatrix[1].z, viewProjMatrix[2].z, viewProjMatrix[3].z);
    _oPosition1.z = dot(_a0012, _worldPos);
    _a0014 = vec4(viewProjMatrix[0].w, viewProjMatrix[1].w, viewProjMatrix[2].w, viewProjMatrix[3].w);
    _oPosition1.w = dot(_a0014, _worldPos);
    gl_TexCoord[0].xy = gl_MultiTexCoord0.xy;
    gl_Position = _oPosition1;
    return;
}
And that shader does not work, but if I alter it to this, it works:

Code: Select all

vec4 _oPosition1;
vec4 _a0008;
vec4 _a0010;
vec4 _a0012;
vec4 _a0014;
uniform mat4 viewProjMatrix;

attribute vec4 uv1;
attribute vec4 uv2;
attribute vec4 uv3;

void main()
{
    vec4 _worldPos;

    _worldPos.x = dot(uv1, gl_Vertex);
    _worldPos.y = dot(uv2, gl_Vertex);
    _worldPos.z = dot(uv3, gl_Vertex);
    _worldPos.w = 1.00000000E+000;
    _a0008 = vec4(viewProjMatrix[0].x, viewProjMatrix[1].x, viewProjMatrix[2].x, viewProjMatrix[3].x);
    _oPosition1.x = dot(_a0008, _worldPos);
    _a0010 = vec4(viewProjMatrix[0].y, viewProjMatrix[1].y, viewProjMatrix[2].y, viewProjMatrix[3].y);
    _oPosition1.y = dot(_a0010, _worldPos);
    _a0012 = vec4(viewProjMatrix[0].z, viewProjMatrix[1].z, viewProjMatrix[2].z, viewProjMatrix[3].z);
    _oPosition1.z = dot(_a0012, _worldPos);
    _a0014 = vec4(viewProjMatrix[0].w, viewProjMatrix[1].w, viewProjMatrix[2].w, viewProjMatrix[3].w);
    _oPosition1.w = dot(_a0014, _worldPos);
    gl_TexCoord[0].xy = gl_MultiTexCoord0.xy;
    gl_Position = _oPosition1;
    return;
}
As you may see, I altered the gl_MultiTexCoordX to use the "attribute vec4 uvX" instead, and then everything works.

So my question is, why are the values of gl_MultiTexCoordX scrambled/incorrect?
Should not those values be the same as the attribute values?
Is there some way to fix this?
paroj
OGRE Team Member
OGRE Team Member
Posts: 2278
Joined: Sun Mar 30, 2014 2:51 pm
x 1241

Re: [1.11.2] Instancing fails on OpenGL

Post by paroj »

are you on NVidia? Because OGRE is doing exactly what is said on the bottom of this page:
https://www.opengl.org/sdk/docs/tutoria ... ibutes.php

you could try extending the HighLevelOutputFixer to replace gl_MultiTexCoordX by uvX:
https://github.com/OGRECave/ogre/blob/m ... m.cpp#L557
rpgplayerrobin
Bugbear
Posts: 804
Joined: Wed Mar 18, 2009 3:03 am
x 463

Re: [1.11.2] Instancing fails on OpenGL

Post by rpgplayerrobin »

Yeah, I am on Nvidia.

I altered the HighLevelOutputFixer and finally got the shader to look correctly in the log file.
However, it still does not render correctly...

If I copy/paste the shader seen in the log and use glsl instead of cg, the shader works correctly. Why? :lol:
It does update the shader correctly with my new values, but for some reason it just does not work while using cg.

Can this be some sort of difference between glslv/glslf and arbvp1/glsl120?
paroj
OGRE Team Member
OGRE Team Member
Posts: 2278
Joined: Sun Mar 30, 2014 2:51 pm
x 1241

Re: [1.11.2] Instancing fails on OpenGL

Post by paroj »

when using GLSL, Cg Plugin does this:

Code: Select all

mDelegate->setParameter("column_major_matrices", "false");
rpgplayerrobin
Bugbear
Posts: 804
Joined: Wed Mar 18, 2009 3:03 am
x 463

Re: [1.11.2] Instancing fails on OpenGL

Post by rpgplayerrobin »

I was doing a workaround by creating two shaders, one with the glslv/glslf and one with arbvp1/arbfp1, and copying the contents from one to the other. It was a huge mess.

Your answered saved this mess! :D
I edited "column_major_matrices" to true instead and it worked! No more need for double shaders.

I have solved most of the problems so far, instancing, shadows and samplers in the same glslv/glslf shader, but it seems that lights are still a huge mess, I will get back tomorrow to try to figure out why the lights are strange in the shader.

For anyone else having problems, I can explain how I solved (most) of my problems so far.



In the end of the "CgProgram::fixHighLevelOutput", I added a little code (a very ugly code, but I did not manage to succeed to build it into CgProgram itself because I want to use a static variable):

Code: Select all

if(LogManager::getSingleton().m_AlterCGShader)
	hlSource = LogManager::getSingleton().m_AlterCGShader->AlterCGShader(this->getName(), hlSource);
Then I added this into the LogManager class:

Code: Select all

class CAlterCGShader_Basic
{
public:
	virtual String AlterCGShader(String cgProgramName, String& code) = 0;
};
CAlterCGShader_Basic* m_AlterCGShader;
And into the LogManager constructor:

Code: Select all

m_AlterCGShader = NULL;
Then in my application code, I added this class:
.h

Code: Select all

class CAlterCGShader : public LogManager::CAlterCGShader_Basic
{
public:
	String AlterCGShader(String cgProgramName, String& code);
	void FixShadersAfterCompile();

	static std::unordered_map<std::string, std::string> m_compiledSources;
};
.cpp

Code: Select all

// Called every time a CG shader gets compiled when using the glslv/glslf profile
std::unordered_map<std::string, std::string> CAlterCGShader::m_compiledSources;
String CAlterCGShader::AlterCGShader(String cgProgramName, String& code)
{
	// Alter the shaders needed
	CString tmpStr = code; // CString is basically just a wrapper for std::string, but with many good functions, like "Contains" and "Replace"
	if (CString(cgProgramName).Contains("InstancingHWBasic") && tmpStr.Contains("gl_MultiTexCoord"))
	{
		tmpStr.Replace("void main", "attribute vec4 uv0;\nattribute vec4 uv1;\nattribute vec4 uv2;\nattribute vec4 uv3;\n\nvoid main");
		tmpStr.Replace("gl_MultiTexCoord", "uv");
	}
	tmpStr.Replace("1.00000000E+000", "1.0");
	tmpStr.Replace("0.00000000E+000", "0.0");

	// You can do alterations to specific shaders
	if (cgProgramName == "HighShader_InstancingHWBasic_PS")
	{
		// It scrambles the samplers
		tmpStr.Replace("uniform sampler2D SpecularMap;", "");
		tmpStr.Replace("uniform sampler2D GlowMap;", "");
		tmpStr.Replace("uniform sampler2D NormalMap;", "");
		tmpStr.Replace("uniform sampler2D ShadowMap;", "");
		tmpStr.Replace("uniform sampler2D CloudShadowMap;", "");
		CString tmpVar = "";
		tmpVar += "uniform sampler2D DiffuseMap;\n";
		tmpVar += "uniform sampler2D NormalMap;\n";
		tmpVar += "uniform sampler2D SpecularMap;\n";
		tmpVar += "uniform sampler2D GlowMap;\n";
		tmpVar += "uniform sampler2D ShadowMap;\n";
		tmpVar += "uniform sampler2D CloudShadowMap;\n";
		tmpStr.Replace("uniform sampler2D DiffuseMap;", tmpVar.m_string);
	}

	m_compiledSources[cgProgramName] = tmpStr.m_string;

	return tmpStr.m_string;
}

// Fixes the shaders after they have been compiled (or you could just simply disable the column major matrices behaviour in the ogre source code)
void CAlterCGShader::FixShadersAfterCompile()
{
	// Loop through the list of compiled sources
	for (std::unordered_map<std::string, std::string>::const_iterator tmpItr = CAlterCGShader::m_compiledSources.begin(); tmpItr != CAlterCGShader::m_compiledSources.end(); ++tmpItr)
	{
		// Get the program and see if it exists
		std::string tmpTarget = tmpItr->first;
		GpuProgramPtr tmpGpuProgramPtr = GpuProgramManager::getSingleton().getByName(tmpTarget);
		if (tmpGpuProgramPtr)
		{
			// Get the delegate of the program and see if it exists
			HighLevelGpuProgramPtr tmpGpuProgramPtr_Delegate = HighLevelGpuProgramManager::getSingleton().getByName(tmpTarget + "/Delegate");
			if (tmpGpuProgramPtr_Delegate)
			{
				// Reload the delegate after we set the column major matrices to true again
				tmpGpuProgramPtr_Delegate->setParameter("column_major_matrices", "true");
				CGeneric::RemoveShaderMicrocodeCache(tmpGpuProgramPtr_Delegate);
				tmpGpuProgramPtr_Delegate->reload();
			}
		}
	}
}
After that, in my application code, I added this before any shaders are loaded:

Code: Select all

if(IsUsingOpenGL)
	LogManager::getSingleton().m_AlterCGShader = new CAlterCGShader();
After the shaders have been loaded/compiled (or reloaded/recompiled), use this code:

Code: Select all

// Fix the shaders now that they have been compiled
if(LogManager::getSingleton().m_AlterCGShader)
	((CAlterCGShader*)LogManager::getSingleton().m_AlterCGShader)->FixShadersAfterCompile();
Then be sure to destroy it somewhere when exiting the application:

Code: Select all

if (LogManager::getSingleton().m_AlterCGShader)
{
	((CAlterCGShader*)LogManager::getSingleton().m_AlterCGShader)->m_compiledSources.clear();
	delete LogManager::getSingleton().m_AlterCGShader;
	LogManager::getSingleton().m_AlterCGShader = NULL;
}
Last edited by rpgplayerrobin on Mon Feb 25, 2019 11:41 pm, edited 1 time in total.
rpgplayerrobin
Bugbear
Posts: 804
Joined: Wed Mar 18, 2009 3:03 am
x 463

Re: [1.11.2] Instancing fails on OpenGL

Post by rpgplayerrobin »

I finally fixed lights to also works. I'll mark this topic as 100% solved now!
Thank you so much for helping me! :D

I added this to the AlterCGShader function to make lights work properly:

Code: Select all

if (cgProgramName == "HighShader_InstancingHWBasic_VS")
{
	tmpStr.Replace("attribute vec4 TANGENT;", "");
	tmpStr.Replace("TANGENT", "tangent");
	tmpStr.Replace("attribute vec4 uv3;", "attribute vec4 uv3;\nattribute vec3 tangent;");
}
I also forgot to show you how I changed the calculations for the glsl variant of the shader to make it work with instancing:

Code: Select all

Instead of doing:
float3x4 calculatedWorldMatrix;
calculatedWorldMatrix[0] = mat14;
calculatedWorldMatrix[1] = mat24;
calculatedWorldMatrix[2] = mat34;
float4 worldPos = float4(mul(calculatedWorldMatrix, position).xyz, 1);

I did this:
float4 worldPos;
worldPos.x = dot(mat14, position);
worldPos.y = dot(mat24, position);
worldPos.z = dot(mat34, position);
worldPos.w = 1;


And instead of doing:
oPosition = mul(viewProjMatrix, worldPos);

I did this:
oPosition.x = dot(float4(viewProjMatrix[0].x, viewProjMatrix[1].x, viewProjMatrix[2].x, viewProjMatrix[3].x), worldPos);
oPosition.y = dot(float4(viewProjMatrix[0].y, viewProjMatrix[1].y, viewProjMatrix[2].y, viewProjMatrix[3].y), worldPos);
oPosition.z = dot(float4(viewProjMatrix[0].z, viewProjMatrix[1].z, viewProjMatrix[2].z, viewProjMatrix[3].z), worldPos);
oPosition.w = dot(float4(viewProjMatrix[0].w, viewProjMatrix[1].w, viewProjMatrix[2].w, viewProjMatrix[3].w), worldPos);