[SOLVED] Is there a plan to support DXT5 format

[cc9cii]

As per the subject title, is there a plan for such support?

I'm using existing assets (there are literally thousands) that cannot easily be converted for distribution (I don't own them and the end-users aren't necessarily technical) using texconv as per this post or this post.

DXGI_FORMAT_BC3_UNORM is the actual texture format. I noticed that the same format textures are accepted for diffuse, and only normals result in an exception:

Code: Select all

                // Not yet supported
                OGRE_EXCEPT(Exception::ERR_NOT_IMPLEMENTED,
                    "unpack from "+getFormatName(pf)+" not implemented",
                    "PixelUtil::unpackColour");

[dark_sylinc]

I assume you mean BC3 (DXT5) as normal maps?

There are no plans to support BC3 as normal maps. Accepted formats for normal mapping are RG8_UNORM, BC5_SNORM, and RG8_SNORM.

If you're interested in adding support the relevant pieces are:
https://github.com/OGRECave/ogre-next/b ... s.cpp#L794

Code: Select all

if (isSigned)
{
    setProperty( PbsProperty::NormalSamplingFormat, PbsProperty::NormalRgSnorm.mHash );
    setProperty( PbsProperty::NormalRgSnorm, PbsProperty::NormalRgSnorm.mHash );
}
else
{
    setProperty( PbsProperty::NormalSamplingFormat, PbsProperty::NormalRgUnorm.mHash );
    setProperty( PbsProperty::NormalRgUnorm, PbsProperty::NormalRgUnorm.mHash );
}

https://github.com/OGRECave/ogre-next/b ... ps.any#L95

Code: Select all

@property( normal_sampling_format == normal_rg_snorm )
	//Normal texture must be in UV8/RG8_SNORM or BC5S format!
	#define getTSNormal( normalMap, samplerState, uv, normalIdx ) reconstructZfromTSNormal( OGRE_SampleArray2D( normalMap, samplerState, uv, normalIdx ).xy )
@end
@property( normal_sampling_format == normal_rg_unorm )
	//Normal texture must be in RG8_UNORM or similar format!
	#define getTSNormal( normalMap, samplerState, uv, normalIdx ) reconstructZfromTSNormal( OGRE_SampleArray2D( normalMap, samplerState, uv, normalIdx ).xy * 2.0 - 1.0 )
@end
@property( normal_sampling_format == normal_la )
	//Normal texture must be in LA format!
	#define getTSNormal( normalMap, samplerState, uv, normalIdx ) reconstructZfromTSNormal( OGRE_SampleArray2D( normalMap, samplerState, uv, normalIdx ).xw * 2.0 - 1.0 )
@end

Most likely you'd want to add PbsProperty::NormalBc3Unorm and sample from .yw * 2.0 - 1.0 (notice the Green and Alpha channel are being used. I assume the textures you're talking about were encoded using this old trick)

I don't know however, why are you getting an exception, as you didn't post the callstack, hence I don't know what else would block you to support BC3

[cc9cii]

Hi,

Thanks for your reply. Call stack is shown below:

Code: Select all

>	OgreMain.dll!Ogre::ExceptionFactory::throwException(Ogre::Exception::ExceptionCodes code, int number, const std::string & desc, const std::string & src, const char * file, long line) Line 282	C++	Symbols loaded.
 	OgreMain.dll!Ogre::PixelUtil::unpackColour(float * r, float * g, float * b, float * a, Ogre::PixelFormat pf, const void * src) Line 997	C++	Symbols loaded.
 	OgreMain.dll!Ogre::PixelUtil::unpackColour(unsigned char * r, unsigned char * g, unsigned char * b, unsigned char * a, Ogre::PixelFormat pf, const void * src) Line 899	C++	Symbols loaded.
 	OgreMain.dll!Ogre::PixelUtil::convertForNormalMapping(const Ogre::PixelBox & src, const Ogre::PixelBox & dst) Line 1316	C++	Symbols loaded.
 	OgreMain.dll!Ogre::HlmsTextureManager::copyTextureToArray(const Ogre::Image & srcImage, Ogre::SharedPtr<Ogre::Texture> dst, unsigned short entryIdx, unsigned char srcBaseMip, bool isNormalMap) Line 215	C++	Symbols loaded.
 	OgreMain.dll!Ogre::HlmsTextureManager::createOrRetrieveTexture(const std::string & aliasName, const std::string & texName, Ogre::HlmsTextureManager::TextureMapType mapType, unsigned int uniqueSpecialId, Ogre::Image * imgSource) Line 717	C++	Symbols loaded.
 	OgreMain.dll!Ogre::HlmsTextureManager::createOrRetrieveTexture(const std::string & texName, Ogre::HlmsTextureManager::TextureMapType mapType) Line 398	C++	Symbols loaded.
 	openmw.exe!NiBtOgre::OgreMaterial::getOrCreateMaterial(const std::string & name) Line 491	C++	Symbols loaded.
 	openmw.exe!NiBtOgre::NiTriBasedGeom::getMaterial() Line 362	C++	Symbols loaded.
 	openmw.exe!NiBtOgre::NiTriBasedGeom::buildSubMesh(Ogre::v1::Mesh * mesh, NiBtOgre::BoundsFinder & bounds) Line 856	C++	Symbols loaded.
 	openmw.exe!NiBtOgre::NiNode::buildMesh(Ogre::v1::Mesh * mesh) Line 208	C++	Symbols loaded.
 	openmw.exe!NiBtOgre::NiMeshLoader::loadManualMesh(Ogre::v1::Mesh * pMesh, const NiBtOgre::NiMeshLoader::ModelBuildInfo & params) Line 156	C++	Symbols loaded.
 	openmw.exe!NiBtOgre::NiMeshLoader::loadResource(Ogre::Resource * res) Line 114	C++	Symbols loaded.
 	OgreMain.dll!Ogre::Resource::load(bool background) Line 212	C++	Symbols loaded.
 	OgreMain.dll!Ogre::v1::MeshManager::load(const std::string & filename, const std::string & groupName, Ogre::v1::HardwareBuffer::Usage vertexBufferUsage, Ogre::v1::HardwareBuffer::Usage indexBufferUsage, bool vertexBufferShadowed, bool indexBufferShadowed) Line 128	C++	Symbols loaded.
 	OgreMain.dll!Ogre::v1::EntityFactory::createInstanceImpl(unsigned int id, Ogre::ObjectMemoryManager * objectMemoryManager, Ogre::SceneManager * manager, const std::map<std::string,std::string,std::less<std::string>,Ogre::STLAllocator<std::pair<std::string const ,std::string>,Ogre::CategorisedAllocPolicy<0>>> * params) Line 2084	C++	Symbols loaded.
 	OgreMain.dll!Ogre::SceneManager::createMovableObject(const std::string & typeName, Ogre::ObjectMemoryManager * objectMemMgr, const std::map<std::string,std::string,std::less<std::string>,Ogre::STLAllocator<std::pair<std::string const ,std::string>,Ogre::CategorisedAllocPolicy<0>>> * params) Line 4811	C++	Symbols loaded.
 	OgreMain.dll!Ogre::SceneManager::createEntity(const std::string & meshName, const std::string & groupName, Ogre::SceneMemoryMgrTypes sceneType) Line 577	C++	Symbols loaded.

What I'm doing: I'm simply trying to use HlmsPbs without any modifications for now (because I can't get my head around creating my own) and I'm simply feeding in diffuse and normal textures. Something like this:

Code: Select all

    Ogre::HlmsTextureManager::TextureLocation texLocation = hlmsTextureManager->
        createOrRetrieveTexture(texName[NiTexturingProperty::Texture_Base], Ogre::HlmsTextureManager::TEXTURE_TYPE_DIFFUSE);

    pbsDatablock->setTexture(Ogre::PBSM_DIFFUSE, texLocation.xIdx, texLocation.texture);
    pbsDatablock->setTextureUvSource(Ogre::PBSM_DIFFUSE, 0); // FIXME: just a guess

Trying the same thing with TEXTURE_TYPE_NORMALS will result in the exception.

All this seems to be beyond my skillset, but I'll try to modify as per your suggestion and see how things go. (EDIT: the suggested code is based on Ogre 2.2 which will make things difficult since I'm still on Ogre 2.1)

[dark_sylinc]

Ahhh!!! 2.1, now that makes sense.

There is no plan on adding BC3 support on Ogre 2.1.
Had you be using Ogre 2.2 it should only be a couple of lines being changed.

You might want to mass-invoke texconv automatically to convert the texture to a format Ogre expects.

[cc9cii]

Hi,

So I made the suggested changes and the exception seems to be gone (yay!)

Code: Select all

diff --git a/Components/Hlms/Pbs/include/OgreHlmsPbs.h b/Components/Hlms/Pbs/include/OgreHlmsPbs.h
index e7122c70d..9429b04b9 100644
--- a/Components/Hlms/Pbs/include/OgreHlmsPbs.h
+++ b/Components/Hlms/Pbs/include/OgreHlmsPbs.h
@@ -491,6 +491,7 @@ namespace Ogre
         static const IdString NormalLa;
         static const IdString NormalRgUnorm;
         static const IdString NormalRgSnorm;
+        static const IdString NormalBc3Unorm;

         static const IdString NormalWeight;
         static const IdString NormalWeightTex;
diff --git a/Components/Hlms/Pbs/src/OgreHlmsPbs.cpp b/Components/Hlms/Pbs/src/OgreHlmsPbs.cpp
index 04243f2ba..2cc873296 100644
--- a/Components/Hlms/Pbs/src/OgreHlmsPbs.cpp
+++ b/Components/Hlms/Pbs/src/OgreHlmsPbs.cpp
@@ -123,6 +123,7 @@ namespace Ogre
     const IdString PbsProperty::NormalLa              = IdString( "normal_la" );
     const IdString PbsProperty::NormalRgUnorm        = IdString( "normal_rg_unorm" );
     const IdString PbsProperty::NormalRgSnorm        = IdString( "normal_rg_snorm" );
+    const IdString PbsProperty::NormalBc3Unorm       = IdString( "normal_bc3_unorm" );

     const IdString PbsProperty::NormalWeight          = IdString( "normal_weight" );
     const IdString PbsProperty::NormalWeightTex       = IdString( "normal_weight_tex" );
@@ -800,6 +801,9 @@ namespace Ogre
             {
                 setProperty( PbsProperty::NormalSamplingFormat, PbsProperty::NormalRgUnorm.mHash );
                 setProperty( PbsProperty::NormalRgUnorm, PbsProperty::NormalRgUnorm.mHash );
+
+                setProperty( PbsProperty::NormalSamplingFormat, PbsProperty::NormalBc3Unorm.mHash );
+                setProperty( PbsProperty::NormalBc3Unorm, PbsProperty::NormalBc3Unorm.mHash );
             }
             //Reserved for supporting LA textures in GLES2.
 //            else
diff --git a/Samples/Media/Hlms/Pbs/Any/Main/800.PixelShader_piece_ps.any b/Samples/Media/Hlms/Pbs/Any/Main/800.PixelShader_piece_ps.any
index c1d767554..068e4518c 100644
--- a/Samples/Media/Hlms/Pbs/Any/Main/800.PixelShader_piece_ps.any
+++ b/Samples/Media/Hlms/Pbs/Any/Main/800.PixelShader_piece_ps.any
@@ -100,6 +100,10 @@
                        //Normal texture must be in RG8_UNORM or similar format!
                        #define getTSNormal( normalMap, samplerState, uv, normalIdx ) reconstructZfromTSNormal( OGRE_SampleArray2D( normalMap, samplerState, uv, normalIdx ).xy * 2.0 - 1.0 )
                @end
+               @property( normal_sampling_format == normal_bc3_unorm )
+                       //Normal texture must be in BC3 or similar format!
+                       #define getTSNormal( normalMap, samplerState, uv, normalIdx ) reconstructZfromTSNormal( OGRE_SampleArray2D( normalMap, samplerState, uv, normalIdx ).yw * 2.0 - 1.0 )
+               @end
                @property( normal_sampling_format == normal_la )
                        //Normal texture must be in LA format!
                        #define getTSNormal( normalMap, samplerState, uv, normalIdx ) reconstructZfromTSNormal( OGRE_SampleArray2D( normalMap, samplerState, uv, normalIdx ).xw * 2.0 - 1.0 )

But I'm seeing this error message from Hlms::collectPieces():

Code: Select all

Error at line 1: @piece 'envSpecularRoughness' already defined

Have I set the properties wrongly in HlmsPbs::calculateHashForPreCreate()? Also, I don't really see any visual differences - is there any way to check that the normals are being applied correctly?

Thanks

[dark_sylinc]

Hi,
So I made the suggested changes and the exception seems to be gone (yay!)

Nice!

But I'm seeing this error message from Hlms::collectPieces():

Code: Select all

Error at line 1: @piece 'envSpecularRoughness' already defined

Have I set the properties wrongly in HlmsPbs::calculateHashForPreCreate()? Also, I don't really see any visual differences - is there any way to check that the normals are being applied correctly?

Thanks

That piece is only defined once in Hlms/Pbs/Any/Main/800.PixelShader_piece_ps.any
I suggest you look in all files for envSpecularRoughness to see if it's defined somewhere else. There could be a loose file.
Or perhaps Hlms/Pbs/Any/Main folder was added twice? (when initializing Hlms)

[cc9cii]

Ah, I left the original (renamed) in the same directory - deleting it got rid of the error message.

However, it's not quite working. I'm not sure how to describe it - it's.. weird. For some meshes I get these exceptions and get a default material which is grey:

Code: Select all

23:39:56: OGRE EXCEPTION(1:InvalidStateException): Renderable can't use normal maps but datablock wants normal maps. Generate Tangents for this mesh to fix the problem or use a datablock without normal maps. in HlmsPbs::calculateHashForPreCreate at C:\Users\cc9c\Dev\src\ogre-next-2.2\Components\Hlms\Pbs\src\OgreHlmsPbs.cpp (line 785)
23:39:56: OGRE EXCEPTION(1:InvalidStateException): Renderable can't use normal maps but datablock wants normal maps. Generate Tangents for this mesh to fix the problem or use a datablock without normal maps. in HlmsPbs::calculateHashForPreCreate at C:\Users\cc9c\Dev\src\ogre-next-2.2\Components\Hlms\Pbs\src\OgreHlmsPbs.cpp (line 785)
23:39:56: Couldn't apply change to datablock '[Hash 0x733abdbe]' for this renderable. Using default one. Check previous log messages to see if there's more information.

But I do generate the tangents using below, and in Ogre 1.10 the normals were working ok. So it seems that there must be other differences with Ogre 2.x.

Code: Select all

unsigned short src, dest;
if (!mesh->suggestTangentVectorBuildParams(Ogre::VES_TANGENT, src, dest))
    mesh->buildTangentVectors(Ogre::VES_TANGENT, src, dest);

[dark_sylinc]

buildTangentVectors is all you need. That has not changed.

I suspect your tangents are being stripped for some (possibly unrelated) reason. If you're converting v1 to v2, make sure the tangents are generated before the conversion.
Also make sure the function is actually getting called before the error.

If you generated the tangents after calling subEntity->setDatablock/setDatablockOrMaterialName you're then modifying the mesh while entities are live and this is not recommended.
If that's the cause, then you may solve the problem by calling subEntity->setDatablock with a dummy datablock then back again with the final datablock; in order to regenerate the hashes and thus the Hlms 'sees' that the mesh now has tangents.

[cc9cii]

Thank you dark_sylinc, it turned out to be a difference at my end. The issue happens for cached materials only - the way to check that a material has normal maps is different for Ogre 2.x.

Cheers,