Few serious questions for Ogre-Next, moving Stunt Rally

[dark_sylinc]

I forgot to answer this one:

BTW, is roughness_map and metalness_map always needed for all layers? I'm getting error if I don't have it in json, if so then what do roughness and metalness values do? I'd prefer to use values, since I don't have them as textures, at least for now.

The way it was written yes; but we provide Terrain as a tutorial rather than a component because when it comes to defining textures on top of the terrain everyone has their own idea of how the blending should happen.
Hence we leave it to users to modify the sources as they need it to get the blending they need/want.

[Crystal Hammer]

Cool. Great, I think I got all Terra params set now from code.
Just curious, what would be the reason for fixed / max 4 layers now? Is it just that 4 from R,G,B,A color of 1 blendmap?
Long ago, in SR with Ogre 1.x we had like a 16 texture limit for 1 pass and that was like: 4 layers * 3 (diff, norm, spec) + 3 shadow maps + something already at maximum. I guess this isn't a problem nowadays? Is it possible to have e.g. max 8 terrain layers now? Are all layers always sampled for each pixel, even if e.g. 1 layer is used?

Is there some inheritance in .material scripts? Like we have in shiny:

Code: Select all

material base_sky
{
    pass
    {
        fog_override true
        lighting off
        ambient 1 1 1
        diffuse 0 0 0 1
        emissive 0 0 0
        texture_unit skyTex
        {
            direct_texture $texture
            create_in_ffp true
            tex_address_mode mirror
        }
    }
}

material sky/dust
{
    parent base_sky
    texture sky_dust.jpg
}

material sky/duskred
{
    parent base_sky
    texture duskred.jpg
}

So with e.g. 20 skies I defined only what changes in each new material.

[dark_sylinc]

Cool. Great, I think I got all Terra params set now from code.
Just curious, what would be the reason for fixed / max 4 layers now? Is it just that 4 from R,G,B,A color of 1 blendmap?

That's basically the reason (simplicity), yes.

Long ago, in SR with Ogre 1.x we had like a 16 texture limit for 1 pass and that was like: 4 layers * 3 (diff, norm, spec) + 3 shadow maps + something already at maximum. I guess this isn't a problem nowadays? Is it possible to have e.g. max 8 terrain layers now? Are all layers always sampled for each pixel, even if e.g. 1 layer is used?

You'll have to modify the shaders / C++ code, but yes you can.

In OpenGL & iOS devices you will be limited to like 31 textures (some of them are taken by stuff like Shadows, etc).

On D3D11 & macOS Metal you have 128 textures available and on Vulkan almost limitless; so you can.

Are all layers always sampled for each pixel, even if e.g. 1 layer is used?

Yes.

Is there some inheritance in .material scripts? Like we have in shiny:

You can use the old material inheritance which works:

Code: Select all

hlms base unlit
{
   depth_check off
}

hlms derived unlit : base
{
    // depth checks will be off because we're deriving it
    texture myDiffuse.png
}

The keyword "abstract" also works.

[Crystal Hammer]

I see. Now I got another batch of questions

Are all layers always sampled for each pixel, even if e.g. 1 layer is used?

Yes.

• So it means having one shader with 8 max layers would be quite wasteful if only 3-4 are used. Even more with triplanar.
  But we have in shiny many shaders made from one, depending on options. E.g. I can set which layers (2 max) have triplanar and only these will sample 3x more, rest just once, w/o triplanar. Shiny has #if and would not add samplers or code blocks etc.
  Is this way also possible in HLMS? Would be also useful to have e.g. 4 shaders, each for 1, 2, 3, or 4 used layers.

• Is material.json more feature rich? Or is everything from material.json also available in .material scripts?

• How can I inherit materials in material.json?

• Can I (and how) set texture mode to warp in .material?
  I use this code now to do it, seems much:

  Code: Select all

  	HlmsSamplerblock sampler;
  	sampler.mMinFilter = FO_ANISOTROPIC;  sampler.mMagFilter = FO_ANISOTROPIC;
  	sampler.mMipFilter = FO_LINEAR;  //?FO_ANISOTROPIC;
  	sampler.mMaxAnisotropy = 16;
  	sampler.mU = TAM_WRAP;  sampler.mV = TAM_WRAP;  sampler.mW = TAM_WRAP;
  	assert( dynamic_cast<Ogre::HlmsPbsDatablock *>( item->getSubItem( 0 )->getDatablock() ) );
  	HlmsPbsDatablock *datablock =
  		static_cast<Ogre::HlmsPbsDatablock *>( item->getSubItem( 0 )->getDatablock() );
  	datablock->setSamplerblock( PBSM_DIFFUSE, sampler );
  	datablock->setSamplerblock( PBSM_NORMAL, sampler );
  

  Or could I set all datablocks to wrap by default?

• Can I toggle wireframe rendering for everything on scene easily, or do I need to iterate through all datablocks or something?

• How can I set some LOD bias factor globally (as user option), e.g. to allow more quality for further distance (more than is in .mesh files).

• Can I change LOD distance values for .mesh files when I load them in code? Or would I need to clone the mesh? Using MeshTool again, seems a bit much to change them. I guess each fern, tree etc. would need some own tweaking.

• How do I add tangents in Mesh (the new v2) created in code?
  Before in Ogre 1.x I did:

  Code: Select all

  	unsigned short src, dest;
  	if (!mesh->suggestTangentVectorBuildParams(VES_TANGENT, src, dest))
  		mesh->buildTangentVectors(VES_TANGENT, src, dest);
  

  Hope there is something similar now too. I'm getting a lot of these:

  Code: Select all

  23:12:20: OGRE EXCEPTION(1:InvalidStateException): Renderable can't use normal maps but datablock wants normal maps. Generate Tangents for this mesh to fix the problem or use a datablock without normal maps. in HlmsPbs::calculateHashForPreCreate at ../../Components/Hlms/Pbs/src/OgreHlmsPbs.cpp (line 933)
  

• What does this mean in Ogre.log?

  Code: Select all

  23:12:20: WARNING: Mesh2 instance 'rd.mesh.101_0' was defined as manually loaded, but no manual loader was provided. This Resource will be lost if it has to be reloaded.

  Is it bad? Will it be gone forever on device reset or something? How to fix it?

• Is it easy to add height fog to Atmosphere? Can I just override something and add one shader part, or would I need to copy whole Atmosphere code and extend?

[Crystal Hammer]

Meh, so I found a way on forum, to have tangents on v2 Mesh. Which is by importing to v1 mesh, building them old way and importing back to v2.
Seriously? Is it still so, after 6 years since this and that post
Here's the code, having a v2 MeshPtr mesh, called sMesh:

Code: Select all

	v1::MeshPtr m1 = v1::MeshManager::getSingleton().create("v1"+sMesh, "General");
	m1->importV2(mesh.get());
	unsigned short src, dest;
	m1->buildTangentVectors(VES_TANGENT, src, dest);
	mesh->importV1(m1.get(), false, false, false);

Wish I knew earlier, I wouldn't convert my code to v2 then I'd save time doing 1 import back.
But honestly, feels like flying from Paris to Rome through Sydney.

[Crystal Hammer]

Would it be possible to get some more meaningful warnings?

Code: Select all

17:30:00: Vertex Shader: 700000912VertexShader_vs
Fragment Shader: 700000912PixelShader_ps
 GLSL validation result : 
active samplers with a different type refer to the same texture image unit

17:30:00: OGRE EXCEPTION(1:InvalidStateException): Renderable can't use normal maps but datablock wants normal maps. Generate Tangents for this mesh to fix the problem or use a datablock without normal maps. in HlmsPbs::calculateHashForPreCreate at ../../Components/Hlms/Pbs/src/OgreHlmsPbs.cpp (line 933)
17:30:00: Couldn't apply datablock '[Hash 0xee1d71a0]' to this renderable. Using default one. Check previous log messages to see if there's more information.

Isn't it possible to have some maps from original string names to hashes and show names here? Maybe also for shaders?

What does this mean and what causes it?

Code: Select all

active samplers with a different type refer to the same texture image unit

[dark_sylinc]

• So it means having one shader with 8 max layers would be quite wasteful if only 3-4 are used. Even more with triplanar.
  But we have in shiny many shaders made from one, depending on options. E.g. I can set which layers (2 max) have triplanar and only these will sample 3x more, rest just once, w/o triplanar. Shiny has #if and would not add samplers or code blocks etc.
  Is this way also possible in HLMS? Would be also useful to have e.g. 4 shaders, each for 1, 2, 3, or 4 used layers.

Yes it's possible. Hlms has @property your code @end akin to #if #endif

But it depends on what you want do and how. Do you want to have to portions of the terrain with different settings? That'd be hard, because that needs two different shaders and that means they can't be merged into one single drawcall, something Terra assumes and does aggressively.

- Is material.json more feature rich? Or is everything from material.json also available in .material scripts?

• Can I (and how) set texture mode to warp in .material?

JSON has all settings, while .material scripts may not have all of them.
HlmsTerraDatablock parses the string looking for most options.

Manipulating texture modes is notably absent from .material scripts.

- How can I inherit materials in material.json?

There isn't one. It's regular JSON so there is no way to inherit. JSON are more suited for being edited by tools and then use HlmsManager::saveMaterials, rather than edited by hand.

On Colibri I implemented "copy_from".

I guess the same can be done using HlmsDatablock::clone or HlmsDatablock::cloneImpl

- Can I toggle wireframe rendering for everything on scene easily, or do I need to iterate through all datablocks or something?

I'm afraid you have to iterated through all datablocks and set their HlmsMacroblock::mPolygonMode.

- How can I set some LOD bias factor globally (as user option), e.g. to allow more quality for further distance (more than is in .mesh files).

See Camera::setLodBias

- How do I add tangents in Mesh (the new v2) created in code?

As you found out, converting it to v1, adding tangents then back to v2 is the only solution so far.

Wish I knew earlier, I wouldn't convert my code to v2 then I'd save time doing 1 import back.

v2 does have performance improvements over v1, so it wasn't for nothing

But honestly, feels like flying from Paris to Rome through Sydney.

Agreed, but since no one seems to need high performance tangent generation, no one has bothered to create one and the v2 -> v1 -> v2 is 'just fine' for most people.

I did write a very crude tangent generator that is designed to work with v2 streams, but it doesn't cover all edge cases like v1 does.

- What does this mean in Ogre.log?

Code: Select all

23:12:20: WARNING: Mesh2 instance 'rd.mesh.101_0' was defined as manually loaded, but no manual loader was provided. This Resource will be lost if it has to be reloaded.

Is it bad? Will it be gone forever on device reset or something? How to fix it?

Yeah, it's about device reset.

If you created it via MeshManager::createByImportingV1, that warning should not appear, as MeshManager::loadResource will take care of reloading it.

It basically calls createManual( ..., this ) where this is a ManualResourceLoader that gets called whenever the mesh needs to be reloaded.

• Is it easy to add height fog to Atmosphere? Can I just override something and add one shader part, or would I need to copy whole Atmosphere code and extend?

The Hlms shader code is in Samples/Media/Hlms/Pbs/Any/Atmosphere and the one for drawing the sky is in Samples/Media/2.0/scripts/materials/Common/Any/AtmosphereNprSky_ps.any (and relevant AtmosphereNprSky_ps.glsl, *.hlsl, etc for language-specific code)

Changing both should do what you need. I may be interested in such changes btw.

[dark_sylinc]

What does this mean and what causes it?

Code: Select all

active samplers with a different type refer to the same texture image unit

That warning can be ignored, it is produced by the driver.

Basically (due to how Ogre hisotrically worked) all samplers and textures are initially set to 0, then we check for errors, then we set them to user parameters (which assigns values like 0, 1, 2, 3, etc).

So when the driver sees that one variable is texture2D myTexture; and another is textureCube anotherTexture; but both variables refer to texture unit 0, it complains since this is invalid.

However before we reach rendering, they will (should) be set to the right texture unit numbers.

Only OpenGL backend has this problem.

[Crystal Hammer]

Okay. Thank you for answers again. I'm moving forward much faster.
Well all those material limits make me think I should write some simple parser and do that in code then. I guess I got used to shiny.
Got a couple of new issues.

• Can I anyhow get only fog from Atmosphere and nothing else?
  I can't change sun light colors if Atmosphere is there and sets it. Also I realized that I get a freaking dull gray color with fresnel covering like half of my terrain. I can't even. I found out it is exactly the skyColour and it destroys diffuse texture from terrain. So e.g. if I set it to R,G,B 0,1,0 then shadows and fresnel (low angles) on terrain are just shades of green, no diffuse texture colors at all. Without Atmosphere colors are normal, also in shadows and I don't have fresnel.
• Can I somehow achieve smooth alpha borders on road with a detail map?
  I got hard edges, but whatever blend for detail I tried, it never changes alpha. Maybe I'm doing something wrong, or does this need a custom shader?
  Maybe per vertex colors if not using texture, but that's less detail I think.

[Crystal Hammer]

Well would be good for me to either get only fog from Atmosphere or undo what it is doing with that fresnel stuff on terrain and all, maybe it wants GI or probes or something but now it's not the time for me to try that.

I think I found some way of having alpha borders on road, still looks bad though.
Where can I find what all those blend modes do in shader? E.g. PBSM_BLEND_NORMAL_NON_PREMUL
It's not exactly what I needed so I'm anyway thinking of having own pieces in shader. And blending factors for detail maps in vertex colors seems like a good idea for road. We actually had many custom parameters in shiny shaders.

Any advice on how to extend or start own .material file? Would I need to create all materials or have some listener and create only those needed? Or create a material.json file from my own? I'm not sure about this, but the .material file limits are bad for me and .json seems too much (to write hundreds of materials in it without inheriting). Also not having wrapping textures by default.

Is there anything special going on with transparent materials now? We have a lot of glass pipes on some tracks (e.g. here). And in old Ogre they were sometimes blinking a lot, order of rendering seemed random each frame. But maybe not every time I started. Are meshed sorted now? Particles were sorted.
Is Order Independent Transparency the only solution for this? But surely way too advanced for me to implement. I'm guessing it'd be difficult, right?

I made some basic grass recently with pages and just v2 meshes. Not very slow but slow anyway since I do everything on CPU 1 core and using Real to compute every vertex in grass patch. Do I guess right that I should only have points (with some extra data) and the rest (4 vertices, UV, normal) would be created from it by geometry shader, similar to particles? I think I'll leave it for much later anyway.

Is there any sample/tutorial close to what's needed for soft particles?
I think I'd use the same for water fading near terrain etc.

[Crystal Hammer]

Okay I've found what's wrong, it's not Atmosphere, its code isn't too complicated.
It is simply mSceneMgr->setAmbientLight( and it looks good only if I set both colors the same.
But if those are different (like mostly set by Atmosphere too), I get bad gray spots on one side (like fresnel) which disregard diffuse texture colors completely.

Code: Select all

sun->setPowerScale( 1.f );
sun->setDiffuseColour( sc->lDiff.GetClr() * 5.f );
sun->setSpecularColour(sc->lSpec.GetClr() * 3.f );
app->mSceneMgr->setAmbientLight(
	sc->lAmb.GetClr() * 1.5f,
	sc->lAmb.GetClr() * 1.5f,
	-dir);

Here's my code, the values coming from sc-> are old SR light values and are in 0..1.
This code with such multipliers is what makes it look good again, close to what it was in old SR.
Anything different here

Code: Select all

	sc->lAmb.GetClr() * 1.5f,
	sc->lAmb.GetClr() * 1.f,

makes it really bad:

19_01-13-13`ss.jpg

Can you tell me why or if I'm doing something wrong?
Update: okay, I have some solution, calling setAmbientLight with my values every frame update. I have fog from Atmosphere, my light values and no gray fresnel.

[dark_sylinc]

Since I don't know what I should be looking at (a big red circle could help), what data is being fed and what you expect it to look like, I can't really help I'm afraid.

I can only advise to try to strip of everything (i.e. no sun light, no nothing) and start adding little by little until you find something you don't expect and ask about it.

Where can I find what all those blend modes do in shader? E.g. PBSM_BLEND_NORMAL_NON_PREMUL

See Samples/Media/Hlms/Pbs/Any/Main/200.BlendModes_piece_ps.any

Are meshed sorted now?

They are, but it is a "rough" sort so it probably won't fix your problem.

You can use MovableObject::setRenderQueueGroup and Renderable::setRenderQueueSubGroup to explicitly control order though.

Is Order Independent Transparency the only solution for this? But surely way too advanced for me to implement

Probably yes, and yes.

I just saw the picture of the mesh you are talking about... yeah that's a nightmare to render. And a prime candidate for OIT.

Most games fix this problem by... not having this type of transparency (a classic joke is "the glitched glass of water was replaced by... a mug").

Any advice on how to extend or start own .material file? Would I need to create all materials or have some listener and create only those needed? Or create a material.json file from my own? I'm not sure about this, but the .material file limits are bad for me and .json seems too much (to write hundreds of materials in it without inheriting). Also not having wrapping textures by default.

Given your unique case of having an existing project with lots of already-existing data, I'd advise to create a few "templates" in JSON or material scripts, and then at init time from C++ you go through all your materials and create them programmatically based on those templates, populating the data you need.

I did something similar with another project, but in my case an offline tool I wrote converts the data from the old files, converts it, and finally dumps it as JSON via saveAllMaterials for the game to consume.

[Lax]

Hi Crystal Hammer,

Okay I've found what's wrong, it's not Atmosphere, its code isn't too complicated.
It is simply mSceneMgr->setAmbientLight( and it looks good only if I set both colors the same.

Cool thanks for finding that out. I struggled with the same issue and now everything looks much better.
I also set each frame my already set ambient light values.

Best Regards
Lax

[Crystal Hammer]

Cool.

So I got an assert in Debug,

Code: Select all

 ../../RenderSystems/GL3Plus/src/OgreGL3PlusHardwareVertexBuffer.cpp:86: virtual void* Ogre::v1::GL3PlusHardwareVertexBuffer::lockImpl(size_t, size_t, Ogre::v1::HardwareBuffer::LockOptions): Assertion `( ( mUsage & HBU_WRITE_ONLY && options != HBL_NORMAL && options != HBL_READ_ONLY ) || !( mUsage & HBU_WRITE_ONLY ) ) && "Reading from a write-only buffer! Create the buffer without HBL_WRITE_ONLY bit"' failed.

callstack:

Code: Select all

__GI_raise (@raise:51)
__GI_abort (@abort:64)
__assert_fail_base (@_nl_load_domain.cold:3)
__GI___assert_fail (@7ffff6be9662..7ffff6be96eb:3)
Ogre::v1::GL3PlusHardwareVertexBuffer::lockImpl(unsigned long, unsigned long, Ogre::v1::HardwareBuffer::LockOptions) (/home/ch/_sr/og3/Ogre/ogre-next/RenderSystems/GL3Plus/src/OgreGL3PlusHardwareVertexBuffer.cpp:86)
Ogre::v1::HardwareBuffer::lock(unsigned long, unsigned long, Ogre::v1::HardwareBuffer::LockOptions) (/home/ch/_sr/og3/Ogre/ogre-next/OgreMain/include/OgreHardwareBuffer.h:218)
Ogre::v1::HardwareBuffer::lock(Ogre::v1::HardwareBuffer::LockOptions) (/home/ch/_sr/og3/Ogre/ogre-next/OgreMain/include/OgreHardwareBuffer.h:230)
Ogre::v1::HardwareBufferLockGuard::lock(Ogre::v1::HardwareBuffer*, Ogre::v1::HardwareBuffer::LockOptions) (/home/ch/_sr/og3/Ogre/ogre-next/OgreMain/include/OgreHardwareBuffer.h:412)
void Ogre::v1::HardwareBufferLockGuard::lock<Ogre::v1::HardwareVertexBuffer>(Ogre::SharedPtr<Ogre::v1::HardwareVertexBuffer> const&, Ogre::v1::HardwareBuffer::LockOptions) (/home/ch/_sr/og3/Ogre/ogre-next/OgreMain/include/OgreHardwareBuffer.h:426)
Ogre::SubMesh::_arrangeEfficient(Ogre::v1::SubMesh*, bool, bool, bool, std::vector<Ogre::VertexElement2, std::allocator<Ogre::VertexElement2> >*, unsigned long) (/home/ch/_sr/og3/Ogre/ogre-next/OgreMain/src/OgreSubMesh2.cpp:1071)
Ogre::SubMesh::importBuffersFromV1(Ogre::v1::SubMesh*, bool, bool, bool, bool, unsigned long) (/home/ch/_sr/og3/Ogre/ogre-next/OgreMain/src/OgreSubMesh2.cpp:552)
Ogre::SubMesh::importFromV1(Ogre::v1::SubMesh*, bool, bool, bool, bool) (/home/ch/_sr/og3/Ogre/ogre-next/OgreMain/src/OgreSubMesh2.cpp:529)
Ogre::Mesh::importV1(Ogre::v1::Mesh*, bool, bool, bool, bool) (/home/ch/_sr/og3/Ogre/ogre-next/OgreMain/src/OgreMesh2.cpp:548)
Ogre::MeshManager::loadResource(Ogre::Resource*) (/home/ch/_sr/og3/Ogre/ogre-next/OgreMain/src/OgreMeshManager2.cpp:174)
Ogre::Resource::load(bool) (/home/ch/_sr/og3/Ogre/ogre-next/OgreMain/src/OgreResource.cpp:214)
Ogre::MeshManager::load(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, Ogre::BufferType, Ogre::BufferType, bool, bool) (/home/ch/_sr/og3/Ogre/ogre-next/OgreMain/src/OgreMeshManager2.cpp:111)
Ogre::ItemFactory::createInstanceImpl(unsigned int, Ogre::ObjectMemoryManager*, Ogre::SceneManager*, Ogre::StdMap<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const*) (/home/ch/_sr/og3/Ogre/ogre-next/OgreMain/src/OgreItem.cpp:369)
Ogre::MovableObjectFactory::createInstance(unsigned int, Ogre::ObjectMemoryManager*, Ogre::SceneManager*, Ogre::StdMap<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const*) (/home/ch/_sr/og3/Ogre/ogre-next/OgreMain/src/OgreMovableObject.cpp:925)
Ogre::SceneManager::createMovableObject(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, Ogre::ObjectMemoryManager*, Ogre::StdMap<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const*) (/home/ch/_sr/og3/Ogre/ogre-next/OgreMain/src/OgreSceneManager.cpp:4103)
Ogre::SceneManager::createItem(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, Ogre::SceneMemoryMgrTypes, bool) (/home/ch/_sr/og3/Ogre/ogre-next/OgreMain/src/OgreSceneManager.cpp:535)

Can't use Debug now, with my road meshes created manually, and v1 used for tangents.

Code: Select all

	String s1 = sMesh+"v1", s2 = sMesh+"v2";
	v1::MeshPtr m1 = v1::MeshManager::getSingleton().create(s1, "General");
	m1->importV2(mesh.get());
	m1->buildTangentVectors();
	mesh = MeshManager::getSingleton().createByImportingV1(s2, "General", m1.get(), false,false,false);
	MeshManager::getSingleton().remove(sMesh);  // not needed

Works in Release, no problem. I wonder how to fix this. And if it will change look.

[dark_sylinc]

Hi, this happens because the mesh was created with buffers that imply "write only" (HBU_WRITE_ONLY), hence when importing it we need to read it (which can't be done).

Thus make sure to pass Ogre::v1::HardwareBuffer::HBU_STATIC to v1::MeshManager::getSingleton().create for both vertex and index buffers (see V2MeshGameState::createScene01 in Samples/2.0/ApiUsage/V2Mesh/V2MeshGameState.cpp).

[Crystal Hammer]

Okay, but:
MeshPtr create(
doesn't have these args,
createOrRetrieve(
returns some ResourceCreateOrRetrieveResult
which I can't seem cast to MeshPtr
Next:
MeshPtr prepare(
and
MeshPtr load(
have such args but they want to load, and I just need to create, before
m1->importV2

[Crystal Hammer]

Okay I got a new very strange issue, made a video of it.
Terrain is changing texture to some random, I only know the 1st one and 2 I have never seen before.
This changes from camera position and rotation, and is even different in dynamic cubemap cameras.
All I did is tblock->setTexture( TERRA_DETAIL0, mTerra->m_blendMapTex );
and HlmsTerraDatablock *tblock = static_cast<HlmsTerraDatablock *>( datablock );
from datablock = mGraphicsSystem->hlmsTerra->createDatablock(

If I remove this line with setTexture it works, if I put e.g. "HeightmapBlendmap.png" there it works fine too.
m_blendMapTex is a manually created texture, and was created almost the same way as m_normalMapTex inside createNormalTexture(), with TextureFlags::ManualTexture and setPixelFormat( PFG_RGBA8_UNORM );
I replaced all Normal to blend blendMap everywhere.
Best thing is it all works. If I save m_blendMapTex to file, I see it as red and it should just result in terrain using 1st detail texture everywhere.
Any hints on what is this magic sorcery?

[dark_sylinc]

Most textures in OgreNext have AutomaticBatching flag applied.

This means that internally a 2D TextureGpu belongs to another 2DArray TextureGpu that acts as a "pool" owner; and each 2D TextureGpu is just a slice to that array/pool.
i.e. we pretend the texture is 2D, even though it's not.

This allows us to batch many materials into the same draw call (because we don't have to switch TextureGpu, internally it's always the same TextureGpu pool).

The texture you created manually must not be of 2DArray type (i.e. a 2DArray of 1 slice) and the shader is not prepared to expect that.

You can either use a 2DArray type (i.e. create a 2DArray with just 1 slice); or create it with the AutomaticBatching flag (recommended for maximum perf). To populate AutomaticBatching textures from RAM, see any of the following samples:

• AreaApproxLightsGameState::createAreaMask in Samples/2.0/ApiUsage/AreaApproxLights/AreaApproxLightsGameState.cpp

• UpdatingDecalsAndAreaLightTexGameState::setupLightTexture in Samples/2.0/ApiUsage/UpdatingDecalsAndAreaLightTex/UpdatingDecalsAndAreaLightTexGameState.cpp

• TestUtils::generateRandomBlankTextures in Samples/2.0/Common/src/Utils/TestUtils.cpp

There's basically 2 ways to populate them: asynchronously (the texture will be white until we're ready memcpy'ing from it) and synchronously

Cheers

[Crystal Hammer]

Thanks for info.
Is there any difference for my case though, when I will be filling texture only by GPU, from a shader in a workspace (not at all by CPU)? Just like normalMap in Terra is done by GpuNormalMapper.
In previous SR and Ogre I just had an RTT texture used as blendmap. Was good for editor, where it needs to be updated every frame when editing terrain (and CPU is busy with heightmap array). So no need to copy or anything and GPU does it faster.
Can I do the same?

[dark_sylinc]

Ah if you need the texture to be a RTT then it's best to not create it as AutomaticBatching, and just create it as a Texture2DArray.

Either with one slice, or many slices. Yyou CAN render to each slice; though which slice is set at RTV level in the compositor:

via scripts:

Code: Select all

compositor_node DiffuseNode
{
    in 0 RenderTarget

// Modify automatically generated RTV
rtv RenderTarget
{
    slice 5 // Draw to slice 5
}

target RenderTarget
{
    pass render_scene
    {
    }
}

out 0 RenderTarget
}

Or via code:

Code: Select all

compositorNodeDef->addTargetPass( renderTargetName, sliceIdx );

Though first try with a sliceCount of 1. Once you get that working, check if it's actually a problem to the number of draw calls for the terrain. If it's not, then don't bother with this.

[Crystal Hammer]

Cool. It works. Strangely now even without the rtv slice, but I kept it.

I still got some weird issue with texture wrap (I can't believe how much time I need for such simple thing).

Happens for v2 meshes I created for flat fluids (v2 createByImportingV1, from v1 createPlane) and for road segments (made v2, then to v1 and back to v2).
Here is a video for fluids and for road (glass pipes).

Everything works normal (textures always wrap) the first time, so after start. But any next track I load will have this issue on materials (datablocks or textures IDK) which already existed. So I think new materials will be okay, but same materials will not have texture wrap and it also depends on camera position, rotation (similar to my older issue on video).

Any idea what's wrong? I think I should do some destroy all for materials or datablocks? or something like that.
In old Ogre and with shiny I did this on map destroy:

Code: Select all

	MeshManager::getSingleton().unloadUnreferencedResources();
	sh::Factory::getInstance().unloadUnreferencedMaterials();
	Ogre::TextureManager::getSingleton().unloadUnreferencedResources();

I also see GPU RAM always increasing on each new map loaded (gets much higher than if I'd load only single map). Previous maps load faster but I see GPU RAM not being freed.
How to fix this in Ogre-Next?

[dark_sylinc]

Cool. It works. Strangely now even without the rtv slice, but I kept it.

I still got some weird issue with texture wrap (I can't believe how much time I need for such simple thing).

Happens for v2 meshes I created for flat fluids (v2 createByImportingV1, from v1 createPlane) and for road segments (made v2, then to v1 and back to v2).
Here is a video for fluids and for road (glass pipes).

Everything works normal (textures always wrap) the first time, so after start. But any next track I load will have this issue on materials (datablocks or textures IDK) which already existed. So I think new materials will be okay, but same materials will not have texture wrap and it also depends on camera position, rotation (similar to my older issue on video).

That looks more like either incorrect creation of a Samplerblock (or any other of the *blocks) or straight using a dangling pointer.
If Ogre.log isn't saying anything useful, I strongly suggest using Valgrind & ASAN (Valgrind can detect more bugs than ASAN, and remember to unset "hide external errors"):

Code: Select all

if( APPLE )
	set( CMAKE_XCODE_GENERATE_SCHEME ON )
	set( CMAKE_XCODE_SCHEME_ADDRESS_SANITIZER ON )
	set( CMAKE_XCODE_SCHEME_ADDRESS_SANITIZER_USE_AFTER_RETURN ON )
elseif( UNIX )
	set( CMAKE_C_FLAGS
		"${CMAKE_C_FLAGS} -fno-omit-frame-pointer -fno-optimize-sibling-calls -fsanitize=address" )
	set( CMAKE_CXX_FLAGS
		"${CMAKE_CXX_FLAGS} -fno-omit-frame-pointer -fno-optimize-sibling-calls -fsanitize=address" )
	set( CMAKE_LINKER_FLAGS "${CMAKE_LINKER_FLAGS} -fsanitize=address" )
elseif( MSVC )
	set( CMAKE_C_FLAGS "${CMAKE_C_FLAGS_DEBUG} /fsanitize=address" )
	set( CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS_DEBUG} /fsanitize=address" )
endif()

Do you have a link to the code where you set the texture samplers to Samplerblock.

Are you running on Debug or Release? OgreNext in Debug mode performs validation checks that Release doesn't.

Additionally, running on Vulkan (instead of GL3+) with Validation enabled (install VulkanSDK and set environment variables VULKAN_SDK, LD_LIBRARY_PATH and VK_LAYER_PATH as shown in setup-env.sh from the VulkanSDK) may show more information as well.

In old Ogre and with shiny I did this on map destroy:

Code: Select all

	MeshManager::getSingleton().unloadUnreferencedResources();
	sh::Factory::getInstance().unloadUnreferencedMaterials();
	Ogre::TextureManager::getSingleton().unloadUnreferencedResources();

I also see GPU RAM always increasing on each new map loaded (gets much higher than if I'd load only single map). Previous maps load faster but I see GPU RAM not being freed.
How to fix this in Ogre-Next?

In Samples/2.0/Tutorials/Tutorial_Memory/Tutorial_MemoryGameState.cpp:

• See MemoryGameState::unloadTexturesFromUnusedMaterials

• See MemoryGameState::unloadUnusedTextures

• See MemoryGameState::minimizeMemory

[Crystal Hammer]

Thanks, it works now.
I added code from Tutorial_MemoryGameState.cpp, and a call to minimizeMemory() without setTightMemoryBudget(); when I destroy all stuff from previous track.
I now see low GPU RAM values, and same values each time same track is loaded. Also after each loading all materials are gray-white, then textures are loaded and all looks good.
I'm glad I didn't have to debug this.

Can I write shaders only using .any files?
I see e.g. for Terra there are .any files but also: e.g. Media/Hlms/Terra/HLSL/PixelShader_ps.hlsl and Hlms/Terra/GLSL/PixelShader_ps.glsl
just using pieces from those .any files.
Or is it nowadays too complicated to have everything generated automatically from just 1 file.
E.g. could Terrain/GLSL/GpuNormalMapper_ps.glsl be written in just 1 .any file or does it need to be both glsl and hlsl?
I was using shiny and it was generating .hlsl and .glsl files from just 1 .shader file.

So is my thinking right for soft particles: I would have to render separately, depth only, the whole scene without particles. Then when rendeing particles use this depth (with also current depth?) and fade if it's close?
Is Tutorial_ReconstructPosFromDepth close to this or not really?

[Crystal Hammer]

I have a new problem. I started restoring my HUD. So we had gauges, minimap, graphs and much more done like so:
ManualObject created in 2D (x,y are always between -1..1) and this was done so:

Code: Select all

	ManualObject* m = mSceneMgr->createManualObject();
	m->setUseIdentityProjection(true);
	m->setUseIdentityView(true);

But I see there are no such Identity methods now? I also found this related topic
and another - Is that the only way we should we do 2D HUD elements now?
Or by multiplying 2D coords to be in 3D with some inverse matrix (if so can you tell how to get it from Camera)?
Or maybe completely different option by doing HUD as postprocess with pixel shader but this seems bit complicated (e.g. demo).

[dark_sylinc]

Code: Select all

	ManualObject* m = mSceneMgr->createManualObject();
	m->setUseIdentityProjection(true);
	m->setUseIdentityView(true);

But I see there are no such Identity methods now? I also found this related topic

They still exist, they were just moved to Renderable. The Hlms implementation is free to ignore this request, but HlmsUnlit will honour setUseIdentityProjection and when that's set it will give you both identity view and proj matrices, which is what you need for 2D rendering.

I'm more concerned about ManualObject though. You can use v1 (which could cause stalls due to the old buffer interface) or the new one which was a community contribution but users have reported a few bugs.

IIRC it only works well if you reconstruct each MO from scratch every time you need to modify it, or if you immediately turn it into a Mesh.

However considering your use case, I'm wondering if you should use Colibri's CustomShape class living in the CustomShape branch, which seems to have the functionality you need.

I created that branch so a colleague could implement a RadarChart (which he is still slowly working on it...) based on it.

Or maybe completely different option by doing HUD as postprocess with pixel shader but this seems bit complicated (e.g. demo).

Yeah, that's hard and overkill. Very cool though.