New InstanceManager: Instancing done the right way

Discussion area about developing or extending OGRE, adding plugins for it or building applications on it. No newbie questions please, use the Help forum for that.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5429
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1337

Re: New InstanceManager: Instancing done the right way

Post by dark_sylinc »

Hi, when all instances are at 0, 0, 0; your shader isn't reading the world matrices correctly. (or you're not using a shader at all, which I doubt considering you)
What technique are you using? can you show a small snippet of the vertex shader?
User avatar
AshMcConnell
Silver Sponsor
Silver Sponsor
Posts: 605
Joined: Fri Dec 14, 2007 11:44 am
Location: Northern Ireland
x 16

Re: New InstanceManager: Instancing done the right way

Post by AshMcConnell »

dark_sylinc wrote:Hi, when all instances are at 0, 0, 0; your shader isn't reading the world matrices correctly. (or you're not using a shader at all, which I doubt considering you)
What technique are you using? can you show a small snippet of the vertex shader?
I fear you have underestimated my stupidity. I'm an idiot, I forgot to use the shader! :)

Thanks, knew it would be a silly mistake - my only consolation is that hopefully this post will help other googlers as silly as me :). I am using HW_Basic with the static flag set btw.

Thanks!
Ash
jessome
Gnoblar
Posts: 11
Joined: Wed May 15, 2013 5:32 pm

Re: New InstanceManager: Instancing done the right way

Post by jessome »

Okay new question:

Code: Select all

      uInt16 flags = Ogre::IM_USE16BIT | Ogre::IM_VTFBESTFIT | Ogre::IM_USEBONEDUALQUATERNIONS | Ogre::IM_VTFBONEMATRIXLOOKUP;
      Ogre::InstancedEntity* referenceEntity = nullptr;
		for( uInt32 i = 0; i < mesh->getNumSubMeshes(); ++i )
		{
			mgrs.push_back( mpSceneManager->createInstanceManager( imName.str(), meshName, group,
																	Ogre::InstanceManager::HWInstancingVTF,
																	50, flags, i ) );
			mgrs.back()->setMaxLookupTableInstances( mesh->getNumAnimations() );
			auto ent = mgrs.back()->createInstancedEntity( materialName );
			if( i == 0 ) referenceEntity = ent;
			else if( !ent->shareTransformWith( referenceEntity ) )
			{
				ASSERT( false, "Something went wrong" );
			}
      }
Okay, so, I used the code on page 1 of this thread to try and setup my animated mesh that has a bunch of submeshes. I think logically this code is trying to say 'do what he's doing' so that the animation is consistent between sub-meshes.

However, that assert trips when I create the second submesh because it's already had it's transform shared with other submeshes of the same kind. An artifact of the LookUpTable flag option? How do I address this issue?
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5429
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1337

Re: New InstanceManager: Instancing done the right way

Post by dark_sylinc »

jessome wrote:mgrs.back()->setMaxLookupTableInstances( mesh->getNumAnimations() );
setMaxLookupTableInstances is a call from an enhancement that Mattan Furst did (not written by me); and, if you see the sample, is used to display a lot of animated entities by reusing/faking animations (i.e. best for crowds)

Unless you actually want the LUT (Look Up Table) functionality he wrote (which I don't know if it's compatible with sharing with other instances... I did not write the lut code) don't call that function.
uInt16 flags = Ogre::IM_USE16BIT | Ogre::IM_VTFBESTFIT | Ogre::IM_USEBONEDUALQUATERNIONS | Ogre::IM_VTFBONEMATRIXLOOKUP
By setting IM_VTFBONEMATRIXLOOKUP you're also requesting LUT. Are you sure you want it?
By setting IM_USEBONEDUALQUATERNIONS, you have to code the shader to use dual quaternions. Are you sure you want this?

Cheers
Matías
User avatar
AshMcConnell
Silver Sponsor
Silver Sponsor
Posts: 605
Joined: Fri Dec 14, 2007 11:44 am
Location: Northern Ireland
x 16

Re: New InstanceManager: Instancing done the right way

Post by AshMcConnell »

Apologies for the delay, but I managed to get a bit more time on my instancing code tonight. I'm was having a problem with the world matrices (after I remembered to use the shader), so I decided to go back and use the exact shaders that are used in the New Instancing sample. Here is the code I am using to create instance managers, it is a bit hard coded for the moment and based on the name of the entity (boo!). There are a number of trees with the entity name in the format : - [type]_[numofpolys]_tree[treeindex], e.g. 01_4_tree134

Code: Select all

	std::tr1::regex rx("(\\d+)_\\d+_tree\\d+");

	std::string entityName = entity->getName();
	std::tr1::cmatch res;

	if (std::tr1::regex_search(entityName.c_str(), res, rx)){
		Ogre::InstanceManager *treeInstanceManager;

		std::string instanceManagerName = string("treeInstanceManager") + string(res[1]);
													
		if (sceneManger->hasInstanceManager(instanceManagerName)){
			treeInstanceManager = sceneManger->getInstanceManager(instanceManagerName);
		}else{
			treeInstanceManager = sceneManger->createInstanceManager(instanceManagerName, entity->getMesh()->getName(), Ogre::ResourceGroupManager::AUTODETECT_RESOURCE_GROUP_NAME, Ogre::InstanceManager::HWInstancingBasic, 50, 0);
		}

		LOGD << "Material Name:" << entity->getSubEntity(0)->getMaterialName();

 		Ogre::InstancedEntity *instancedEntity = sceneManger->createInstancedEntity(entity->getSubEntity(0)->getMaterialName(), instanceManagerName);
		Ogre::SceneNode *sn = sceneManger->getRootSceneNode()->createChildSceneNode(entity->getParentSceneNode()->_getDerivedPosition(), entity->getParentSceneNode()->_getDerivedOrientation());
		sn->attachObject(instancedEntity);


		removeEntity = true;
	}

One of the tree materials looks like this: -

Code: Select all

material TREES01 : Examples/Instancing/HWBasic
{
	set_texture_alias     DiffuseMap    trees01.dds
}
Here is how it looks: -

Image

Any ideas what could be going on? To be honest it's a few years since I did any shader work (and not an awful lot back then). Any suggestions on how to debug would be appreciated.

Thanks Guys!
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5429
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1337

Re: New InstanceManager: Instancing done the right way

Post by dark_sylinc »

Thanks for reminding me why I'm writting the HLMS... stuff like this will be part of the past.

As for your problem, normally it's because the mesh has multiples UVs (or some exotic element in the vertex buffer) so the shader reads from TEXCOORD1, 2 & 3; when it should be reading from TEXCOORD2, 3 & 4 (assuming it has 2 UVs instead of 1).

Stuff like this gets spot in a minute when looking at the mesh' vertex declaration in Ogre Meshy (Info Tab-> Geometry->Submesh N->Vertex Declaration). If you can post pictures of the fully expanded tree of the vertex declaration I can see if something looks odd.

Another problem could be around row major vs column major compiled shaders.

I assume this is Direct3D9?
User avatar
AshMcConnell
Silver Sponsor
Silver Sponsor
Posts: 605
Joined: Fri Dec 14, 2007 11:44 am
Location: Northern Ireland
x 16

Re: New InstanceManager: Instancing done the right way

Post by AshMcConnell »

Thanks Matias, sounds like the problem indeed. I will attempt to grab some time tonight to get to the bottom of the problem. I am using D3D9.

Your HLMS stuff looks great, quite excited about it!
User avatar
AshMcConnell
Silver Sponsor
Silver Sponsor
Posts: 605
Joined: Fri Dec 14, 2007 11:44 am
Location: Northern Ireland
x 16

Re: New InstanceManager: Instancing done the right way

Post by AshMcConnell »

Hi Matias,

I couldn't get Ogre Meshy to run on my desktop or laptop (both win 8.1 64 bit) even after downloading the vs_redist 2008 and latest directx, not sure what is wrong, searched the forums, but couldn't find anything obviously wrong - also tried running it from the command prompt, just returns with no message. I just used VS Debugger to get the vertex declaration, here are the (I think) relevant fields: -

Code: Select all

Offset	0
mType	VET_FLOAT3
mSemantic	VES_POSITION
mIndex	0	

mOffset	12
mType	VET_FLOAT3
mSemantic	VES_NORMAL
mIndex	0	

mOffset	24
mType	VET_FLOAT3
mSemantic	VES_TANGENT
mIndex	0

mOffset	36
mType	VET_FLOAT2
mSemantic	VES_TEXTURE_COORDINATES		
mIndex	0

mOffset	44
mType	VET_FLOAT2
mSemantic	VES_TEXTURE_COORDINATES		
mIndex	1

mOffset	52
mType	VET_FLOAT2
mSemantic	VES_TEXTURE_COORDINATES		
mIndex	2

mOffset 60
mType	VET_FLOAT2
mSemantic	VES_TEXTURE_COORDINATES		
mIndex	3

mOffset	69
mType	VET_FLOAT2
mSemantic	VES_TEXTURE_COORDINATES		
mIndex	4

mOffset	76
mType	VET_FLOAT2
mSemantic	VES_TEXTURE_COORDINATES		
mIndex	5
We are generating these mesh's from 3ds using Ogremax (the old free version), I'm not sure why there are so many texture co-ordinates in the declaration, this is a pretty simple static mesh of a tree, I've attached an example.

Any help appreciated, I'll try and get it working tonight if I can
You do not have the required permissions to view the files attached to this post.
User avatar
AshMcConnell
Silver Sponsor
Silver Sponsor
Posts: 605
Joined: Fri Dec 14, 2007 11:44 am
Location: Northern Ireland
x 16

Re: New InstanceManager: Instancing done the right way

Post by AshMcConnell »

I tried changing the VS_INPUT to mirror the Vertex Declaration + 3 float4s for the world matrix like so: -

Code: Select all

struct VS_INPUT
{
	float3 Position	:	POSITION;
	float3 Normal	:	NORMAL;
	float3 Tangent	:	TANGENT;
	float2 uv0		:	TEXCOORD0;
	float2 uv1		:   TEXCOORD1;
	float2 uv2		:   TEXCOORD2;
	float2 uv3		:   TEXCOORD3;
	float2 uv4		:   TEXCOORD4;
	float2 uv5		:   TEXCOORD5;
	float2 uv6		:   TEXCOORD6;
	float4 mat14	:	TEXCOORD7;
	float4 mat24	:	TEXCOORD8;
	float4 mat34	:	TEXCOORD9;
};
No luck, am I missing something else? Do I need to modify the material too?
User avatar
AshMcConnell
Silver Sponsor
Silver Sponsor
Posts: 605
Joined: Fri Dec 14, 2007 11:44 am
Location: Northern Ireland
x 16

Re: New InstanceManager: Instancing done the right way

Post by AshMcConnell »

Finally got some time tonight (in a semi-lucid state - baby problems!) to take another look at this. I managed to get the instancing working - realised that in my half asleep state I had too many texcoords in my input struct. I changed it to: -

Code: Select all

struct VS_INPUT
{
	float3 Position	:	POSITION;
	float3 Normal	:	NORMAL;
	float3 Tangent	:	TANGENT;
	float2 uv0		:	TEXCOORD0;

	float4 mat14	:	TEXCOORD6;
	float4 mat24	:	TEXCOORD7;
	float4 mat34	:	TEXCOORD8;
};
I am having another problem where it the whole batch seems to be culled even when looking at the trees. Has anyone had this problem? Have I missed something to do with bounding boxes? I'm thinking it might be to do with scaling from reading this post http://www.ogre3d.org/forums/viewtopic.php?f=2&t=79656

Here is a video of the problem

[youtube]IsbnfuCGdQM[/youtube]
User avatar
Jefferian
Halfling
Posts: 63
Joined: Sun Jun 12, 2011 12:53 am
Location: Italy
x 1

Re: New InstanceManager: Instancing done the right way

Post by Jefferian »

After some work and massive amounts of failures, i've successfully been able to instance and properly place a mesh with multiple materials (after some modification to the xml file to make vertices unshared) and to make one of those materials work with some custom parameters. I would eventually need to attach it to bones, and while that isn't supported, a workaround has been posted some pages ago. So what i should do now is to make it work with shadows.
And thus a question: does it require modifications to the shadow caster compared to a non-instancing one, or all i have to do is to properly modify the vertex shaders in the parts related to shadow handling?
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5429
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1337

Re: New InstanceManager: Instancing done the right way

Post by dark_sylinc »

AshMcConnell wrote:I am having another problem where it the whole batch seems to be culled even when looking at the trees. Has anyone had this problem? Have I missed something to do with bounding boxes? I'm thinking it might be to do with scaling from reading this post http://www.ogre3d.org/forums/viewtopic.php?f=2&t=79656
Mmmm, IIRC two things happen: internally there is an aabb for the whole batch (if this aabb test fails, the whole batch is skipped), and then a per-entity test. I suspect the whole batch' aabb is failing. Probably not being updated correctly. Are you using static mode? If you move stuff after setting static mode, the aabb is probably out of date.

Also, I haven't tried Pixel Based LOD with 1.x version of InstanceManager, don't know if it works as intended.
Jefferian wrote:And thus a question: does it require modifications to the shadow caster compared to a non-instancing one, or all i have to do is to properly modify the vertex shaders in the parts related to shadow handling?
You'll need a custom vertex shader to handle the sending of matrices.
Beware of of VTF instancing. It *needs* a whole shader caster material because the instance manager will clone it, then set the VTF texture in a texture unit (use shadow_caster_material).
User avatar
AshMcConnell
Silver Sponsor
Silver Sponsor
Posts: 605
Joined: Fri Dec 14, 2007 11:44 am
Location: Northern Ireland
x 16

Re: New InstanceManager: Instancing done the right way

Post by AshMcConnell »

dark_sylinc wrote:
AshMcConnell wrote:I am having another problem where it the whole batch seems to be culled even when looking at the trees. Has anyone had this problem? Have I missed something to do with bounding boxes? I'm thinking it might be to do with scaling from reading this post http://www.ogre3d.org/forums/viewtopic.php?f=2&t=79656
Mmmm, IIRC two things happen: internally there is an aabb for the whole batch (if this aabb test fails, the whole batch is skipped), and then a per-entity test. I suspect the whole batch' aabb is failing. Probably not being updated correctly. Are you using static mode? If you move stuff after setting static mode, the aabb is probably out of date.

Also, I haven't tried Pixel Based LOD with 1.x version of InstanceManager, don't know if it works as intended.
Yeah, it does seem to be the whole batch's aabb as lots of trees disappear at once. I tried it with and without static mode (I am not updating the position of the entities), but with the same result. The tree doesn't have any LOD at all, so hopefully that isn't the problem.

Is there anything I can do to debug? Otherwise, I think I might of had an unstable version of 1.9 the last time I updated (was a while given the issue I had with CMake + VS2012/13), I will update to the latest 1.10 branch and see if the issue goes away

Thanks for your help - hopefully when I get a few hours I'll be able to track down the problem properly, I'm getting 30 mins here and there atm and it's not working :)

- Ash
User avatar
Jefferian
Halfling
Posts: 63
Joined: Sun Jun 12, 2011 12:53 am
Location: Italy
x 1

Re: New InstanceManager: Instancing done the right way

Post by Jefferian »

dark_sylinc wrote: You'll need a custom vertex shader to handle the sending of matrices.
A custom vertex shader for the shadow caster, right? (aside from the custom vertex shader for the materials affected that i already had to do to make instancing work)
Thus, since i would also use the "normal" shadow caster in all the other cases, i suppose i have to declare the instancing shadow caster vertex program in the affected materials as "shadow_caster_vertex_program_ref" (and it should use the normal shadow caster fragment program by itself, so no need to declare it as shadow_caster_fragment_program_ref)?
If it is, i'm already on the right track. If it isn't, then i probably missed something.
dark_sylinc wrote: Beware of of VTF instancing. It *needs* a whole shader caster material because the instance manager will clone it, then set the VTF texture in a texture unit (use shadow_caster_material).
I'm using HWInstancingBasic as now, but since i'll also have to apply instancing to an animated mesh, this information will likely come in handy anyway.
Thank you very much.
phoenix_64
Gold Sponsor
Gold Sponsor
Posts: 34
Joined: Sat Nov 03, 2007 10:08 pm
x 4

Re: New InstanceManager: Instancing done the right way

Post by phoenix_64 »

I like the new instancing very much, thank you for implementing it :-)

The only thing that I don't like completely is the way the world matrix is supplied.
Because it is always supplied in the first free three TEXCOORD registers, one has to change these everytime a mapping channel is added or removed.
And: You need as much shadow caster shaders as you have different mapping channels. E.g. you have a material with zero, one material with 1, and one material with 4 mapping channels -> you need 3 different shadow caster shaders. Which of course also have to be changed, when you change material channels.
Since normal shaders are not directly compatible with instance shaders anyway, why not supply the matrix always in TEXCOORD0/1/2 (at least configurable)?
You then never would have to change these registers, and you would only need two instanced shadow caster shaders ever (one with, one without transparency).

OK, it's just a little inconvenience, but might be worth thinking about.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5429
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1337

Re: New InstanceManager: Instancing done the right way

Post by dark_sylinc »

@phoenix_64: I'm replying to your post to let you know we're not dead :)
phoenix_64 wrote:The only thing that I don't like completely is the way the world matrix is supplied.
Because it is always supplied in the first free three TEXCOORD registers, one has to change these everytime a mapping channel is added or removed. (...)
Yes. Back when I wrote it, I didn't foresee this problem. After implementing it in a real world application, the problem became quite apparent, and very annoying indeed.
It's not as easy as it sounds though: Adding 3 extra texcoords in its own buffers at the end is easy. But the opposite may not be so: If your original vertex format was "pos, normal, uv"; all in the same buffer, then this means that "uv" has to be texcoord3 while the floats passed as matrix (which are in its own vertex buffer and technically come after "uv") must be texcoord0, 1 & 2.
Some APIs may not like it (most likely driver bug, but still, it's playing with fire).
To be honest at the time I just did it so for simplicity.

However we are fixing this in Ogre 2.0 Final:
  • HLMS (High Level Material System) regenerates the shader on the fly automatically if the vertex format changes (i.e. you were using a mesh with 1 UVs, now you use a mesh with 2 UVs; the HLMS sees if the shader needs to change, makes the change, and recompiles; unless shader for Two-UV meshes has already been cached)
  • In Ogre 2.0 Final, the main method of passing per instance data will be the way it should be: through constant buffers (or texture buffers); no more texcoords.
  • If the vertex & index buffer refactor works out as expected, things like Instancing manager may end up looking like stone age (Entities would auto-instance into one draw call, even if they don't share the same material or are different submeshes!). There might be some edge cases where manually using an Instance Manager could speed things up, but I think that will be rare.
Cheers
Matías.
User avatar
syedhs
Silver Sponsor
Silver Sponsor
Posts: 2703
Joined: Mon Aug 29, 2005 3:24 pm
Location: Kuala Lumpur, Malaysia
x 51

Re: New InstanceManager: Instancing done the right way

Post by syedhs »

1) Is there any frame stutter if entities are being added/removed to instancing as the camera moves around (paging)?

2) How about VRAM usage with static instancing - does it get increased (if yes, hopefully not much), remain the same (good!) or less? Of course this is a comparison between normal entity creation vs static instancing.

Thank you!
A willow deeply scarred, somebody's broken heart
And a washed-out dream
They follow the pattern of the wind, ya' see
Cause they got no place to be
That's why I'm starting with me
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5429
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1337

Re: New InstanceManager: Instancing done the right way

Post by dark_sylinc »

syedhs wrote:1) Is there any frame stutter if entities are being added/removed to instancing as the camera moves around (paging)?
Instances are added and preallocated in batches. So, if you have 2 batches, 100 entities per batch; and you've got 199 entities on your scene; adding one more is almost free.
But adding one more (now we've got 201); will cause a small stutter as we need to create the new batch (an API vertex buffer creation call gets made) and allocate 100 more instances.
Now that you have 201; then remove 80; then much later add 90; it will be again almost free as batches don't get automatically removed.

This can lead to fragmentation though. And fragmentation prevents efficient frustum culling.
I recommend you read section 5.6 of the porting manual. 99% of what applies to instancing in that manual is for both 1.9 and 2.0

syedhs wrote:2) How about VRAM usage with static instancing - does it get increased (if yes, hopefully not much), remain the same (good!) or less? Of course this is a comparison between normal entity creation vs static instancing.
If you compare Entity vs InstancedEntity; in theory instancing should require more memory because of the GPU buffers used to pass instance data from CPU to GPU (one per batch, its size depends on the number of instances per batch).
But in practice (speaking about total RAM, not just VRAM) I've noticed much lower consumption when using Instancing, as InstancedEntity is much more lightweight (i.e. no facial animation, among other things) and shares some attributes with their siblings that are stored in the batch. This applies to 1.9 though. I haven't benchmarked 2.0 about this; where regular Entities have much lower memory consumption now.

Nonetheless, inadecuate settings can lead to major memory waste. See section 5.2.1 "Instances per batch" in the manual.

As for comparing RAM/VRAM usage from InstancedEntity vs static InstancedEntity, the memory consumption should be more or less the same.
User avatar
syedhs
Silver Sponsor
Silver Sponsor
Posts: 2703
Joined: Mon Aug 29, 2005 3:24 pm
Location: Kuala Lumpur, Malaysia
x 51

Re: New InstanceManager: Instancing done the right way

Post by syedhs »

Thank you dark_sylinc. Once I get the code done I will publish my findings (if they are of something useful). :)
A willow deeply scarred, somebody's broken heart
And a washed-out dream
They follow the pattern of the wind, ya' see
Cause they got no place to be
That's why I'm starting with me
User avatar
AshMcConnell
Silver Sponsor
Silver Sponsor
Posts: 605
Joined: Fri Dec 14, 2007 11:44 am
Location: Northern Ireland
x 16

Re: New InstanceManager: Instancing done the right way

Post by AshMcConnell »

Hi Folks,

I've discovered that my InstancedEntities are being shown when rendering my self-shadowing render-target. I tried adding each InstancedEntity to a SceneNode for my track which is detached before rendering my self-shadowing RT, but that slowed things down (reduced fps by around 20%). What is the most efficient way of hiding quite a lot of InstancedEntities at once?

*Edit* show I be looking at visibility flags for this sort of thing rather than hiding / showing?

Thanks!
Ash
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5429
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1337

Re: New InstanceManager: Instancing done the right way

Post by dark_sylinc »

AshMcConnell wrote:*Edit* show I be looking at visibility flags for this sort of thing rather than hiding / showing?
Yes. To flexibly and efficiently handle this either set the visibility flags, or put the instances in its own render queue (iirc you'll have to call instancedEntity->_getBatch()->setRenderQueueGroup) and avoid rendering those flags / render queue when you don't want it.

In Ogre 2.0 I can tell you that RQs allow you to bypass the batches entirely (i.e. no CPU overhead is spent at all) while visibility flags get parsed but then skipped. The disadvantage is that RQ also alter the order of rendering (which might be something you don't want) and that RQs aren't as flexible as visibility flags. What you end up using is up to you.
In Ogre 1.x it doesn't really matter because the skipped RQs will still be processed.
User avatar
AshMcConnell
Silver Sponsor
Silver Sponsor
Posts: 605
Joined: Fri Dec 14, 2007 11:44 am
Location: Northern Ireland
x 16

Re: New InstanceManager: Instancing done the right way

Post by AshMcConnell »

Thanks Matias,

There doesn't seem to be a _getBatch() so I used: - (this is incorrect - see update below)

Code: Select all

instancedEntity->setRenderQueueGroup(Ogre::RENDER_QUEUE_WORLD_GEOMETRY_2);

When creating the instancedEntity.

I then put this code in the preRenderTargetUpdate method: -

Code: Select all

_sceneManager->clearSpecialCaseRenderQueues();
	 _sceneManager->addSpecialCaseRenderQueue(RENDER_QUEUE_OVERLAY);
	_sceneManager->addSpecialCaseRenderQueue(RENDER_QUEUE_WORLD_GEOMETRY_2);
	_sceneManager->setSpecialCaseRenderQueueMode(Ogre::SceneManager::SCRQM_EXCLUDE);
But the render queue doesn't seem to be set at all as it's still being rendered. I noticed the tree materials being supplied to the MaterialManager::Listener callback: -

Code: Select all

Ogre::Technique * SelfShadowDynamicTextureOgre::handleSchemeNotFound( unsigned short schemeIndex, const String & schemeName, Material * originalMaterial, unsigned short lodIndex, const Renderable * rend )
	{
		if (dynamic_cast<const Ogre::InstanceBatchHW *> (rend)){
			LOGD << originalMaterial->getName() << " RenderQueue " << (dynamic_cast<const Ogre::InstanceBatchHW *> (rend))->getRenderQueueGroup();
		}

		// This sets all the materials in the scene to use the shadow material technique instead of their normal one
		return _shadowMakerMaterial->getTechnique(0);
	}
My debug shows that the renderqueue is set to 2 (which doesn't appear in the RenderQueueGroupID enum). Is there an issue here or a different way to set the renderqueue for the InstancedEntity?

Thanks for your help :)


UPDATE

I used: -

Code: Select all

instancedEntity->_getOwner()->setRenderQueueGroup(Ogre::RENDER_QUEUE_WORLD_GEOMETRY_2);
This sets the renderqueue correctly (I see TREES03 RenderQueue 75 in debug), but strangely the materials for the instanced items are being passed through to the handleSchemeNotFound. I thought they would be ignored as they are explicitly ignored for this rendertarget

UPDATE 2

Seems that the instanced entities aren't being rendered, despite that each of them is being processed (to find missing material schemes at least). Seems a bit inefficient, is this what you meant by "In Ogre 1.x it doesn't really matter because the skipped RQs will still be processed". I need to get onto 2.0! :)
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5429
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1337

Re: New InstanceManager: Instancing done the right way

Post by dark_sylinc »

AshMcConnell wrote:Seems that the instanced entities aren't being rendered, despite that each of them is being processed (to find missing material schemes at least). Seems a bit inefficient, is this what you meant by "In Ogre 1.x it doesn't really matter because the skipped RQs will still be processed". I need to get onto 2.0! :)
Yup. That's what I meant.
Perhaps there might be something you can avoid processing by debugging Ogre and seeing there is one flag to set that could avoid the handling scheme not found. But I doubt it. The RQ system in 1.x is a nightmare.
Nickak2003
Goblin
Posts: 272
Joined: Thu Jun 10, 2004 4:19 am
x 26

Re: New InstanceManager: Instancing done the right way

Post by Nickak2003 »

I'm using robot.mesh, and they are appearing at 0,0,0- setposition and orient does not work. Shaderbased looks normal while VTF looks red. the createInstancedEntity string is as follows: Examples/Instancing/Shaderbased/Robot. I get tons of error messages about shaders. The actual ogre demo works fine for all modes.
my code is as follows:

Code: Select all


class TestNewInstancing : public OgreTest {
protected:
	bool mUseSceneNodes{ true }, mSetStatic{ false };
	std::size_t mInstanceNum{ 5 };
	std::string mMeshName{ "robot.mesh" };
	std::size_t mInstancingTechnique{ 0 };
	Ogre::InstanceManager *mCurrentManager{ nullptr };
	std::vector<Ogre::InstancedEntity*>   mMovedInstances;
	std::vector<Ogre::MovableObject*>     mEntities;
	std::vector<Ogre::SceneNode*>         mSceneNodes;
	std::set<Ogre::AnimationState*>       mAnimations;

	std::vector<Ogre::String> mMaterialsTechniques{
		"Examples/Instancing/ShaderBased/Robot",	//working
		"Examples/Instancing/VTF/Robot",			//missing something - red tint
		"Examples/Instancing/HWBasic/Robot",		//working
		"Examples/Instancing/VTF/HW/Robot",			//missing something - red tint
		"Examples/Instancing/VTF/HW/LUT/Robot",		//missing something - red tint
		"Examples/Instancing/ShaderBased/Robot" };	//working

	virtual bool frameStarted(const Ogre::FrameEvent& frameEvent) {
		if (OgreTest::frameStarted(frameEvent) == false) return false;

		return true;
	}

	void createScene() {
		
		mGraphics->getSceneManager()->setAmbientLight(Ogre::ColourValue(1, 1, 1));
		mCamera.setPosition(Ogre::Vector3(0, 0, 300));

		setupContent();

	}
	void setupContent() {
		//Initialize the techniques and current mesh variables
		checkHardwareSupport();

		mGraphics->getSceneManager()->setShadowTechnique(Ogre::SHADOWTYPE_TEXTURE_ADDITIVE_INTEGRATED);
		mGraphics->getSceneManager()->setShadowTextureSelfShadow(true);
		mGraphics->getSceneManager()->setShadowCasterRenderBackFaces(true);

		if (Ogre::Root::getSingletonPtr()->getRenderSystem()->getName().find("OpenGL ES 2") == Ogre::String::npos){
			mGraphics->getSceneManager()->setShadowTextureConfig(0, 2048, 2048, Ogre::PF_FLOAT32_R);
		}else{
			// Use a smaller texture for GL ES 3.0
			mGraphics->getSceneManager()->setShadowTextureConfig(0, 512, 512, Ogre::PF_FLOAT32_R);
		}

		//LiSPSMShadowCameraSetup *shadowCameraSetup = new LiSPSMShadowCameraSetup();
		Ogre::FocusedShadowCameraSetup *shadowCameraSetup = new Ogre::FocusedShadowCameraSetup();
		//PlaneOptimalShadowCameraSetup *shadowCameraSetup = new PlaneOptimalShadowCameraSetup();

		mGraphics->getSceneManager()->setShadowCameraSetup(Ogre::ShadowCameraSetupPtr(shadowCameraSetup));

		mEntities.reserve( mInstanceNum );
		mSceneNodes.reserve(mInstanceNum );

		setupLighting();

		switchInstancingTechnique();
	}

	void switchInstancingTechnique()
	{

		if (mCurrentManager)
			mGraphics->getSceneManager()->destroyInstanceManager(mCurrentManager);
		
		//if(!supported technique ){}

		if (mInstancingTechnique < mMaterialsTechniques.size() ){
			//Instancing

			//Create the manager if we haven't already (i.e. first time)
			//Because we use IM_USEALL as flags, the actual num of instances per batch might be much lower
			//If you're not bandwidth limited, you may want to lift IM_VTFBESTFIT flag away

			Ogre::InstanceManager::InstancingTechnique technique = Ogre::InstanceManager::ShaderBased;

			switch (mInstancingTechnique)
			{
			case 0: technique = Ogre::InstanceManager::ShaderBased; break;
			case 1: technique = Ogre::InstanceManager::TextureVTF; break;
			case 2: technique = Ogre::InstanceManager::HWInstancingBasic; break;
			case 3:
			case 4: technique = Ogre::InstanceManager::HWInstancingVTF; break;
			}

			Ogre::uint16 flags = Ogre::IM_USEALL;
		
			if (mInstancingTechnique == 4){
				flags |= Ogre::IM_VTFBONEMATRIXLOOKUP;
			}
			//Only one weight is recommended for the VTF technique, but force the use of more for the demo
			if (mInstancingTechnique == 1 && (flags & Ogre::IM_USEBONEDUALQUATERNIONS)){
				flags &= ~Ogre::IM_USEONEWEIGHT;
			}

			mCurrentManager = mGraphics->getSceneManager()->createInstanceManager(
				"InstanceMgr" + Ogre::StringConverter::toString(mInstancingTechnique), mMeshName,
				Ogre::ResourceGroupManager::AUTODETECT_RESOURCE_GROUP_NAME, technique,
				mInstanceNum, flags);

			createInstancedEntities();
		}
		else
		{
			//Non-instancing
			//createEntities();

			//Hide GUI features available only to instancing
			//mCurrentManager = 0;
			//mDefragmentBatches->hide();
			//mDefragmentOptimumCull->hide();
		}
		//Here the SceneNodes are created. Since InstancedEntities derive from MovableObject,
		//they behave like regular Entities on this.
		Ogre::SceneNode *rootNode = mGraphics->getSceneManager()->getRootSceneNode();

		if (mUseSceneNodes) 
			for (int i = 0; i<mEntities.size(); ++i){
				Ogre::SceneNode *sceneNode = rootNode->createChildSceneNode();
				sceneNode->attachObject(mEntities[i]);
			//	sceneNode->yaw(Radian(randGenerator.nextFloat() * 10 * 3.14159265359f)); //Random orientation
				sceneNode->setPosition(Ogre::Vector3( i*50,i*50,0 ) );
				mSceneNodes.push_back(sceneNode);
			}

		//Show/hide "static" button, and restore config. Do this _after_ createSceneNodes()
		if (mInstancingTechnique == Ogre::InstanceManager::HWInstancingBasic ||
			mInstancingTechnique == Ogre::InstanceManager::HWInstancingVTF ||
			mInstancingTechnique == Ogre::InstanceManager::HWInstancingVTF + 1) // instancing with lookup
		{
			if (mSetStatic)
				mCurrentManager->setBatchesAsStaticAndUpdate(mSetStatic);
		}
	}
	void createInstancedEntities(){
	
		for (int i = 0; i<mInstanceNum; ++i)
		{

			//Create the instanced entity
			Ogre::InstancedEntity *ent = mCurrentManager->createInstancedEntity(mMaterialsTechniques[mInstancingTechnique]);
			Ogre::String s = mMaterialsTechniques[mInstancingTechnique];
		//	Ogre::Entity* ent = mGraphics->getSceneManager()->createEntity(mMeshName);
			mEntities.push_back(ent);

			if (ent->getAllAnimationStates() != nullptr && mInstancingTechnique != Ogre::InstanceManager::HWInstancingBasic){
				//Get the animation
				Ogre::AnimationState *anim = ent->getAnimationState("Walk");
				anim->setEnabled(true);
				anim->addTime(0); //Random start offset
				mAnimations.insert(anim);
			}

			if (mInstancingTechnique < mMaterialsTechniques.size()  && !mUseSceneNodes){
			//	mMovedInstances.push_back(ent);
				ent->setOrientation(Ogre::Quaternion::IDENTITY);
				ent->setPosition(Ogre::Vector3(i*50, 0, 0));
			}
		}
		
	}
	void setupLighting(){
		mGraphics->getSceneManager()->setAmbientLight(Ogre::ColourValue(0.40f, 0.40f, 0.40f));

		Ogre::ColourValue lightColour(1, 0.5, 0.3);

		//Create main (point) light
		Ogre::Light* light = mGraphics->getSceneManager()->createLight();
		light->setDiffuseColour(lightColour);
		light->setPosition(0.0f, 25.0f, 0.0f);
		light->setSpecularColour(0.6, 0.82, 1.0);
		light->setAttenuation(3500, 0.085, 0.00008, 0.00006);
		light->setCastShadows(false);

		//Create a dummy spot light for shadows
		light = mGraphics->getSceneManager()->createLight();
		light->setType(Ogre::Light::LT_SPOTLIGHT);
		light->setDiffuseColour(Ogre::ColourValue(0.15f, 0.35f, 0.44f));
		light->setPosition(250.0f, 200.0f, 250.0f);
		light->setDirection((Ogre::Vector3::UNIT_SCALE * -1.0f).normalisedCopy());
		light->setSpecularColour(0.2, 0.12, 0.11);
		light->setAttenuation(3500, 0.005, 0.00002, 0.00001);
		light->setSpotlightRange(Ogre::Degree(80), Ogre::Degree(90));
		light->setCastShadows(true);
		light->setLightMask(0x00000000);
	}
	void checkHardwareSupport()
	{
		//Check Technique support
		for (int i = 0; i<mMaterialsTechniques.size()-1; ++i)
		{
			Ogre::InstanceManager::InstancingTechnique technique;
			switch (i){
			case 0: technique = Ogre::InstanceManager::ShaderBased; break;
			case 1: technique = Ogre::InstanceManager::TextureVTF; break;
			case 2: technique = Ogre::InstanceManager::HWInstancingBasic; break;
			case 3:
			case 4: technique = Ogre::InstanceManager::HWInstancingVTF; break;
			}

			Ogre::uint16 flags = Ogre::IM_USEALL;
			if (i == 4)
			{
				flags |= Ogre::IM_VTFBONEMATRIXLOOKUP;
			}

			int numInstancesPerBatch = 1000;
			const size_t numInstances = mGraphics->getSceneManager()->getNumInstancesPerBatch(mMeshName,
				Ogre::ResourceGroupManager::AUTODETECT_RESOURCE_GROUP_NAME,
				mMaterialsTechniques[i], technique, numInstancesPerBatch, flags);

			int ix = 0;
			//mSupportedTechniques[i] = numInstances > 0; //larger than 0 if technique is supported
		}
		//non instancing is always supported
		//
	}

	void destroyScene() {
		if (mCurrentManager)
			mCurrentManager->cleanupEmptyBatches();

		mEntities.clear();
		mMovedInstances.clear();
		mSceneNodes.clear();
		mAnimations.clear();

		mGraphics->getSceneManager()->destroyInstanceManager(mCurrentManager);;
	}
public:
	TestNewInstancing(Graphics* graphics, Input* input)
		:OgreTest(graphics, input)
	{}



};

I think I copied all the relevant code from the example demo. I am clearly missing something. Is the real time shader system required?
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5429
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1337

Re: New InstanceManager: Instancing done the right way

Post by dark_sylinc »

Nickak2003 wrote:I think I copied all the relevant code from the example demo. I am clearly missing something. Is the real time shader system required?
No, it's not required.

You likely missed a shader-related file (that includes the script and materials); or you're not loading correctly. The shaders are the most important factor on getting instancing running. Without the proper shaders, all objects are going to ignore your setPosition and setOrientation calls; and colouring is going to be all weird.
IIRC the DX9 shaders used Cg. If you didn't include the plugin, well, they're gonna have errors.

Posting the Ogre.log may hint a clue on what's going on.