New InstanceManager: Instancing done the right way

Discussion area about developing or extending OGRE, adding plugins for it or building applications on it. No newbie questions please, use the Help forum for that.
cyrfer
Orc
Posts: 424
Joined: Wed Aug 01, 2007 8:13 pm
Location: Venice, CA, USA
x 7

Re: New InstanceManager: Instancing done the right way

Post by cyrfer »

Zamx wrote:Now another question, currently instancing only works for triangles. But we are also using lines to present wireframes. Is this something you are working or is this something I could do?
Hi Zamx,
I usually draw lines by defining a mesh resource with the vertex and index data I want for my lines, then I change the Submesh::operationType member to be Ogre::RenderOperation::OT_LINE_LIST or whatever topology your vertex and index data defines. I think that member is Ogre::RenderOperation::OT_TRIANGLE_LIST by default.

If you have adjusted the 'operationType', so that an Entity uses it correctly, then I would expect the instancing code to work well with it too. Good luck.

User avatar
ahmadi
Gnome
Posts: 312
Joined: Sat Nov 26, 2005 4:03 pm

Re: New InstanceManager: Instancing done the right way

Post by ahmadi »

Hi dark_sylinc :D
First i must say thank you for developing the new InstanceManager, Its great work!
I encountered with a problem while want to use the new InstanceManager.
I have an Entity that use a complex material, For example a "CarPaint" material that have 4 texture units and a complex shader (too many process on vertex and fragment program not just a single diffuse texture or color).
How can i use the complex material with New InstanceManager, because the instance material use itself vertex shader! and as i know we can not use 2 vertex shaders on one pass.
The only solution that reach to my mind is that we combine your instancing shader with the "CarPaint" shader but it need long debug/develop time.
Do you know any better idea?
Thank you for your attention.
_________________________________
Best regards

Ahmadi

User avatar
m2codeGEN
Halfling
Posts: 52
Joined: Tue Apr 26, 2011 9:13 am
Location: Russia, Tver
x 2

Re: New InstanceManager: Instancing done the right way

Post by m2codeGEN »

m2codeGEN wrote:I think result won't keep itself waiting long as next week it is necessary to give the version on an alpha testing.
At current time I encode a position, yaw rotation, scale and height on each tree in 8 bytes.
Well, this is my implementation Instancing for forest rendering (aka PagedGeometry 2.0 8)
100 FPS when moving with speed about 1000 km/h

[youtube]ufqBr62YI_g[/youtube]

User avatar
icaro56
Greenskin
Posts: 105
Joined: Sun Feb 15, 2009 2:03 am
Location: Belo Horizonte, MG, BRA
x 1

Re: New InstanceManager: Instancing done the right way

Post by icaro56 »

can I use mouse pick in new InstanceManager ?
Ícaro Motta

User avatar
icaro56
Greenskin
Posts: 105
Joined: Sun Feb 15, 2009 2:03 am
Location: Belo Horizonte, MG, BRA
x 1

Re: New InstanceManager: Instancing done the right way

Post by icaro56 »

Hi,

Would use a mouse to get the pick entity InstanceEntity?

Edit:
Forgive me, I repeated a post without paying attention.
Last edited by icaro56 on Mon Jul 09, 2012 4:12 pm, edited 1 time in total.
Ícaro Motta

User avatar
duststorm
Minaton
Posts: 921
Joined: Sat Jul 31, 2010 6:29 pm
Location: Belgium
x 80
Contact:

Re: New InstanceManager: Instancing done the right way

Post by duststorm »

m2codeGEN wrote:Well, this is my implementation Instancing for forest rendering (aka PagedGeometry 2.0 8)
100 FPS when moving with speed about 1000 km/h
Great work m2codeGEN. I'm happy to see that the wonderful pagedGeometry component is still being developed. :)
Developer @ MakeHuman.org

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4260
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 831
Contact:

Re: New InstanceManager: Instancing done the right way

Post by dark_sylinc »

Hi everyone!!!

Some of you may be aware that I'm working with Sandswept to bring you The Dead Linger.
We're working hard so I don't have much time. Basically we're doing a deferred shader, and HW Basic is a great candidate to draw the light volumes. But we needed per-instanced-entity custom parameters. The good guys at Sandswept allowed me to release the Ogre changes, be thankfull to them and make sure to pass by!!!

Per-instanced-entity custom parameters was implemented!
  • Right now only HW Basic implements them, but any of you can extend it to work with HW VTF (I recommend to pass the custom parameters in the VTF texture)
  • Parameters are actually stored in the InstanceBatch. This way there's very little memory overhead for those who don't need them, and want to use thousands (or millions) of instanced entities.
How to use them:
1. Before creating your first InstancedEntity, call:

Code: Select all

instanceManager->setNumCustomParams( numParameters );
Where numParameters is the number of parameters per instance you want to use. 1 param = 4 floats (One Vector4)
Take in mind each technique may have a limit on the max. number of params, and the more params you use, the slower (GPU-bound) it will be.

2. To set the custom parameter, call

Code: Select all

instancedEntity->setCustomParam( idx, Vector4 );
Where idx is the index of the param, must be in the range [0; numParameters)
If this function is never called, all instances default to Vector4::ZERO. Watch out! If you destroy an instanced entity and then create it again (remember! Instanced entities are pre-allocated) it's param will contain the old value before it was destroyed.

3.
a. For HW Basic:
In the vertex shader add another TEXCOORD at the end, it has to be a float4. If there are more custom params, use more TEXCOORDs.
b. For HW VTF
Not implemented. It would be possible to pass them as TEXCOORDs (like HW Basic), but if someone wants to accept the challenge; I HIGHLY RECOMMEND to pass them with the VTF texture.

Now you can pass custom values to your instances (player colour, colour variation/randomization, vertex tree/grass animation data, material parameters, ID data, etc).

Enjoy!
Dark Sylinc

PS. This code is in 1.9

slemh
Gnoblar
Posts: 4
Joined: Mon Dec 12, 2011 1:50 pm

Re: New InstanceManager: Instancing done the right way

Post by slemh »

Is it possible to add MovableObjects to InstancedEntities? I've researched a bit and it seems that it's not supported in it's current form. Would it be possible to manually transform the animations which is supported by the standard way or is there and any setback by doing that? I mean Ogre::Bone::setManuallyControlled.

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4260
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 831
Contact:

Re: New InstanceManager: Instancing done the right way

Post by dark_sylinc »

You mean attach objects to bones?.
No. It's not supported. The method we currently use in Ogre is bloated, flawed in many ways, and slowdowns too much; which is why I decided not to support it.

You can still attach to bones by retrieving the SkeletonInstance, then retrieve the Bone you want to attach to, and get it's derived position & derived orientation, like this:

Code: Select all

Ogre::Vector3 attachmentPos = instancedEntity->getPosition() + bone->_getDerivedPosition() + bone->_getDerivedOrientation() * offset;
Ogre::Vector3 attachmentRot = instancedEntity->_getDerivedOrientation() * bone->_getDerivedOrientation() * qOffset;
You can also use SkeletonInstance->getSkeleton->createTagPoint() and attach a listener to that TagPoint (not recommended.
setManuallyControlled has nothing to do here btw.

Cheers
Dark Sylinc

al2950
OGRE Expert User
OGRE Expert User
Posts: 1216
Joined: Thu Dec 11, 2008 7:56 pm
Location: Bristol, UK
x 150

Re: New InstanceManager: Instancing done the right way

Post by al2950 »

dark_sylinc wrote: Per-instanced-entity custom parameters was implemented!
This is great, I was just starting to look at options for instancing light volumes and other deferred issues. Thank you very much dark_sylinc & Sandswept :D

bstone
OGRE Expert User
OGRE Expert User
Posts: 1920
Joined: Sun Feb 19, 2012 9:24 pm
Location: Russia
x 199

Re: New InstanceManager: Instancing done the right way

Post by bstone »

Yeah, I think that's pretty cool. Any one of you mind sharing the performance boost number after applying that to light volumes?

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4260
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 831
Contact:

Re: New InstanceManager: Instancing done the right way

Post by dark_sylinc »

There's no "performance boost comparison" numbers to show, because we went right away towards Instancing for the light volumes. We didn't even hesitate (we use a custom deferred lighting solution, not the one that comes with Ogre).

We still send the light parameters (diffuse, specular, outer angle, range, etc) through a texture managed from C++; and send the necessary UVs in the custom params, to sample the right pixels. We felt like it was much more memory efficient than passing it as Texcoords.

bstone
OGRE Expert User
OGRE Expert User
Posts: 1920
Joined: Sun Feb 19, 2012 9:24 pm
Location: Russia
x 199

Re: New InstanceManager: Instancing done the right way

Post by bstone »

I see. It's a pity, I hoped to hear how you weren't satisfied with the naive approach and had a tremendous boost with instancing. All right, I'll save my ears for the glory of another day :)

User avatar
mkultra333
Gold Sponsor
Gold Sponsor
Posts: 1889
Joined: Sun Mar 08, 2009 5:25 am
x 105

Re: New InstanceManager: Instancing done the right way

Post by mkultra333 »

I have a couple of questions about instancing. I already use a home-brewed "pseudo-instancing" method, and I'm trying to work out whether I'd be better off changing to the Ogre method.

1. What's the batching overhead? I already have a system that allows 6 to 12 entities to be rendered in a single batch. This is probably what the player would typically encounter, 6 to 12 of the same type of monster at once. I suspect for these low numbers, my system would batch better than the Ogre method, which I'm guessing uses at least 2 or 3 batches to render any meshes.

2. Can I change the structure of the skeleton used by each individual entity? My entities explode into lots of wreckage when they die, and at that point I modify their skeletons, so that all the child bones become free from their parents. I'm wondering if this is also allowed in the Ogre instancing.

3. What are the limitations of the different Ogre instancing methods? I've read this whole thread now, looked at the api, and read the header files, and I still don't understand what the different methods mean. There's this in the header:

ShaderBased, //Any SM 2.0+ @See InstanceBatchShader
TextureVTF, //Needs Vertex Texture Fetch & SM 3.0+ @See InstanceBatchVTF
HWInstancingBasic, //Needs SM 3.0+ and HW instancing support @See InstanceBatchHW
HWInstancingVTF, //Needs SM 3.0+, HW instancing support & VTF @See InstanceBatchHW_VTF

I gather HWInstancingBasic doesn't allow any animation, while the other methods do. But apparently there are some limitations to the animation in the different version... what are they?

Also, what's the difference between the version that uses scene nodes and the version that doesn't?

Any info appreciated, thanks.
"In theory there is no difference between practice and theory. In practice, there is." - Psychology Textbook.

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4260
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 831
Contact:

Re: New InstanceManager: Instancing done the right way

Post by dark_sylinc »

mkultra333 wrote:1. What's the batching overhead? I already have a system that allows 6 to 12 entities to be rendered in a single batch. This is probably what the player would typically encounter, 6 to 12 of the same type of monster at once. I suspect for these low numbers, my system would batch better than the Ogre method, which I'm guessing uses at least 2 or 3 batches to render any meshes.
That's hard to answer. But if you're using the same material, and the same mesh; and you've given a good hint about the instance count per batch (i.e. don't tell the engine you'll have 1000 instances when you're going to use 6) then the overhead is very low.
mkultra333 wrote:2. Can I change the structure of the skeleton used by each individual entity? My entities explode into lots of wreckage when they die, and at that point I modify their skeletons, so that all the child bones become free from their parents. I'm wondering if this is also allowed in the Ogre instancing.
My guess is no. But I don't know what exactly are you planning to do. All instances share the same skeleton, so if you change the skeleton, it will affect all instances.
However, you can create a keyframed animation that sets positions & orientations as if the bones were parentless. If the interpolation doesn't screw it too much, you can get away with that; by playing the "exploding animation" for those entities.
Another solution is to swap the instanced entity with a normal Entity that can explode (or for instanced entities with skeletons that allow exploding). This is why InstancedEntity derives from MovableObject, so that you can treat them almost equally and swap with little issue.
mkultra333 wrote: 3. What are the limitations of the different Ogre instancing methods? I've read this whole thread now, looked at the api, and read the header files, and I still don't understand what the different methods mean. There's this in the header:

ShaderBased, //Any SM 2.0+ @See InstanceBatchShader
TextureVTF, //Needs Vertex Texture Fetch & SM 3.0+ @See InstanceBatchVTF
HWInstancingBasic, //Needs SM 3.0+ and HW instancing support @See InstanceBatchHW
HWInstancingVTF, //Needs SM 3.0+, HW instancing support & VTF @See InstanceBatchHW_VTF

I gather HWInstancingBasic doesn't allow any animation, while the other methods do. But apparently there are some limitations to the animation in the different version... what are they?
In short:
ShaderBased -> Crap, but runs on old ShaderModel 2.0. Number of instances per batch can be quite limited.
TextureVTF-> Crap, but runs on older OpenGL versions that didn't support HW instancing. Can be very fast, but can waste a lot GPU bandwidth.
HWInstancingBasic -> Awesome, use this one if your model doesn't use skeletal animations
HWInstancingVTF-> Awesome, use this one if your model uses skeletal animations. Needs DX10 capable hardware to run at full speed (even if you're using D3D9 or OGL rendersystems). In other words, any GeForce 8000+ or ATI Radeon HD 2000+

Note that the "crap" versions is compared to the "HWInstancing" versions. All of the techniques surpass normal Entity rendering.
mkultra333 wrote:Also, what's the difference between the version that uses scene nodes and the version that doesn't?
Using SceneNodes is more generic and easier to work with (since Entities and InstancedEntities are treated equally, and will behave in the same way)
Not using SceneNodes will give you raw performance (it's faster)

Transporter
Minaton
Posts: 933
Joined: Mon Mar 05, 2012 11:37 am
Location: Germany
x 110

Re: New InstanceManager: Instancing done the right way

Post by Transporter »

Hi,

I like to use hardware instancing but it's not working. Can someone create a tutorial for the wiki please?

Code: Select all

/*
-----------------------------------------------------------------------------
Filename:    TutorialApplication.h
-----------------------------------------------------------------------------

This source file is part of the
   ___                 __    __ _ _    _ 
  /___\__ _ _ __ ___  / / /\ \ (_) | _(_)
 //  // _` | '__/ _ \ \ \/  \/ / | |/ / |
/ \_// (_| | | |  __/  \  /\  /| |   <| |
\___/ \__, |_|  \___|   \/  \/ |_|_|\_\_|
      |___/                              
      Tutorial Framework
      http://www.ogre3d.org/tikiwiki/
-----------------------------------------------------------------------------
*/
#ifndef __TutorialApplication_h_
#define __TutorialApplication_h_

#include "BaseApplication.h"

#define NUM_INST_ROW 50
#define NUM_INST_COLUMN 50

class TutorialApplication : public BaseApplication
{
private:
	Ogre::InstanceManager* mInstanceManager;
	std::vector<Ogre::MovableObject*> mEntities;
	std::vector<Ogre::InstancedEntity*> mMovedInstances;
	std::set<Ogre::AnimationState*> mAnimations;

public:
    TutorialApplication(void);
    virtual ~TutorialApplication(void);

protected:
    virtual void createScene(void);
	virtual bool frameRenderingQueued(const Ogre::FrameEvent& evt);
	virtual void destroyScene(void);
};

#endif // #ifndef __TutorialApplication_h_

Code: Select all

/*
-----------------------------------------------------------------------------
Filename:    TutorialApplication.cpp
-----------------------------------------------------------------------------

This source file is part of the
   ___                 __    __ _ _    _ 
  /___\__ _ _ __ ___  / / /\ \ (_) | _(_)
 //  // _` | '__/ _ \ \ \/  \/ / | |/ / |
/ \_// (_| | | |  __/  \  /\  /| |   <| |
\___/ \__, |_|  \___|   \/  \/ |_|_|\_\_|
      |___/                              
      Tutorial Framework
      http://www.ogre3d.org/tikiwiki/
-----------------------------------------------------------------------------
*/
#include "TutorialApplication.h"

//-------------------------------------------------------------------------------------
TutorialApplication::TutorialApplication(void)
{
	mInstanceManager = NULL;
}

//-------------------------------------------------------------------------------------
TutorialApplication::~TutorialApplication(void)
{
}

//-------------------------------------------------------------------------------------
void TutorialApplication::createScene(void)
{
	// SceneManager
	mSceneMgr->setShadowTechnique( Ogre::SHADOWTYPE_TEXTURE_ADDITIVE );
	mSceneMgr->setShadowTextureConfig( 0, 2048, 2048, Ogre::PF_FLOAT32_R );
	mSceneMgr->setShadowTextureSelfShadow( true );
	mSceneMgr->setShadowCasterRenderBackFaces( true );
	Ogre::FocusedShadowCameraSetup *shadowCameraSetup = new Ogre::FocusedShadowCameraSetup();
	mSceneMgr->setShadowCameraSetup( Ogre::ShadowCameraSetupPtr(shadowCameraSetup) );
	mSceneMgr->setSkyBox(true, "Examples/CloudyNoonSkyBox");
	mSceneMgr->setAmbientLight( Ogre::ColourValue( 0.40f, 0.40f, 0.40f ) );

	//Create a dummy spot light for shadows
	Ogre::Light* light = mSceneMgr->createLight();
	light->setType( Ogre::Light::LT_SPOTLIGHT );
	light->setDiffuseColour( Ogre::ColourValue( 0.15f, 0.35f, 0.44f ) );
	light->setPosition( 250.0f, 200.0f, 250.0f );
	light->setDirection( (Ogre::Vector3::UNIT_SCALE * -1.0f).normalisedCopy() );
	light->setSpecularColour( 0.2, 0.12, 0.11 );
	light->setAttenuation( 3500, 0.005, 0.00002, 0.00001 );
	light->setSpotlightRange( Ogre::Degree(80), Ogre::Degree(90) );
	light->setCastShadows( true );
	light->setLightMask( 0x00000000 );

	// set initial camera position and speed
	mCamera->setPosition( 0, 120, 100 );

	// Ground
	Ogre::MeshManager::getSingleton().createPlane("ground", Ogre::ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME, Ogre::Plane(Ogre::Vector3::UNIT_Y, 0), 10000, 10000, 20, 20, true, 1, 6, 6, Ogre::Vector3::UNIT_Z);
	Ogre::Entity* ground = mSceneMgr->createEntity("Ground", "ground");
	ground->setMaterialName("Examples/Instancing/Misc/Grass");
	ground->setCastShadows(true);
	mSceneMgr->getRootSceneNode()->attachObject(ground);

	// InstanceManager
	mInstanceManager = mSceneMgr->createInstanceManager(
		"InstanceMgr-HWInstancingVTF", "Sinbad.mesh",
		Ogre::ResourceGroupManager::AUTODETECT_RESOURCE_GROUP_NAME,
		Ogre::InstanceManager::HWInstancingVTF,
		NUM_INST_ROW * NUM_INST_COLUMN,
		Ogre::IM_USEALL | Ogre::IM_VTFBONEMATRIXLOOKUP
	);

	// Create the instanced entities
	srand( (unsigned)time( NULL ) );
	for( int i=0; i<NUM_INST_ROW; ++i )
	{
		for( int j=0; j<NUM_INST_COLUMN; ++j )
		{
			Ogre::InstancedEntity *ent = mInstanceManager->createInstancedEntity( "Examples/Instancing/VTF/HW/LUT/Sinbad" );
			//ent->setScale(Ogre::Vector3(1.0f) * 100.0f);
			mEntities.push_back( ent );
			Ogre::AnimationState *anim = ent->getAnimationState( "RunBase" );
			anim->setEnabled( true );
			anim->addTime( (rand() / (float)0xFFFFFFFF) * 10);
			mAnimations.insert( anim );
			mMovedInstances.push_back( ent );
			ent->setOrientation(Ogre::Quaternion(Ogre::Radian((rand() / (float)0xFFFFFFFF) * 10 * 3.14159265359f), Ogre::Vector3::UNIT_Y));
			ent->setPosition( Ogre::Vector3(mEntities[0]->getBoundingRadius() * (i - NUM_INST_ROW * 0.5f), 0, mEntities[0]->getBoundingRadius() * (j - NUM_INST_COLUMN * 0.5f)) );
		}
	}

}

//-------------------------------------------------------------------------------------
bool TutorialApplication::frameRenderingQueued(const Ogre::FrameEvent& evt)
{
	Ogre::Quaternion rotVal;
	Ogre::Real fMovSpeed = 1.0f;

	for(std::vector<Ogre::InstancedEntity*>::const_iterator itor = mMovedInstances.begin(); itor != mMovedInstances.end(); ++itor)
	{
		//Calculate bounces
		Ogre::InstancedEntity* pEnt = *itor;
		Ogre::Vector3 entityPos = pEnt->getPosition();
		Ogre::Vector3 planeNormal = Ogre::Vector3::ZERO;
		if( pEnt->getPosition().x < -5000.0f )
		{
			planeNormal = Ogre::Vector3::UNIT_X;
			entityPos.x = -4999.0f;
		}
		else if( pEnt->getPosition().x > 5000.0f )
		{
			planeNormal = Ogre::Vector3::NEGATIVE_UNIT_X;
			entityPos.x = 4999.0f;
		}
		else if( pEnt->getPosition().z < -5000.0f )
		{
			planeNormal = Ogre::Vector3::UNIT_Z;
			entityPos.z = -4999.0f;
		}
		else if( pEnt->getPosition().z > 5000.0f )
		{
			planeNormal = Ogre::Vector3::NEGATIVE_UNIT_Z;
			entityPos.z = 4999.0f;
		}

		if( planeNormal != Ogre::Vector3::ZERO )
		{
			const Ogre::Vector3 vDir(pEnt->getOrientation().xAxis().normalisedCopy() );
			Ogre::Vector3 normDir = planeNormal.reflect( vDir ).normalisedCopy();
			Ogre::Vector3 xVec = Ogre::Vector3::UNIT_Y.crossProduct( normDir );
			xVec.normalise();
			Ogre::Vector3 yVec = normDir.crossProduct( xVec );
			yVec.normalise();
			rotVal.FromAxes( xVec, yVec, normDir );
			pEnt->setOrientation( rotVal, false );
			pEnt->setPosition( entityPos, false);
		}

		Ogre::Vector3 transAmount = Ogre::Vector3::UNIT_X * evt.timeSinceLastFrame * fMovSpeed;
		pEnt->setPosition( pEnt->getPosition() + pEnt->getOrientation() * transAmount );
	}

	return BaseApplication::frameRenderingQueued(evt);
}

//-------------------------------------------------------------------------------------
void TutorialApplication::destroyScene(void)
{
	if(mInstanceManager != NULL)
	{
		mInstanceManager->cleanupEmptyBatches();
		mSceneMgr->destroyInstanceManager(mInstanceManager);
		mInstanceManager = NULL;
	}
	BaseApplication::destroyScene();
}

#if OGRE_PLATFORM == OGRE_PLATFORM_WIN32
#define WIN32_LEAN_AND_MEAN
#include "windows.h"
#endif

#ifdef __cplusplus
extern "C" {
#endif

#if OGRE_PLATFORM == OGRE_PLATFORM_WIN32
    INT WINAPI WinMain( HINSTANCE hInst, HINSTANCE, LPSTR strCmdLine, INT )
#else
    int main(int argc, char *argv[])
#endif
    {
        // Create application object
        TutorialApplication app;

        try {
            app.go();
        } catch( Ogre::Exception& e ) {
#if OGRE_PLATFORM == OGRE_PLATFORM_WIN32
            MessageBox( NULL, e.getFullDescription().c_str(), "An exception has occured!", MB_OK | MB_ICONERROR | MB_TASKMODAL);
#else
            std::cerr << "An exception has occured: " <<
                e.getFullDescription().c_str() << std::endl;
#endif
        }

        return 0;
    }

#ifdef __cplusplus
}
#endif

Code: Select all

material Examples/Instancing/VTF/HW/LUT/Sinbad : Examples/Instancing/HW_VTF_LUT
{
	set_texture_alias	DiffuseMap	sinbad_body.tga
}
The bad thing is, that only the eyes are visible.
Result
Result
screenshot01222013_115003076.jpg (44.19 KiB) Viewed 9467 times
Any idea what's wrong?

Thanks,
Transporter

TheSHEEEP
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 972
Joined: Mon Jun 02, 2008 6:52 pm
Location: Berlin
x 63

Re: New InstanceManager: Instancing done the right way

Post by TheSHEEEP »

That looks... scary.

Does it work with other meshes that use multiple materials?
My site! - Have a look :)
Also on Twitter - extra fluffy

bstone
OGRE Expert User
OGRE Expert User
Posts: 1920
Joined: Sun Feb 19, 2012 9:24 pm
Location: Russia
x 199

Re: New InstanceManager: Instancing done the right way

Post by bstone »

Does your mesh use shared vertices between submeshes?

Transporter
Minaton
Posts: 933
Joined: Mon Mar 05, 2012 11:37 am
Location: Germany
x 110

Re: New InstanceManager: Instancing done the right way

Post by Transporter »

I don't know about shared vertices! I use the Sinbad.zip package from the SDK (https://bitbucket.org/sinbad/ogre/src/3 ... at=default).

I have only tested the robot mesh from the example. It's a single material mesh and it's working.

bstone
OGRE Expert User
OGRE Expert User
Posts: 1920
Joined: Sun Feb 19, 2012 9:24 pm
Location: Russia
x 199

Re: New InstanceManager: Instancing done the right way

Post by bstone »

Transporter wrote:I don't know about shared vertices!
It's time to get acquainted with them. Any mesh using more than one material will have submeshes. Ogre supports a simple way of compacting mesh data by reusing one and the same vertex data between all submeshes. That is optional (usually depends on how your exporter prefers it) but is not supported by the hardware instancing implementation as far as I remember.

Not much can be done about the resources from Sinbad.zip but you will have to find a way to export your meshes w/o shared vertices for the content you author.

JordanWeber
Gnoblar
Posts: 7
Joined: Fri Feb 01, 2013 1:18 pm
x 1

Re: New InstanceManager: Instancing done the right way

Post by JordanWeber »

The issue with the sinbad model isn't shared vertices. It is with the instance manager. 1 instance manager uses 1 submesh. (Look at the first page of this thread to see how to setup submeshes).

Also, you can use the following to verify if it is using sharedVertices:
bool sharedVerts = mesh->getSubMesh(i)->useSharedVertices;

Edit2:
There is a known bug in the tracker currently involving HW instancing and submeshes not showing (or showing up distorted), which reflects testing of my submeshes. So for now, if you have multiple submeshes you may need to stick with TextureVTF / ShaderBased until this gets resolved.



I am having issues with submeshes, and useSharedVertices = false; on the 2nd call to: createInstanceManager, it crashes with numVerts = 0.

Heres a snippet of my current code:

Code: Select all

        InstancedEntity* mainInst;

        //Create one instance mgr per submesh, managerNames contains unique random names
        Ogre::InstanceManager* mgr;
        //size_t i = 0; if ( i == 0 )
        for( size_t i = 0; i < mesh->getNumSubMeshes(); ++i )
        {
            static const InstanceManager::InstancingTechnique technique = InstanceManager::ShaderBased;
            mgr = mSceneMgr->createInstanceManager( meshName + " SubEnt:" + Ogre::StringConverter::toString(i), meshName,
                              ResourceGroupManager::AUTODETECT_RESOURCE_GROUP_NAME, technique, 5 /*numInstancesPerBatch*/, IM_USEALL, i /*submeshnum*/);
            InstancedEntity *instEnt = mgr->createInstancedEntity("Examples/Instancing/ShaderBased/Robot" );
            mScnNode->attachObject( instEnt );

            //This function will tell the other entities to use the resources from the first entity. Saves RAM & CPU
            if ( i ) //if( i != 0 )
            {
                    if( !instEnt->shareTransformWith( mainInst ) )
                    {
                    //If we get here, the instanced entity doesn't have a skeleton to share or the technique selected doesn't support skeletal animation
                    }
            } else
                mainInst = instEnt;
        }
Crash happens in D3D9HardwareBufferManagerBase::createVertexBuffer
from: setupVertices( baseSubMesh ); in InstanceBatch::build( const SubMesh* baseSubMesh )

Which it seems to be validated a few steps prior:
checkSubMeshCompatibility (also note, if there really is issues with sharedvertices, then perhaps have a check here?)

I've noticed the 2nd instance manager gets the following value causing the numVerts to = 0;
mInstancesPerBatch = 0
So the built mesh from the following line is 0 and causes the assert, from InstanceBatchShader::setupVertices :
thisVertexData->vertexCount = baseVertexData->vertexCount * mInstancesPerBatch;

Edit:
I found out the issue was because 1 of my submeshes somehow had 89 bone references (>80 for arraysize in calcMaxNumInstances), causing the following values to be set:
InstanceBatchShader::calculateMaxNumInstances
maxInstPerBatch = 0
mInstancesPerBatch = std::min( maxInstPerBatch, mInstancesPerBatch );

jadam
Kobold
Posts: 26
Joined: Thu May 31, 2012 7:05 pm

Re: New InstanceManager: Instancing done the right way

Post by jadam »

Hi,

I'm having an issue with the instanced entities absolutely not showing up, no matter what technique I use. They consist of one submesh and are based on manual object (but tried some .mesh resources too from the ogre samples, no success either).
The exact same code with regular entities makes them show up but when I switch the code only to create instanced entities, nothing is visible.

Does anyone have any idea what can be the issue?
Also, I tried a couple of different numberOfInstancedPerBatch, still no success.

Code: Select all

m_pInstanceMgr= m_pSceneManager->createInstanceManager(instMgrName,meshName, Ogre::ResourceGroupManager::AUTODETECT_RESOURCE_GROUP_NAME,Ogre::InstanceManager::TextureVTF,10,Ogre::IM_USEALL);
m_pInstancedEntity = m_pSceneManager->createInstancedEntity(materialName,instSceneMgrName); //I also tried this way: m_pInstancedEntity = m_pInstanceMgr->createInstancedEntity(materialName);

m_pInstEntitySceneNode= m_pSomeSceneNode->createChildSceneNode(instSceneNodeName,Ogre::Vector3(20.0,20.0,0.0));
m_pInstEntitySceneNode->attachObject(m_pInstancedEntity );

//I also tried to set the sceneNode visible, the entity visible, set directly the entity's position, etc, but still no success
Thanks,
Adam.

cloud
Gremlin
Posts: 196
Joined: Tue Aug 08, 2006 6:45 pm
x 14

Re: New InstanceManager: Instancing done the right way

Post by cloud »

I was thinking of adding animation data into a texture essentially write out each bones full transform for each animation for each frame with a fps of about 60 into a texture.
Then pass into shader an animation index + a frame index + the vertex bone index -> texture lookup for the dual quaternion to use. I've done this for a single entity and the animation works fine.

The question I have is primarily is this a good idea? And if it is how I should go about adding this. If VTF means its fully animated already and presumably does all the matrix computations per frame for all the instances, then I can't use that... if I use the HW non VTF method then bone indexs and weights has been removed and the uv's carry the matrices for the instance positions, and I can't use that either.

If I use the shader method then I get an array of world matrices for the instance positions which I could use provided it doesn't get rid of the bone indices and weights, I could subclass or modify it so if there was a skeleton it would add in the animation texture creation stuff and update the frame data, but then I'd be limiting the instance count, is there a way to use HW VTF but just have positions of the instances in the texture? would it be slower than the array of world matrices approach.

Any suggestions as to how to best go about this?

cloud
Gremlin
Posts: 196
Joined: Tue Aug 08, 2006 6:45 pm
x 14

Re: New InstanceManager: Instancing done the right way

Post by cloud »

So I created a new technique, called it InstanceBatchHW_VTFAnim subclassed from InstanceBatchHW but now I'm starting to think it would have been better if I'd just added the animation texture and lookup stuff to InstanceBatchHW, *sighs* still it works... But is currently limited to playing one animation and if the skeletons changed or animations added then the texture is out dated and obviously it won't work right, also it needs the instance id to do the lookup and that limits the shader profiles + only dual quaternion method currently. A shader might look like this

Code: Select all

#version 150

// Vertex input
in vec4 vertex;
in vec3 normal;
in vec4 uv0;
in vec4 uv1;
in vec4 uv2;
in vec4 uv3;
in vec3 tangent;
in vec4 blendIndices;
in vec4 blendWeights;

// Parameters
uniform mat4 viewProjMatrix;
uniform vec4  animationTextureSize;
uniform sampler2D sAnimation;
uniform int animation_offset[200];

// Output
out vec4 oUv;
//out vec3 oNormal;


mat2x4 GetBoneMatrix(int idx, float boneIdx)
{
    idx += 2 * int(boneIdx); // storing dual quaternions
    
    ivec2 uv;
    uv.x = idx % int(animationTextureSize.x);
    uv.y = idx / int(animationTextureSize.x);
    
    mat2x4 result;
    result[0]= texelFetch(sAnimation, uv, 0);
    result[1]= texelFetch(sAnimation, uv + ivec2(1, 0), 0);
    
    return result;
}

vec3 calculateBlendPosition(vec3 position, mat2x4 blendDQ)
{
	vec3 blendPosition = position + 2.0*cross(blendDQ[0].yzw, cross(blendDQ[0].yzw, position) + blendDQ[0].x*position);
	vec3 trans = 2.0*(blendDQ[0].x*blendDQ[1].yzw - blendDQ[1].x*blendDQ[0].yzw + cross(blendDQ[0].yzw, blendDQ[1].yzw));
	blendPosition += trans;
    
	return blendPosition;
}

vec3 calculateBlendNormal(vec3 normal, mat2x4 blendDQ)
{
	return normal + 2.0*cross(blendDQ[0].yzw, cross(blendDQ[0].yzw, normal) + blendDQ[0].x*normal);
}



void main(void)
{
    //  MUST use varibale instanceIdx or animation_offset size gets messed up
    int instanceIdx = gl_InstanceID;
    int offset = animation_offset[instanceIdx];
    
    mat2x4 blendDQ  = GetBoneMatrix(offset, int(blendIndices[0]));
	mat2x4 blendDQ2 = GetBoneMatrix(offset, int(blendIndices[1]));
    
    // Accurate antipodality handling. For speed increase, remove the following line
	if (dot(blendDQ[0], blendDQ2[0]) < 0.0) blendDQ2 *= -1.0;
	
	// Blend the dual quaternions based on the weights
	blendDQ *= blendWeights.x;
	blendDQ += blendWeights.y*blendDQ2;
    
	// Normalize the resultant dual quaternion
	blendDQ /= length(blendDQ[0]);
    
    vec4 blendPos = vec4(calculateBlendPosition(vertex.xyz, blendDQ), 1.0);
	vec3 blendNorm = calculateBlendNormal(normal, blendDQ);
    
    
	mat4 worldMatrix;
	worldMatrix[0] = uv1;
	worldMatrix[1] = uv2;
	worldMatrix[2] = uv3;
	worldMatrix[3] = vec4( 0, 0, 0, 1 );
    
	vec4 worldPos		= blendPos * worldMatrix;
	vec3 worldNorm		= blendNorm * mat3(worldMatrix);
    
	//Transform the position
	gl_Position			= viewProjMatrix * worldPos;
	
    oUv = uv0;
	//oNormal = worldNorm;
}


Should I try and submit a patch for this? It should be quick in the sense that no animation needs to be done on the cpu, but its crap in the sense its not very general. I could maybe add support for 2 animations and a weight between them, then just lerp'ing between the two states, which would probably look alright. I've had some trouble getting it to work in the NewInstancing sample code so I don't have any frame rate comparisons as of yet, in fact I'm going to give up trying unless its actually wanted.

User avatar
AshMcConnell
Silver Sponsor
Silver Sponsor
Posts: 605
Joined: Fri Dec 14, 2007 11:44 am
Location: Northern Ireland
x 15
Contact:

Re: New InstanceManager: Instancing done the right way

Post by AshMcConnell »

Hi Folks,

I tried to use the new InstanceManager last night to reduce batching for my trees, but for some reason couldn't manage to position the instances. I created an InstanceManager for each type of tree (35 types in total - with a total of ~1300 instances). I created an InstancedEntity for each instance and attached it to a SceneNode with position and orientation, but for some reason all the instances are appearing at 0,0,0. I also noticed that even if I *didn't* attach the entity to the SceneNode it was visible (At 0,0,0), is this "normal"?

Any ideas what I have missed here? I'll give it another try when I get home from work, just wanted to check I hadn't missed anything silly, or a gotcha :)

Thanks for your help!

(The good news is that even with all the trees visible I'm getting 30% more FPS :) )

Post Reply