New InstanceManager: Instancing done the right way
-
- Gnome
- Posts: 363
- Joined: Wed Feb 25, 2009 6:15 pm
- Location: UK
- x 8
Re: New InstanceManager: Instancing done the right way
Does this work with mogre, and if it doesn't, would it be difficult to make it work with it?
My scene has about 1200 trees which make a huge difference to speed (visible gives me 25-50FPS, invisible gives me a near-constant 60), using about 12 different models. At the moment I'm using static geometry, which is the best solution so far but also makes the game use up about twice as much RAM.
Though I tried using ogre 1.7's instance manager and it wasn't much faster than regular entities. I'm assuming this instancemanager is a lot better than that, right?
My scene has about 1200 trees which make a huge difference to speed (visible gives me 25-50FPS, invisible gives me a near-constant 60), using about 12 different models. At the moment I'm using static geometry, which is the best solution so far but also makes the game use up about twice as much RAM.
Though I tried using ogre 1.7's instance manager and it wasn't much faster than regular entities. I'm assuming this instancemanager is a lot better than that, right?
-
- Goblin
- Posts: 287
- Joined: Mon Dec 08, 2008 4:49 pm
- x 10
Re: New InstanceManager: Instancing done the right way
The speed improvement depends on a lot of factors. What I think are the two most important factors:
1) does the target hardware support some kind of hardware instancing?
2) do you use scenenodes to position the trees, or do you set the positions of the instancedEntities directly?
I find the extensive test that was done earlier in the thread by Mattan very helpful (although the exact figures really depend on your hardware and your scene)
1) does the target hardware support some kind of hardware instancing?
2) do you use scenenodes to position the trees, or do you set the positions of the instancedEntities directly?
I find the extensive test that was done earlier in the thread by Mattan very helpful (although the exact figures really depend on your hardware and your scene)
-
- Gnome
- Posts: 363
- Joined: Wed Feb 25, 2009 6:15 pm
- Location: UK
- x 8
Re: New InstanceManager: Instancing done the right way
uh, I think so? The normal ogre instancing thing requires a shaderjonim8or wrote:The speed improvement depends on a lot of factors. What I think are the two most important factors:
1) does the target hardware support some kind of hardware instancing?
I set the positions and stuff of "InstancedGeometry.InstancedObject"2) do you use scenenodes to position the trees, or do you set the positions of the instancedEntities directly?
Hrm, my instanced things are all not moving and have no animation with shadows optional... the speed increase doesn't look that significant for something like thatI find the extensive test that was done earlier in the thread by Mattan very helpful (although the exact figures really depend on your hardware and your scene)
-
- Halfling
- Posts: 52
- Joined: Tue Apr 26, 2011 9:13 am
- Location: Russia, Tver
- x 2
Re: New InstanceManager: Instancing done the right way
@dark_sylinc
1) Can you remove warning C4512: 'Ogre::InstanceManager' : assignment operator could not be generated (I use Visual Studio 2005 sp1)
For example you can declare assignment operator and copy constructor as private functions and don`t implement it.
2) Add #include "OgreInstanceManager.h" #include "OgreInstancedEntity.h" #include "OgreInstanceBatch.h" to common Ogre header "Ogre.h"
https://sourceforge.net/tracker/?func=d ... tid=302997
3) See at constructor InstanceBatch::InstanceBatch
How do you think, what will be in release build?
Regards,
Vadim
1) Can you remove warning C4512: 'Ogre::InstanceManager' : assignment operator could not be generated (I use Visual Studio 2005 sp1)
For example you can declare assignment operator and copy constructor as private functions and don`t implement it.
2) Add #include "OgreInstanceManager.h" #include "OgreInstancedEntity.h" #include "OgreInstanceBatch.h" to common Ogre header "Ogre.h"
https://sourceforge.net/tracker/?func=d ... tid=302997
3) See at constructor InstanceBatch::InstanceBatch
Code: Select all
{
assert( mInstancesPerBatch );
//Force batch visibility to be always visible. The instanced entities
//have individual visibility flags. If none matches the scene's current,
//then this batch won't rendered.
mVisibilityFlags = std::numeric_limits<Ogre::uint32>::max();
if( indexToBoneMap )
assert( !(meshReference->hasSkeleton() && indexToBoneMap->empty()) );
mFullBoundingBox.setExtents( -Vector3::ZERO, Vector3::ZERO );
mName = batchName;
}
Code: Select all
if( indexToBoneMap ) /* m2codeGEN fix */ {
assert( !(meshReference->hasSkeleton() && indexToBoneMap->empty()) ); /* m2codeGEN fix */ }
mFullBoundingBox.setExtents( -Vector3::ZERO, Vector3::ZERO );
Vadim
-
- OGRE Team Member
- Posts: 5433
- Joined: Sat Jul 21, 2007 4:55 pm
- Location: Buenos Aires, Argentina
- x 1341
Re: New InstanceManager: Instancing done the right way
A level 4 warning eh? I guess that's why I missed it (VS 2008 Express here).m2codeGEN wrote: 1) Can you remove warning C4512: 'Ogre::InstanceManager' : assignment operator could not be generated (I use Visual Studio 2005 sp1)
For example you can declare assignment operator and copy constructor as private functions and don`t implement it.
I'm not sure how to proceed here. I definitely do NOT want accidental copies, which could happen even if the function is private when done from inside InstanceManager. The reason behind this is to raise an alert on future developers that InstanceManager is maintained by SceneManager, and the name is important, so you can't go carelessly making copies. Besides there shouldn't be a need for that.
Another way to workaround this is to include a pragma, an which may I'm not too keen with either.
I'm hearing for options or arguments.
Done. For some reason Mercurial refused the patch. I hate that in Hg. Don't know why it happens so often.2) Add #include "OgreInstanceManager.h" #include "OgreInstancedEntity.h" #include "OgreInstanceBatch.h" to common Ogre header "Ogre.h"
https://sourceforge.net/tracker/?func=d ... tid=302997
So I did it manually.
Nice catch! Although if a properly implemented assert is used this shouldn't happen, it's better safe than sorry.3) See at constructor InstanceBatch::InstanceBatchHow do you think, what will be in release build?Code: Select all
{ assert( mInstancesPerBatch ); //Force batch visibility to be always visible. The instanced entities //have individual visibility flags. If none matches the scene's current, //then this batch won't rendered. mVisibilityFlags = std::numeric_limits<Ogre::uint32>::max(); if( indexToBoneMap ) assert( !(meshReference->hasSkeleton() && indexToBoneMap->empty()) ); mFullBoundingBox.setExtents( -Vector3::ZERO, Vector3::ZERO ); mName = batchName; }
Code: Select all
if( indexToBoneMap ) /* m2codeGEN fix */ { assert( !(meshReference->hasSkeleton() && indexToBoneMap->empty()) ); /* m2codeGEN fix */ } mFullBoundingBox.setExtents( -Vector3::ZERO, Vector3::ZERO );
Thanks
Dark Sylinc
-
- Halfling
- Posts: 53
- Joined: Mon Dec 05, 2011 9:43 pm
- x 1
Re: New InstanceManager: Instancing done the right way
Hi!
I dont know if this is the right place for the question, if it isn't, sry =(
The question is How can I generate a mesh supported by the instanceManager (i.e. without shares vertices) ?, I had all the system ready to work, and with the robot mesh work great!, but with my mesh (exported from blender) the system dont work. With the debug I found that the shares vertices line is where crash it =( (the robot mesh has set in false, my mesh in true)
Thanks in advance!
I dont know if this is the right place for the question, if it isn't, sry =(
The question is How can I generate a mesh supported by the instanceManager (i.e. without shares vertices) ?, I had all the system ready to work, and with the robot mesh work great!, but with my mesh (exported from blender) the system dont work. With the debug I found that the shares vertices line is where crash it =( (the robot mesh has set in false, my mesh in true)
Thanks in advance!
-
- Halfling
- Posts: 52
- Joined: Tue Apr 26, 2011 9:13 am
- Location: Russia, Tver
- x 2
Re: New InstanceManager: Instancing done the right way
I found another issue in InstanceManager. It`s about double precision problem and big scenes. It`s vertex shaking or Jitting
After some time I will give a code that can fix this.
But first, let's take a look at the problem. Let us consider only the technique HardwareInstaceBasic.
1) HardwareInstaceBatch is the successor of the classes Renderandeble and MovebleObject. Thus he should be attached to Scene Nodes to be rendered. Accordingly, when it can be drawn, the calculation of MODELVIEW matrix is done on the central processor (in doubles). But
So gl_ModelViewProjection is a combination of the view and projection matrix.
2) In a Vertex Shader position of vertex calculates in world coords. But in glsl 4.0-- mat4 and vec4 is a float type.
Prospective ways of the decision of problems.
1) As attributes of the vertices transfer calculated on the CPU modelview matrix. Need modify InstanceBatchHW::updateVertexBuffer
But using a static buffer immediately becomes not possible.
2) Use relative coordinates entities when recording in the vertex buffer. And in method InstanceBatchHW::getWorldTransforms returns transform of parent SceneNode (let`s set him position in center of all entities). Of course there's a problem, if the fact will be too far removed from the SceneNode position. But it is better than the jitter in world coords.
need modify Vertex Shader like this
After some time I will give a code that can fix this.
But first, let's take a look at the problem. Let us consider only the technique HardwareInstaceBasic.
1) HardwareInstaceBatch is the successor of the classes Renderandeble and MovebleObject. Thus he should be attached to Scene Nodes to be rendered. Accordingly, when it can be drawn, the calculation of MODELVIEW matrix is done on the central processor (in doubles). But
Code: Select all
void InstanceBatchHW::getWorldTransforms( Matrix4* xform ) const { *xform = Matrix4::IDENTITY;}
2) In a Vertex Shader position of vertex calculates in world coords. But in glsl 4.0-- mat4 and vec4 is a float type.
Code: Select all
uniform mat4 viewProjMatrix;
void main(void)
{
mat4 worldMatrix;
worldMatrix[0] = uv1;
worldMatrix[1] = uv2;
worldMatrix[2] = uv3;
worldMatrix[3] = vec4( 0, 0, 0, 1 );
vec4 worldPos = vertex * worldMatrix;
gl_Position = viewProjMatrix * worldPos;
Prospective ways of the decision of problems.
1) As attributes of the vertices transfer calculated on the CPU modelview matrix. Need modify InstanceBatchHW::updateVertexBuffer
Code: Select all
attribute vec4 vertex;
attribute vec3 normal;
attribute mat4 modelView
void main()
{
vec4 worldPos = vertex * modelView;
2) Use relative coordinates entities when recording in the vertex buffer. And in method InstanceBatchHW::getWorldTransforms returns transform of parent SceneNode (let`s set him position in center of all entities). Of course there's a problem, if the fact will be too far removed from the SceneNode position. But it is better than the jitter in world coords.
need modify Vertex Shader like this
Code: Select all
attribute vec4 vertex;
attribute vec3 normal;
attribute mat4 relativeTransform; // for small writing
uniform mat4 modelViewProjection;
void main()
{
vec4 vertexPos = relativeTransform * vertex;
gl_Position = modelViewProjection * vertexPos;
-
- OGRE Team Member
- Posts: 5433
- Joined: Sat Jul 21, 2007 4:55 pm
- Location: Buenos Aires, Argentina
- x 1341
Re: New InstanceManager: Instancing done the right way
Hi, thanks again for the feedback. I know english isn't your mother language but you should try to exercise it more often, your grammar makes it hard to understand where you're getting at.
I'm very familiar with extremely large scenes.
For example Havok has it's own method called "shiftAllObjects" to solve exactly this problem
Cheers
Dark Sylinc
It's a common problem in all rendering engines. Normal Entity rendering has the same problem.m2codeGEN wrote:I found another issue in InstanceManager. It`s about double precision problem and big scenes. It`s vertex shaking or Jitting
I'm very familiar with extremely large scenes.
I'm afraid that's quite a big oversimplification.m2codeGEN wrote:Accordingly, when it can be drawn, the calculation of MODELVIEW matrix is done on the central processor (in doubles).
- In D3D, calculation will be done single or double precision depending on the floating point mode.
- In GL, you'll have to set the fp control word manually and check it periodically to be sure that you're running at double precision mode. Chances are this is true, but not guaranteed.
- In both cases, the calculation may be performed at double precision internally, but it is eventually truncated/rounded to single precision when stored in memory, although alleviated when compiled with OGRE_DOUBLE_PRECISION.
- In both cases, the problem is NOT because it's done in doubles (although it helps); but rather because the world matrix multiplication is done in the shader. Because we do (vertex * world) * viewProj; the first parenthesis becomes too large and loses a lot of precision; then multiplying by viewProj will resize those numbers down, but the precision has already been lost. The solution is to first perform (world * viewProj) and then multiply it by the vertex. Watch out for the order of operations. Needless to say, when your technique (i.e. HW VTF) uses more than one bone per vertex, the number of operations grows very quick because you're performing lots of (world*viewProj). Performing "relative" world transforms for bone poses (as you propose) can help alleviating the problem too.
- You'll note the same jittering & shaking when rendering regular Entities when skeletally animated in the vertex shader.
- Calling SceneManager::setCameraRelativeRendering( true ) will greatly alleviate these problems; but it's possible you will still see some jittering and shaking in the small details (i.e. the eyes of a 1.75m character).
- Be sure all your libraries (Ogre included) is compiled with Floating Point Model 'precise' or 'strict'. Ogre default parameter is 'fast' which is very innacurate.
- In D3D9, set the floating point mode to 'Consistent'
- In all rendersystems, use _controlfp and periodically check it's still in double processing mode.
- Compile with floating point model 'precise'
- Call SceneManager::setCameraRelativeRendering( true )
For example Havok has it's own method called "shiftAllObjects" to solve exactly this problem
Cheers
Dark Sylinc
-
- Halfling
- Posts: 52
- Joined: Tue Apr 26, 2011 9:13 am
- Location: Russia, Tver
- x 2
Re: New InstanceManager: Instancing done the right way
Well, I will try to give to it more attention. Often I can't be understood and in a native languagedark_sylinc wrote:I know english isn't your mother language but you should try to exercise it more often, your grammar makes it hard to understand where you're getting at.
I have already build OGRE with the OGRE_DOUBLE_PRECISION and use GL render system. Ogre::Entity attached to SceneNode visualized without graphic artifacts. The problem is very similar to this http://www.fxpression.com/phpBB3/viewto ... 72b2015b42
Call SceneManager::setCameraRelativeRendering( true ) reduces fps in 2-3 times (we can`t use static vertex buffer).
But the universal decision doesn't exist. I suggest to make a small patch which will carry out calculations in relative coordinates, and you then estimate it and if necessary will make changes to a repository.
-
- Halfling
- Posts: 52
- Joined: Tue Apr 26, 2011 9:13 am
- Location: Russia, Tver
- x 2
Re: New InstanceManager: Instancing done the right way
I can`t submit path (SourceForge.net executed with error on page). So diffs will be posted here.
NewInstacing. Small performance improvement
Updated
patches in SourceForge.net
https://sourceforge.net/tracker/?func=d ... tid=302997
https://sourceforge.net/tracker/?func=d ... tid=302997
NewInstacing. Small performance improvement
Code: Select all
# HG changeset patch
# User Vadim Sergeev <m2codeGEN@rambler.ru>
# Date 1328265980 -14400
# Branch v1-8
# Node ID 2127c412f549c090a1d5254edac8ebbf199017c4
# Parent 4ac925a83e6a315eedb672124ab08d81b02653bd
NewInstacing. Small performance improvement
diff -r 4ac925a83e6a -r 2127c412f549 OgreMain/src/OgreInstanceBatch.cpp
--- a/OgreMain/src/OgreInstanceBatch.cpp Tue Jan 31 02:02:13 2012 -0600
+++ b/OgreMain/src/OgreInstanceBatch.cpp Fri Feb 03 14:46:20 2012 +0400
@@ -166,6 +166,8 @@
//InstancedEntity is not part of the scene graph (i.e. ultimate parent is root node)
//to avoid unnecessary wasteful calculations
mVisible |= (*itor)->findVisible( mCurrentCamera );
+ if (mVisible)
+ break; // performance trick
++itor;
}
}
@@ -219,13 +221,12 @@
for( size_t i=0; i<numFloats >> 2; i += 3 )
{
- const Vector3 worldTrans( mat3x4[(i+0) * 4 + 3], mat3x4[(i+1) * 4 + 3],
- mat3x4[(i+2) * 4 + 3] );
- const Vector3 newPos( worldTrans - cameraRelativePosition );
+ const Vector3 worldTrans(mat3x4[(i+0) * 4 + 3], mat3x4[(i+1) * 4 + 3], mat3x4[(i+2) * 4 + 3]);
+ const Vector3 newPos = worldTrans - cameraRelativePosition;
- mat3x4[(i+0) * 4 + 3] = newPos.x;
- mat3x4[(i+1) * 4 + 3] = newPos.y;
- mat3x4[(i+2) * 4 + 3] = newPos.z;
+ mat3x4[(i+0) * 4 + 3] = (float)newPos.x;
+ mat3x4[(i+1) * 4 + 3] = (float)newPos.y;
+ mat3x4[(i+2) * 4 + 3] = (float)newPos.z;
}
}
//-----------------------------------------------------------------------
patches in SourceForge.net
https://sourceforge.net/tracker/?func=d ... tid=302997
https://sourceforge.net/tracker/?func=d ... tid=302997
-
- OGRE Team Member
- Posts: 5433
- Joined: Sat Jul 21, 2007 4:55 pm
- Location: Buenos Aires, Argentina
- x 1341
Re: New InstanceManager: Instancing done the right way
Hi!
Have you tested there is actual performance improvement?
Why did you change the assignment on constructor by assignment by '=' operator? Is there a special reason?
The C cast will be changed to static_cast; which is the C++ way of handling conversions.
The if(mVisible) break; can be replaced by "while( itor != end && !mVisible )" which tends to compile into branch-prediction-friendlier code.
I can't download the local translation patch I get a php file of SF.Net website instead :\
Have you tested there is actual performance improvement?
Why did you change the assignment on constructor by assignment by '=' operator? Is there a special reason?
The C cast will be changed to static_cast; which is the C++ way of handling conversions.
The if(mVisible) break; can be replaced by "while( itor != end && !mVisible )" which tends to compile into branch-prediction-friendlier code.
I can't download the local translation patch I get a php file of SF.Net website instead :\
-
- Halfling
- Posts: 52
- Joined: Tue Apr 26, 2011 9:13 am
- Location: Russia, Tver
- x 2
Re: New InstanceManager: Instancing done the right way
Hi! This is small improving for ShaderBased and VTFdark_sylinc wrote: Have you tested there is actual performance improvement?
As a whole it is the equivalent operations. Assignment ' = ' operator returns the link to Vector3, but to read the code begins hardly easier.dark_sylinc wrote: Why did you change the assignment on constructor by assignment by '=' operator? Is there a special reason?
You are absolutely right. It is necessary to add that if static_cast it is not has gone right, it will be applied reinterpret_cast.dark_sylinc wrote: The C cast will be changed to static_cast; which is the C++ way of handling conversions.
For the built in data types (such as float, int, etc.) C application cast is quite admissible, since the code is easily read. For the user data types it is forbidden (our corporate standards)
I have simply shown a way for improving. Such variant more can will approach for the working version.dark_sylinc wrote: The if(mVisible) break; can be replaced by "while( itor != end && !mVisible )" which tends to compile into branch-prediction-friendlier code.
:\
Code: Select all
void InstanceBatch::updateVisibility(void)
{
bool bVisible = mVisible = false; // <-- may be register variable
if (!mInstancedEntities.empty())
{
InstancedEntityVec::const_iterator itor = mInstancedEntities.begin();
InstancedEntityVec::const_iterator end = mInstancedEntities.end();
do {
bVisible = (*itor)->findVisible( mCurrentCamera );
} while(!bVisible && ++itor != end );
}
mVisible = bVisible;
}
Yesterday I too had difficulties with this site. Today all works.dark_sylinc wrote: I can't download the local translation patch I get a php file of SF.Net website instead :\
But all above described are trifles It is much more important, that you have paid the attention to this patch since I could forget casually something
http://sourceforge.net/tracker/?func=de ... tid=302997
-
- OGRE Team Member
- Posts: 5433
- Joined: Sat Jul 21, 2007 4:55 pm
- Location: Buenos Aires, Argentina
- x 1341
Re: New InstanceManager: Instancing done the right way
The reason I ask is that it may not be a performance improvement, but rather worse. Do you have a comparison in milliseconds?m2codeGEN wrote:Hi! This is small improving for ShaderBased and VTF
Oh my God. NO.m2codeGEN wrote: I have simply shown a way for improving. Such variant more can will approach for the working version.Or use the pointer on the pointer instead of iterators. But then the container should to be serial is mandatory.Code: Select all
void InstanceBatch::updateVisibility(void) { bool bVisible = mVisible = false; // <-- may be register variable if (!mInstancedEntities.empty()) { InstancedEntityVec::const_iterator itor = mInstancedEntities.begin(); InstancedEntityVec::const_iterator end = mInstancedEntities.end(); do { bVisible = (*itor)->findVisible( mCurrentCamera ); } while(!bVisible && ++itor != end ); } mVisible = bVisible; }
We're not in 2000 anymore. An IF statement (or conditional evaluation to be exact) is not a performance improvement. It has to be actually tested to see that.
http://assemblyrequired.crashworks.org/tag/intrinsics/
http://research.scee.net/files/presenta ... CAP_09.pdf (see slide 27)
This applies for PCs, handhelds and consoles; but specially for the two latter ones.
Since processor speed doubles every 1,5 years, but memory bandwidth & latency doesn't; the conditional evaluation as a performance improvement is less & less an "improvement". Furthermore Ogre is expanding to those areas other than PC, so it's a no go.
"if" statements can increase performance in those areas where massive ammount of code will be skipped.
I kept getting a corrupted file until I signed in; then I was able to download your patch. I'll be evaluating it this weekend.m2codeGEN wrote: But all above described are trifles It is much more important, that you have paid the attention to this patch since I could forget casually something
http://sourceforge.net/tracker/?func=de ... tid=302997
Cheers
Dark Sylinc
-
- Halfling
- Posts: 52
- Joined: Tue Apr 26, 2011 9:13 am
- Location: Russia, Tver
- x 2
Re: New InstanceManager: Instancing done the right way
Thanks, I read this document.dark_sylinc wrote: http://research.scee.net/files/presenta ... CAP_09.pdf (see slide 27)
In our case mInstancedEntities.size() == 2500 for example. I don't think that is necessary to execute all 2500 iterations.
Moreover 95 % of the Ogre code are written in the bad style if to follow article instructions. I am poor assembly programmer and I trust the compiler
-
- OGRE Team Member
- Posts: 5433
- Joined: Sat Jul 21, 2007 4:55 pm
- Location: Buenos Aires, Argentina
- x 1341
Re: New InstanceManager: Instancing done the right way
I've reviewed your patch (haven't got the time to apply it yet). My conclusions:
I like your changes, I really do. I like that:
As for the local transformations, which is the main point of your patch:
I like it too; BUT what I don't like is where the code is used. I'm not comfortable with the idea that a perfectly working scene breaks when recompiling with OGRE_DOUBLE_PRECISION. It's not what a common user would expect; it's just not intuitive.
Furthermore this means that you can no longer use the same shaders (with little modifications through macros) for Ogre Entity & InstancedEntity
Luckily my brain "clicked" when seeing your code and saw a solution. Like I said in a previous post; this problem is also common to regular Entity(ies) using HW skeletal animation. My proposal is:
I'm not sure I have the time to implement this by myself; would you like to do it instead? Eventually the precision in Entity problem is going to kick you just like InstancedEntity did unless you always stick to SW animation.
You have to understand I need to ensure the system is usable for everyone and covers everyone's needs, not just a particular situation.
I'll be uploading the overflow and bool to int fixed around these days.
Thanks
Dark Sylinc
I like your changes, I really do. I like that:
- You fixed the bool to int warnings. They were annoying
- You fixed buffer overflows when compiled with double precision.
As for the local transformations, which is the main point of your patch:
I like it too; BUT what I don't like is where the code is used. I'm not comfortable with the idea that a perfectly working scene breaks when recompiling with OGRE_DOUBLE_PRECISION. It's not what a common user would expect; it's just not intuitive.
Furthermore this means that you can no longer use the same shaders (with little modifications through macros) for Ogre Entity & InstancedEntity
Luckily my brain "clicked" when seeing your code and saw a solution. Like I said in a previous post; this problem is also common to regular Entity(ies) using HW skeletal animation. My proposal is:
- For InstancedEntity, add a new option to tell whether the user wants local or full transforms
- For regular Entities, add a new material variable called world_matrix_local_array_3x4 (and add a new function MovableObject::getWorldLocalTransforms() ) that returns the local bone transformations; and modify world_matrix, worldview_matrix & worldviewproj_matrix so that they always return the world matrix of the scene node (IIRC, the problem currently is that they'll return the first matrix of the array which is one of the bones'). Note that Ogre::Entity already contains the local transforms in their memory so it's not necessary to "undo" the world transformation like you do with Instancing
I'm not sure I have the time to implement this by myself; would you like to do it instead? Eventually the precision in Entity problem is going to kick you just like InstancedEntity did unless you always stick to SW animation.
You have to understand I need to ensure the system is usable for everyone and covers everyone's needs, not just a particular situation.
I'll be uploading the overflow and bool to int fixed around these days.
Thanks
Dark Sylinc
-
- Halfling
- Posts: 52
- Joined: Tue Apr 26, 2011 9:13 am
- Location: Russia, Tver
- x 2
Re: New InstanceManager: Instancing done the right way
It is good that we have found things in common
Changeover of variable "viewProjMatrix" by a variable "worldViewProjMatrix" would not lead to performance penalty (We continue to multiply only one matrix). In a case if Real is float type the world matrix will be identity.
By the way, I not so like the decision to transfer transformation of instancies in attributes of a vertex shader as a matrix 4х3. Yes it saves uniform variables for ShaderBased, but on the other hand we reserve 3 textural units at usage HardwareBasic technics. Can be costs then to the successor of class InstanceBatch to define in what format to write down transformations in a vertex shader?
I could deal with this problem, but most likely not earlier than May. Till May at all of us forces are thrown on the internal project.
Regards,
Vadim
In it you are certainly right, but this first decision, which has come to to me mind not breaking Ogre APIdark_sylinc wrote: As for the local transformations ...
I'm not comfortable with the idea that a perfectly working scene breaks when recompiling with OGRE_DOUBLE_PRECISION. It's not what a common user would expect; it's just not intuitive.
Furthermore this means that you can no longer use the same shaders (with little modifications through macros) for Ogre Entity & InstancedEntity
Code: Select all
void InstanceBatch::getWorldTransforms( Matrix4* xform ) const
{
#ifdef OGRE_DOUBLE_PRECISION
*xform = _getParentNodeFullTransform();
#else
*xform = Matrix4::IDENTITY;
#endif
}
I have understood idea, but these are already essential changes in API.dark_sylinc wrote: My proposal is:This way, handling of Entities & InstancedEntities can be done consistently; and users may want to use light & camera bindings in object space for their lighting calculations. This change will go to the default branch, not 1.8
- For InstancedEntity, add a new option to tell whether the user wants local or full transforms
- For regular Entities, add a new material variable called world_matrix_local_array_3x4 (and add a new function MovableObject::getWorldLocalTransforms() ) that returns the local bone transformations; and modify world_matrix, worldview_matrix & worldviewproj_matrix so that they always return the world matrix of the scene node (IIRC, the problem currently is that they'll return the first matrix of the array which is one of the bones'). Note that Ogre::Entity already contains the local transforms in their memory so it's not necessary to "undo" the world transformation like you do with Instancing
By the way, I not so like the decision to transfer transformation of instancies in attributes of a vertex shader as a matrix 4х3. Yes it saves uniform variables for ShaderBased, but on the other hand we reserve 3 textural units at usage HardwareBasic technics. Can be costs then to the successor of class InstanceBatch to define in what format to write down transformations in a vertex shader?
In our company I am the team leader of programmers engaged in three-dimensional visualisation, therefore time for the free creativity at me also end-to-end.dark_sylinc wrote: I'm not sure I have the time to implement this by myself; would you like to do it instead? Eventually the precision in Entity problem is going to kick you just like InstancedEntity did unless you always stick to SW animation.
I could deal with this problem, but most likely not earlier than May. Till May at all of us forces are thrown on the internal project.
Regards,
Vadim
-
- OGRE Team Member
- Posts: 5433
- Joined: Sat Jul 21, 2007 4:55 pm
- Location: Buenos Aires, Argentina
- x 1341
Re: New InstanceManager: Instancing done the right way
Don't know why you bring this up, but I'm not concerned about performance in this case.m2codeGEN wrote: Changeover of variable "viewProjMatrix" by a variable "worldViewProjMatrix" would not lead to performance penalty (We continue to multiply only one matrix). In a case if Real is float type the world matrix will be identity.
If it's because I said "Note that Ogre::Entity already contains the local transforms in their memory so it's not necessary to "undo" the world transformation like you do with Instancing" I was talking about Ogre::Entity.
In instancing you're doing:
Code: Select all
+#ifdef OGRE_DOUBLE_PRECISION
+
+ Vector3 posSceneNode = (mFullBoundingBox.getMaximum() + mFullBoundingBox.getMinimum()) * 0.5;
+ mFullBoundingBox.setMaximum(mFullBoundingBox.getMaximum() - posSceneNode);
+ mFullBoundingBox.setMinimum(mFullBoundingBox.getMinimum() - posSceneNode);
+
+ getParentSceneNode()->setPosition(posSceneNode);
+
+#endif
May be I didn't explain myself correctly. For instancing; the local transform would be used exactly as in your patched code (except the developer could opt to use your behaviour or the old one; rather than hardcoding through macros)m2codeGEN wrote: By the way, I not so like the decision to transfer transformation of instancies in attributes of a vertex shader as a matrix 4х3. Yes it saves uniform variables for ShaderBased, but on the other hand we reserve 3 textural units at usage HardwareBasic technics. Can be costs then to the successor of class InstanceBatch to define in what format to write down transformations in a vertex shader?
As for normal Entities (no instancing); the worldMatrix array would be binded with world_matrix_local_array_3x4 instead of the usual world_matrix_array_3x4; and then they should multiply against worldViewProj instead of the usual viewProj matrix.
"world_matrix_local_array_3x4" should not be used for instancing (i.e. raise an exception).
Similar schedule here. I'll check in the patch for the buffer overrun & the compiler warnings; but the local transform code will have to wait until one of us is ready to polish it. I hope you can still run your own modified code meanwhile.m2codeGEN wrote: In our company I am the team leader of programmers engaged in three-dimensional visualisation, therefore time for the free creativity at me also end-to-end.
I could deal with this problem, but most likely not earlier than May. Till May at all of us forces are thrown on the internal project.
Cheers
Dark Sylinc
-
- Halfling
- Posts: 52
- Joined: Tue Apr 26, 2011 9:13 am
- Location: Russia, Tver
- x 2
Re: New InstanceManager: Instancing done the right way
I think it it will be interesting. Early results experiments of large forests rendering using New InstanceManager
[youtube]UiWeXwSAdWU[/youtube]
[youtube]UiWeXwSAdWU[/youtube]
-
- Gremlin
- Posts: 193
- Joined: Fri Jan 25, 2008 6:55 pm
- Location: Nantes / France
- x 17
Re: New InstanceManager: Instancing done the right way
Hi everyone,
I'm currently writing a paged entity system (pretty like PagedGeometry plugin, but using Ogre's internal paging system), and was planning to build static geometry for near pages, but I take a look to this Instance manager, and it seems it feet well to my needs, but I have a little problem.
In my paged entity system, I made a class "EntityPagedWorldSection" which inherit from "PagedWorldSection", and takes a "Entity" as parameter (pretty much like pagedTerrain, but with an entity instead of TerrainGroup). I next have 2 PageContent inherited classes, EntityPageContentImpostor for far tiles, and EntityPageContentBatched for near tiles. Obviously, i want to use instanced entities for the near tiles.
But as it's written in the documentation:
My problem is ideally, I should set up one batch per "EntityPageContentBatched" instance, but batches creation process seems to be quite "rigid" in InstanceManager": a newly batch is created when the first one is "full of entities". I can't create an instanced entity in a specified batch neither.
The other solution is to create an instanceManager per "EntityPageContentBatched" instance, but as far as i understand how it works, it shall not be very efficient.
Does anybody saw another method?
<edit>My real question is, does modifying the createInstancedGeometry function this way means anything? If no targetInstanceBatch is provided, then the actual behavior stay the same, but It's possible to force the batch on which the instanced entity will be added. What do you think of that?</edit>
I'm currently writing a paged entity system (pretty like PagedGeometry plugin, but using Ogre's internal paging system), and was planning to build static geometry for near pages, but I take a look to this Instance manager, and it seems it feet well to my needs, but I have a little problem.
In my paged entity system, I made a class "EntityPagedWorldSection" which inherit from "PagedWorldSection", and takes a "Entity" as parameter (pretty much like pagedTerrain, but with an entity instead of TerrainGroup). I next have 2 PageContent inherited classes, EntityPageContentImpostor for far tiles, and EntityPageContentBatched for near tiles. Obviously, i want to use instanced entities for the near tiles.
But as it's written in the documentation:
So, my plan was to set up the InstanceManager for a mesh when creating the "EntityPagedWorldSection" object, and then call "createInstancedEntity" function in the prepare process of each "EntityPageContentBatched" instance.It's wasteless to create two InstanceManagers with the same mesh reference, instancing technique and instances per batch count.
My problem is ideally, I should set up one batch per "EntityPageContentBatched" instance, but batches creation process seems to be quite "rigid" in InstanceManager": a newly batch is created when the first one is "full of entities". I can't create an instanced entity in a specified batch neither.
The other solution is to create an instanceManager per "EntityPageContentBatched" instance, but as far as i understand how it works, it shall not be very efficient.
Does anybody saw another method?
<edit>My real question is, does modifying the createInstancedGeometry function this way
Code: Select all
InstancedEntity* InstanceManager::createInstancedEntity( const String &materialName, InstanceBatch* targetInstanceBatch=0)
-
- Gremlin
- Posts: 193
- Joined: Fri Jan 25, 2008 6:55 pm
- Location: Nantes / France
- x 17
Re: New InstanceManager: Instancing done the right way
For information, finally, I just went the easy way, ie one InstanceManager per subentity on every page.
-
- OGRE Team Member
- Posts: 5433
- Joined: Sat Jul 21, 2007 4:55 pm
- Location: Buenos Aires, Argentina
- x 1341
Re: New InstanceManager: Instancing done the right way
I just saw your post. Sorry for the late answer.
Using multiple InstanceManagers will waste video RAM, to be exact. If memory or bandwidth isn't your bottleneck you'll probably won't see much difference.
Anyway, alternatives I can think of:
Dark Sylinc
Using multiple InstanceManagers will waste video RAM, to be exact. If memory or bandwidth isn't your bottleneck you'll probably won't see much difference.
Anyway, alternatives I can think of:
- Inherit InstanceManager and manage it yourself by allocating one yourself. This way you can have full control on how batches are created/destroyed. The only "gotcha" is that InstanceManager::_updateDirtyBatches needs to be called in each _renderScene() call. To overcome that, we could add something like SceneManager::registerCustomInstanceManager if you're interested.
- At loading time, call getMaxOrBestNumInstancesPerBatch and then InstanceManager::createInstancedEntity as many times as necessary (i.e. you need 5 batches? Call it numInstancePerBatch * 5 times) and then remove all of them. Later use getInstanceBatchIterator()->createInstancedEntity() to control on which batch the entity gets created.
- Same as above, but instead of calling createInstancedEntity so many times, we add a function "preallocateBatches( numBatches ). This is possibly the most flexible, efficient and cleanest solution.
Dark Sylinc
-
- Gremlin
- Posts: 193
- Joined: Fri Jan 25, 2008 6:55 pm
- Location: Nantes / France
- x 17
Re: New InstanceManager: Instancing done the right way
Ok, thanks, I will keep that idea in mind, but for the moment i'm wasting time to make paged component working, it's a lot of retro-engineering, as nothing is documented, and it seems that it contains some bugs :/
I will stay "the easy way" for the moment, while i'm debugging paged compo, but thanks for the hint, it'll help me when I'll just have to optimize it.
I will stay "the easy way" for the moment, while i'm debugging paged compo, but thanks for the hint, it'll help me when I'll just have to optimize it.
-
- OGRE Team Member
- Posts: 5433
- Joined: Sat Jul 21, 2007 4:55 pm
- Location: Buenos Aires, Argentina
- x 1341
Re: New InstanceManager: Instancing done the right way
@m2codeGEN:
I added your changes involving compiler warnings and double to float conversions. Thanks for the patches.
However, as I said I would do, the "local" transformation enhancement were left out.
I manually typed the code changes. Hg seems to hate me. It refuses to apply patches, don't know why.
I added your changes involving compiler warnings and double to float conversions. Thanks for the patches.
However, as I said I would do, the "local" transformation enhancement were left out.
I manually typed the code changes. Hg seems to hate me. It refuses to apply patches, don't know why.
-
- OGRE Team Member
- Posts: 4270
- Joined: Sun Feb 25, 2007 4:56 am
- Location: Bloomington, MN
- x 126
Re: New InstanceManager: Instancing done the right way
@dark_sylinc Would you be able to close the patches on the tracker too?
-
- OGRE Team Member
- Posts: 5433
- Joined: Sat Jul 21, 2007 4:55 pm
- Location: Buenos Aires, Argentina
- x 1341
Re: New InstanceManager: Instancing done the right way
Last time I checked, I'm not added in SF.Net so I don't have those access privileges.
Edit: My sf.net account name is 'darksylinc' (no spaces)
Edit: My sf.net account name is 'darksylinc' (no spaces)