Project idea:Instancing & crowds.

[Crashy]

Now full support of 99.999999% of the meshes

Except animated meshes. They don't work, but If I delete the reference to the .skeleton file in the .mesh.xml, and then recompile the .mesh, it works good.
If I keep the reference to the skeleton, I got an error with the vertex declaration.


I've also encountered an "error" with some meshes that have multiple submeshes using the same materials.
The fact is that the submeshes are listed as different queued geometries, and I cannot really know when to increment the index.
For the moment I increment the index each time I loop in the

Code: Select all

for (gi = mQueuedGeometry.begin(); gi != giend; ++gi)

Good for the majority of the meshes, but not in the case of multiple sumbeshes having the same ID.

[Crashy]

MMhh i've been working all the day to make the method that add a new batch instance.

I' m working on a design where 1 region = 1 batch instance.

But I've roblem when calling this method: nothing new appears. I think there is something worng with the renderable or the bounding box, but i don't know what exactly

Edit: Mainly resolved, but demo not ready at all. Woking on it for monday.

[Crashy]

New sources:
http://crashy.cartman.free.fr/SOC/samples.rar

New diagram:



So whats new: full support of 100% of the non animated objects...I hope

The bouding boxes are not correct, I've made no work on it with the new implementation.

This implementation is heavily based on the static geometry, I've kept some names but the future names are between brackets in the diagram.

Some functions of the static geometry are still present but not used, because they'll be usefull in the future.

For the moment only position of instances is customizable, I'm now working on more control(full transformation matrix)

And wiki page is updated:
http://www.ogre3d.org/wiki/index.php/SoC2006_Instancing

[Crashy]

http://crashy.cartman.free.fr/SOC/samples.rar

File updated, no problems with the bouding boxes anymore.

Now working on rotations/scale and individual access methods.

[syedhs]

Code: Select all

1>c:\code\3d\ogrenew\Samples\instancing\include\BaseBatch.h(31) : fatal error C1083: Cannot open include file: 'batchInstance.h': No such file or directory

[tuan kuranes]

it is not needed you can comment this include and include OgreSimpleRenderable.h instead.

[syedhs]

Okay thanks Tuan. Compiled and run flawlessly.

@Crashy,

Btw, I think there is still bounding error. I move a lil bit closer to the ninjas and then yaw to the right - the screen turned all black.

[tuan kuranes]

from its post above :

The bouding boxes are not correct, I've made no work on it with the new implementation.

[Crashy]

Strange, there should'nt be any problem with the bounding.
You should have the bouding boxes visible like this
http://crashy.cartman.free.fr/SOC/bbox.JPG

If not, redownload and compile the file, I could have made an error with the previous pack.

[syedhs]

But I have read this in his next posting

File updated, no problems with the bouding boxes anymore.

It is nothing, just to highlight the problem which may not be realized yet.

But actually it is my fault, I tested the 'second latest' code. Downloading the latest one.. yes it is working well along with bounding box shown in white color.

[Crashy]

@kuranes: How does it works for the mid term evaluation?

[tuan kuranes]

@Crashy: as planned if you did received my email.

[Crashy]

I've almost finished the part on static objects.
I must make an improvement on the bouding boxes(I've just seen a bug today), and correct a memory leak.

I've not located where the memory leak is, I'm working on it, but if you find something, say it to me please
Basically, if you create a batch, destroy it, and recreate a batch, you'll have a loss of fps, and the memory usage of the process increased by 1.4 mo.

The new demo &source&new shader is here
http://crashy.cartman.free.fr/SOC/instancing_demo.rar
I've just started to work on this demo, so there are many bugs and things not working. But with it I've found many problemes that i resolved, and other one that I'm trying to resolve
+/- to add a mesh to the batch. Notice that if the instancing is used, you'll have 50 objects max for 1 batch, and after that, if you want to have 51 objects, it will create a new batch instance, so you'll have 100 objects displayed, and not 51.

If you use static geometry, it will be fine.

Page down/up to change the model. Notice that it still crashes with meshes that have a skeleton, that will be my next improvement.

F1 to use instancing.
F2 to use static
F3 to use independant rendering(not in the demo yet).

G: destroy the batch
H: recreate the batch
If you've a crash when you exit the demo after having destroyed a batch, but not recreated one, it is normal.

So, if you play with G/H, you'll see the fps decrease, and the memory usage increase.
I've made several debugs, but I didn't saw where the leak is.


And the updated uml diagram

[Crashy]

Updated sources& demo. Most of bugs are corrected. I've removed all the lighting stuff in the instancing shader, to see the real performance.
I need to add some little tricks for to make it works with athene-like meshes, but apart from that and meshes with skeletons, everything works fine.

http://crashy.cartman.free.fr/SOC/samples.rar

Controls:


F1 to use instancing.
F2 to use static
F3 to use independant rendering

+/- of the numpad to add/remove a mesh to the batch. Notice that if the instancing is used, you'll have 50 objects max for 1 batch, and after that, if you want to have 51 objects, it will create a new batch instance, so you'll have 100 objects displayed, and not 51.

If you use static geometry or independant objects, it will be fine

*/ / of the numpad to add/remove 50 objects.

Page down/up to change the model. Notice that it still crashes with meshes that have a skeleton.


Some performance informations:

With 50 ninjas, static are better than independant objects and instancing.
With more than 400 ninjas, the instancing is better than instancing, but never more than ~10fps with all the object visible
If you are in the center of the batches, instancing will be really better!
I think it is due to the shader stuff.
But notice also that if only a little part of the objects are visible, independant rendering is better.

I'm a little bit disappointed of the little difference between instancing and static. I think that with bigger batches (100/200objects), instancing would be far better in the demo cases, but I'm limited to 50 float4x4 matrices in the shader.

If someone has a shader 3 compatible card here, is it possible to test with batches of more than 50?(like 200).

[klauss]

Crashy... you forget the real difference between static and instancing.
Where instancing can help, static not always can. They're different things, so if they're similar in performance, it's a huge thing, because it means the speedups of static can be used in many more situations through instancing.

Do me this favor - apply a looping matrix animation to each instance. Like making it move forward and backwads in an oscillating manner, at different frequencies and amplitudes.

Imagine doing that with static.
Do that on the demo - it will demonstrate the power of instancing.

[tuan kuranes]

Nice, the demo is indeed very handy to test multiple configuration.

Profiling is the key to performance :
Time to verify the reasons of performance loss/gain using nvperfkit if having a recent nvidia card or Ati PIX plugin.

Perhaps :

- Batch instance could have a vertex shader only (no pixel shader at all), a vertex shader + pixel shader unlit, and a vertex shader + pixel shader material option, so that comparison can be done. All that runtime changeable using a batchinstance method.
- A smaller vertex shader (vs1) could be integrated too for older card, and why not runtime settable on batch instance (for test too).

[Crashy]

Where instancing can help, static not always can. They're different things, so if they're similar in performance, it's a huge thing, because it means the speedups of static can be used in many more situations through instancing.

Yeah yeah, I know, but it was just a very very little disappointment

Do me this favor - apply a looping matrix animation to each instance. Like making it move forward and backwads in an oscillating manner, at different frequencies and amplitudes.
.

Mhhh yes if you want, I'll add this as an option in the demo.

- Batch instance could have a vertex shader only (no pixel shader at all), a vertex shader + pixel shader unlit, and a vertex shader + pixel shader material option, so that comparison can be done. All that runtime changeable using a batchinstance method.

Mhh, excuse my ignorace, I didn't know that only a vertex shader can be used I thougt that there must always be a fragment program when a vertex program is used.

- A smaller vertex shader (vs1) could be integrated too for older card, and why not runtime settable on batch instance (for test too).

mhh you mean different shader used on each batch instance?

[tuan kuranes]

Mhh, excuse my ignorace, I didn't know that only a vertex shader can be used

first video card had vertex shader only...

ou mean different shader used on each batch instance

Not really was more for all batch instance to test, but incidentally that leads to user being able to set a different shader used on each batch instance. (who knows perhaps in some case it can benefits of using a smaller shader version ?)

[Captain Nemo]

or Ati PIX plugin.

On the ATI Page I see only that the PIX plugin is "coming soon". (I have seen this already months ago, BTW).
Or is there some news about it that I have missed?

[tuan kuranes]

@Captain Nemo: sorry. I think they release it once but now they no more seems to. I have one in my ati sdk directoty (in \tools.) Check there.(Still Pix gives useful information even without plugin.)

@Crashy: I'm not sure VS3.0 increased Constant number. I think that's one the reason of Vertex Texture in VS3.0 specification. Anyway, VS3.0 could use hardware instancing.

And about speed, keep in mind that static geometry will not generate the same number of batch as it's not limited by Vertex Shader constants.
200 meshes instance of 30 tris each mesh, gives 4 draw call when using shader instanced at 50 mesh per batch, whereas sstatic geometry batching gives 1 draw call.

[Captain Nemo]

He, you are right, it's in the ATI SDK! It never occured to me to look there. Thanks!
Don't want to hijack this thread, so I am diving away now

[Crashy]

New sources& shaders:
http://crashy.cartman.free.fr/SOC/samples.rar

If you wanna change the shader type used with instancing, press F5-F6-F7

I've also changed the names of the classes:
BaseBatch->InstancedGeometry
Region->BatchInstance

[tuan kuranes]

Indeed nice and clean.

Comparison is certainly more fair now, and fps difference is indeed better here when per-vertex shader lighting is disabled.

(You should add ambient and diffuse on material without pixel shader, no ?)

[Crashy]

(

You should add ambient and diffuse on material without pixel shader, no ?)

Mmhh, according to the manual, as long as a shader is set in the material, the lighting is disabled.

This attribute has no effect if a asm, CG, or HLSL shader program is used. With GLSL, the shader can read the OpenGL material state.

[tuan kuranes]

I was indeed wrong. At first load, all ninjas are black here, switching to any other shader type and back does make it lighted. At first, I thought it was a lighting problem...