Project idea:Instancing & crowds.

[Crashy]

Code: Select all

Which suggests more or less that in DX9 you can try to have the
transformes in a texture. 

Possible uniquely with sm3, because with sm 2 you cannot have textures units in vertex shader.

[Crashy]

Yeeepeee, I've just found what slowed down the framerate:
The draw of skeleton.

I disabled this, and I won a lot of FPS, now the crowd using instancing is really faster than single entity.
For 50 Robots:
410fps for instancing
250 fps for single entity.
With skeleton showed, I get only 180 fps for the instancing

The corwd shader is not shader for the moment, but I don't think there will be a bit performance hit .


So, good news.

I think I'll made two separate demos, one for static objects, and another one with animated objects. But I'll see that with my mentor as soon as he's back from holydays

[Lee04]

Possible uniquely with sm3, because with sm 2 you cannot have textures units in vertex shader.

Yes nVidia SM3.0 not ATI

And the 6x00 cards from NVidia takes up to 20 clock cycles to sample the vertex texture as to the 7x00 cards I don't have a number but the vertex processor is said to be redone for the 7x00 cards.

cheers

Lee04

[Crashy]

Hi.

Here is the new demos and optimized shaders.

Some remarks :

-there is a big problem with object that have LOD. It simply crashes at launch in release mode, and it crashes when I delete the batches in debug. As usual, I'm on it, but if you see anything, thanks


-I separated the demo for animated objects. It is for the moment extremly basic, just the same as for static objects but with animations. I also deleted the staticgeometry mode in this demo, because it is a non sense.

-the vertex lighted shader for static objects give a bad result. This is really strange because I use the same lighting algorithm for animated objects, and it works fine.

-For the moment, for animated objects, there is only the vertex lighted shader. I think I'll add a flat lighted shader, as for static shaders.

-You can enable/disable shadows by pressing the M key.(it's easy Mmm key?)
With shadows activated, there is a big performance hit for the instancing, because it also need a vertex program for shadow casting. So in this case, the instancing is still the faster but less than without shadows.

Same thing with animated objects, without shadows, instancing is about 200% faster than single objects, but with shadows activated, the performance are about the same. Still with an advantage for the instancing, though.


Here is the code:
http://crashy.cartman.free.fr/SOC/samples.rar


just for fun a little screen with 1000 robots.
http://crashy.cartman.free.fr/SOC/crowd1000.JPG

next, a demo with CEGUI, to be more professional

[Crashy]

New demos, with full GUI. The demos are now in the same folder instancing_crowd, just pick the project you want to compile the demo you want

As you will see, the shadow of the robots on the terrain, in the crowd demo, are weird, is there a tip to have more precises shadows, or is it just a matter of light configurations, to have the light just at the good place.

Second little remark: the fog is not correct on instancied object. That's normal, but I just prevent

Finally: don't go too fast on the object count slide bar, or you'll have freezes when the count is high.


The next step is to write some docs on how to implement HW instancing. I've downloaded a lot of tutorials and demos, it should'nt be long

It also seems that there is a way to make HW Instancing avaible on X800 series, even if they don't support SM3. I've read that they support HW instancing and that a driver trick to make Direct X believe that they support HW Instancing is possible.

After that merge as much as possible the code with the static geometry.

link to the demo->
http://crashy.cartman.free.fr/SOC/samples.rar

I'll show the sample to a searcher of the Inria tomorrow.

EDIT: I just met the searcher, he said me that Sinbad will meet him at genoble in a few monthes. The world is small

[Crashy]

The last "not in the ogremain" demo releases.
I've made the integration into the ogremain dll on my side but I cannot access to the cvs repository for the moment :s

http://crashy.cartman.free.fr/SOC/samples.rar

And a little document to see how behaves the instancing

http://crashy.cartman.free.fr/SOC/Insta ... rmance.pdf

[sinbad]

For info, Crashy's code is now in CVS under the branch 'soc06-instancing'.

[Lee04]

Ogre instancing is cool! I love the fact that it's in Ogre now.

How about a bit control over the small fellows?

Here is a small example of behaviors moving massive amounts of agents all drawn with one instancing call..

Typically good for controlling instanced crowds sense the behaviors all run on the GPU.

Oh! Yeh! It's the TOP VIEW!


AVI film...

http://www.abcmedia.se/MassiveAgents.rar

These have also states if you need them for animation AI etc.

Codec http://www.camstudio.org/CamStudioCodec10.zip

We have also develop decision trees as shaders for massive crowds
running on the GPU as GPGPU.

Here we have 2000+ units... but they can hardly move anywhere it's too crowded. But they are still all moving when they can and updating fine.



This system also have a CPU version, but that can’t handle massive crowds just crowds.

We are also looking into another thing more news on this later when it's ready.

<edit>One student is close to finnsih the massive crowd animation system here is a peek: </edit>

And now a film with 2K crowds, be where there are some chunky results because Camtaiesa screen recorder is pulling images from the back buffer
to make this move.

(VARNING! BIG file)
http://www.abcmedia.se/ogreforum/massiv ... crowd.html

These projects are all student projects I lead on the side of my job, so these students didn’t know GPU architecture or shader programing when they started but they learn really fast. Usually they know basic OpenGL or DirectX when they start.

We are about to port the work to Ogre...
T
cheers

Lee04, Tai, Linus

[Assaf Raman]

We are talking about instancing for use in a forest demo here -
http://www.ogre3d.org/phpBB2/viewtopic. ... 527#170527

It might interest some of you - for the next stage - using the instancing code - if it is relevent - in the case of a forest.

[Crashy]

I'm really interested to see if using instancing is relevant for your code. I'm making an improved paper to present what I've done. Maybe, reading it, you could say if it is a goot idea or not(for the moment I didn't looked at your method, so I cannot help until a few days )

@kuranes: did you receive my mail about the static object-demo?

[tuan kuranes]

@Crash: yes, received, sorry, didn't saw there was 2 project, that's perfect. Maybe soon a binary demo so that you can get some fps on many cards ?

@Assaf Raman : consider impostoring for not so far forest, and instancing/static geometry at low lod with near forest, and only very near tree must be at full resolution.

[Crashy]

Maybe soon a binary demo so that you can get some fps on many cards ?

You mean an independant demo release(as you do with the PSLM)? Why not

[Praetor]

Many cards? You mean high in number or high in variety. I am right in that instancing is SM3.0 right? That would seem to suggest there are relatively few cards that will support it....yet. Still large-scale testing is important. And, I wasn't nitpicking you Tuan, just saw an opportunity to clarify something.

[Crashy]

I am right in that instancing is SM3.0 right? T

You're wrong . I've implemented Shader Instancing, which is slightly different from Harware instancing.
Only Hardware instancing needs SM 3.0, and is limited to DX, afaik;
The shader instancing can work even on vs 1.1 card, even if it is not so interesting/

[Lee04]

Only Hardware instancing needs SM 3.0, and is limited to DX, afaik;

There is a driver and extension for hardware instancing in OpenGL.

http://developer.nvidia.com/object/open ... -2006.html

Thnx

Lee04

[Lee04]

Hi Crashy

I downloaded your last demo projects samples but the demo stops when an overlay is null

Code: Select all

CEGUI::utf8*)"InstancingDemo.layout");
		mGUISystem->setGUISheet(mEditorGuiSheet);
		//just show the tip overlay 
		Overlay*tipOverlay = Ogre::OverlayManager::getSingleton().getByName("Example/InstancingOverlay");

Do you have these files somewhere still?

cheers

Lee04

[sinbad]

That file is in the CVS branch 'soc06-instancing' under Samples\Media\gui. The overlay is also there in the overlays folder. I have the merge of the SoC work into the main CVS HEAD on my imminent TODO.

[Ajare]

In the demos, the entities seem to be darker when using instancing. With static geometry/separate objects they look fine. This is on a 6800 using the DirectX rendersystem.

edit: is this what you mean by 'the fog is not correct'?

[Crashy]

In the demos, the entities seem to be darker when using instancing. With static geometry/separate objects they look fine. This is on a 6800 using the DirectX rendersystem.

With nVidia cards and no shading, the lighting is black.
Check the "vertex lighting" checkbox to have a (bugged) lighting.

@lee04 :

this archive has everything.
http://crashy.cartman.free.fr/SOC/Instancing_Demos.rar

[Ajare]

With nVidia cards and no shading, the lighting is black.
Check the "vertex lighting" checkbox to have a (bugged) lighting.

I meant with the lighting on. It's definitely slightly darker.

[Crashy]

Ok, I'll check it at work. But yes, as I said, the lighting shader is a bit weird, I'm going to resolve this problem.

[syedhs]

Yes it is a bit darker. I think static geometry is about 2x brighter.

[Crashy]

Hello.

I've a few time this week to correct the few things that are necessary before the integration into OGRE.


I have two design proposition for the material generation at the batch creation.

the first:

If The user create a batch without any other arguments than the object name, the program decide that the default material(s) already have all the link to an instancing shader.

If the user gives a String as an additional argument, the String is the name of the instancing vertex program. Then the code create new materials for the object. But the program cannot know what are the names and the semantic of the vertex program parameters. So the user needs afterwards to configure that.


------------------------------------------------
the second:

In this one, the code doesn't deal at all with material /shader configuration. The user has to parse all the batches after their creation and to configure manually everything.
The only thing that the code should do is to use copies (if they don't already exist) of the original materials.



What sounds the best?

note: as many changes have been changed into OGRE recently, the demo needs a lot of work to work on the HEAD version(change the inputs, rewrite the CEGUI layouts, etc).

[sinbad]

I think the user should choose which material to use and should be responsible for making sure it contains everything needed to perform instancing. They should be able to set it on the batch, and that shouldn't be tied together with the creation of the batch, because they may want to change the materials on the fly later. I don't see why a copy of the material needs to be made.

I think we come from different perspectives here - I think you're trying to make it so that the user doesn't have to worry about the materials so much, and trying to tack on the instancing shaders for them (hence the need for a material copy). I think this is commendable, but ultimately when people start using features like this they're going to need to specify their vertex shaders themselves. This is just like skeletal animation, the difference being that we can't give them a software fallback. We can provide them with examples for common vertex shader setups. but if they want something different they should write it themselves following the same guidelines. I don't think we should try to 'merge in' instancing support into their materials, personally, any more than we try to merge in hardware skeletal animation.

So I think just deciding on the link between the code and the shaders that people write (the constants needed, bindings for them) is enough to make this work. If you wanted we could introduce another flag on vertex programs to indicate whether instancing is supported, like includes_skeletal_animation does.

[Crashy]

I've almost finished. I did it quite simply at last:
before adding the entity to the batch, in the sample code(and not in the instancedgeometry file in Ogremain), I just read it , create new materials, and add instancing vp to it, and apply the new materials on the entity.

So that's to the user to setup the materials.

If it's ok for you, it will be avaible in the week