Project idea:Instancing & crowds.

Threads related to Google Summer of Code
Post Reply
Crashy
Google Summer of Code Student
Google Summer of Code Student
Posts: 1005
Joined: Wed Jan 08, 2003 9:15 pm
Location: Lyon, France
x 49
Contact:

Project idea:Instancing & crowds.

Post by Crashy »

Summer Of Code Proposal
Jean-Baptiste Griffo
crashy(at)antharia.org

Proposal : Instancing, Crowd Rendering.

The main topic of the project is to have good performance even if the scene shows a huge number of objects.

Addition to Ogre would be :
- Instancing of static renderables using shaders.
- Instancing of animated renderables using shaders.
- Instanced renderable control interface application side.
- "Showcase(s) demo(s)", with comparative techniques(as the shadow demos).
- Guidelines to batching in ogre application developpement using your GeometryBatcher.

About myself(Sinbad said "Sell yourself", so I sel myself :lol: ) :

-I use ogre for 3 years(I began with 0.11 If I remember well), so I'm not the last of the noobs :D

-A little professional game developpement experience, doing at the moment a racing simulation prototype using OGRE&ODE(yes, OGRE!!) for a pro game studio.
Last edited by Crashy on Sat Sep 30, 2006 5:27 pm, edited 2 times in total.
Follow la Moustache on Twitter or on Facebook
Image
User avatar
tuan kuranes
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 2653
Joined: Wed Sep 24, 2003 8:07 am
Location: Haute Garonne, France
x 4
Contact:

Post by tuan kuranes »

Instancing, Crowd Rendering=> Static, Instanced and animated Geometry batcher

depending to ... The final user only has to add the object to the scene, the engine will do the remaining.[/
Depending on heuristic to determine, using benchmark when varying different renderable parameters (materials, poly count, unique objects, static or moving, etc...) and different hardware (VS support and which version) => very complex decision situation there...

And that's perhaps trying to be smarter than user and therefore making choice against its will. More control user has, less complex code in Ogre.
Perhaps User would/shoud be the one to make the decision...
User choose based on its scene situation (static, instanced, animated)and application minimum requirements.
In static geometry, which is a good balance on that Idea

Randomness => at least find a way to let user add that.

About "skinned object" which is skeletal or vertex animated entities, only user control needed is Animation control, no ?
(position control is already need on non-animated renderables.)

You forgot to sell yourself on Antharia (best ogre forum showcase thread traffic !)
Crashy
Google Summer of Code Student
Google Summer of Code Student
Posts: 1005
Joined: Wed Jan 08, 2003 9:15 pm
Location: Lyon, France
x 49
Contact:

Post by Crashy »

About "skinned object" which is skeletal or vertex animated entities, only user control needed is Animation control, no ?
(position control is already need on non-animated renderables.)
Yes of course. In fact I had in mind that if the user wants to attach an AI to the instancied character, it must be possible.
You forgot to sell yourself on Antharia (best ogre forum showcase thread traffic !)
Yes, I wanted to add that, but I don't think the "demo" I released was in my favour :lol:


And that's perhaps trying to be smarter than user and therefore making choice against its will. More control user has, less complex code in Ogre.
I think that the user only has to choose between unique and an array of object when he add it to the scene, and static/moving.

After that the engine should choose between instancing or static geometry.
If it is "too much" or not necessary, no problems to remove it from features :lol:
Follow la Moustache on Twitter or on Facebook
Image
User avatar
Olex
Hobgoblin
Posts: 593
Joined: Fri Apr 08, 2005 6:08 pm
Location: WA, USA

Post by Olex »

I would like to agree that more user control is better.

If you do that, you put workload of figuring out what is faster/slower on the user.

If you don't, you have to do all that for yourself. And chances are you will be wrong. Think of how many cards are out there, they will give different results b/c of PC express versus AGP, memory and chip clock speed, etc., etc. This would be too much to assess. But even if not it's still takes a lot of time to gather all the data to compute this.

At least first step should be complete user choice. After you gather enough data from users, you can set up guidelines for them to use in different situations, then if you see that it holds up in 90% or so cases, you can consider adding an automatic option (that is on by default maybe), but still leaving user control in place anyway.

Also I would like to point out performance is not always the final product the user may desire. For example, a user may want to distribute the load, and better performance of instancing might interfere with another process and slow down the application as a whole even if instancing was fast.

In the end, this feature is definitely awesome to have, Crashy. :D
User avatar
Lee04
Minaton
Posts: 945
Joined: Mon Jul 05, 2004 4:06 pm
Location: Sweden
x 1

Several students welcome

Post by Lee04 »

This project needs more than one student.
I know I really already had two students and I am on my third doing instancing and massive crowd Ai, animation etc. for a commercial version.

Each student works over a two month period or more.
All the work so far has been in OpenGl/DirectX only none Ogre because
Ogre adds a learning threshold.

But the idea is to make it work on Ogre, so I am really interested to see this done on Ogre.

I guess features like vertex textures, ping / ponging and simulation GPGPU stuff that we already done in OpenGL need to be done for Ogre. Forcing pixel buffer to become vertex buffers etc.

Really interesting area.
Ph.D. student in game development
User avatar
tuan kuranes
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 2653
Joined: Wed Sep 24, 2003 8:07 am
Location: Haute Garonne, France
x 4
Contact:

Post by tuan kuranes »

you can set up guidelines for them to use in different situations
That could be part of the proposal.

@Lee04: Crowd rendering (3D) only, not crowd simulation (AI)
Last edited by tuan kuranes on Sun May 07, 2006 8:35 am, edited 1 time in total.
Crashy
Google Summer of Code Student
Google Summer of Code Student
Posts: 1005
Joined: Wed Jan 08, 2003 9:15 pm
Location: Lyon, France
x 49
Contact:

Post by Crashy »

So replace automatic choice by some heavy docs?
Follow la Moustache on Twitter or on Facebook
Image
User avatar
Olex
Hobgoblin
Posts: 593
Joined: Fri Apr 08, 2005 6:08 pm
Location: WA, USA

Post by Olex »

Crashy wrote:So replace automatic choice by some heavy docs?
Well, they don't have to be heavy, but should contain general ideas of the approach used, and when they usually work better and the statistics of them being used in different scenes.
User avatar
Lee04
Minaton
Posts: 945
Joined: Mon Jul 05, 2004 4:06 pm
Location: Sweden
x 1

Post by Lee04 »

Crowd rendering (3D) only, not crowd simulation (AI)
OK No crowd simulation...
Ph.D. student in game development
User avatar
tuan kuranes
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 2653
Joined: Wed Sep 24, 2003 8:07 am
Location: Haute Garonne, France
x 4
Contact:

Post by tuan kuranes »

Addition to Ogre would be :
- Instancing of static renderables using shaders.
- Instancing of animated renderables using shaders.
- Instanced renderable control interface application side.
- "Showcase(s) demo(s)" to your soc proposal ?
- Guidelines to batching in ogre application developpement using your GeometryBatcher.

I would say that all other addition would be sort of bonus-work/example implementation of part of "showcase(s) demo(s)" (modifying existing particle/billboard system to use instancing if available on user GPU, randomness of each instance, distance shading, etc...).
Last edited by tuan kuranes on Sun May 07, 2006 11:44 am, edited 1 time in total.
Crashy
Google Summer of Code Student
Google Summer of Code Student
Posts: 1005
Joined: Wed Jan 08, 2003 9:15 pm
Location: Lyon, France
x 49
Contact:

Post by Crashy »

Your description is finally better than mine, thanks ;)
Follow la Moustache on Twitter or on Facebook
Image
Vectrex
Ogre Magi
Posts: 1266
Joined: Tue Aug 12, 2003 1:53 am
Location: Melbourne, Australia
x 1
Contact:

Post by Vectrex »

Lee04 wrote:
Crowd rendering (3D) only, not crowd simulation (AI)
OK No crowd simulation...
well actually... you could use Open Steer to get an easy crowd simulation without too much trouble :)
- http://opensteer.sourceforge.net/
- A partial Ogre implementation of OpenSteer
http://www.ogre3d.org/phpBB2/viewtopic.php?t=15640
- jacmoe has taken over the project so maybe this would be a great project to use while developing the Ogre version more
http://conglomerate.berlios.de/phpBB2/v ... d9b04201da
Crashy
Google Summer of Code Student
Google Summer of Code Student
Posts: 1005
Joined: Wed Jan 08, 2003 9:15 pm
Location: Lyon, France
x 49
Contact:

Post by Crashy »

Oups, I also forgot my name(it could be usefull no?).
Made an edit of the first post.
Follow la Moustache on Twitter or on Facebook
Image
User avatar
tuan kuranes
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 2653
Joined: Wed Sep 24, 2003 8:07 am
Location: Haute Garonne, France
x 4
Contact:

Post by tuan kuranes »

you should really sign up/post proposal now on google http://code.google.com/summerofcode.html
Crashy
Google Summer of Code Student
Google Summer of Code Student
Posts: 1005
Joined: Wed Jan 08, 2003 9:15 pm
Location: Lyon, France
x 49
Contact:

Post by Crashy »

Done. I've just seen that there was a little mistake in the idea description(bad copy/paste from the forum :x )

But apart from that, it should be right .
Follow la Moustache on Twitter or on Facebook
Image
User avatar
Lee04
Minaton
Posts: 945
Joined: Mon Jul 05, 2004 4:06 pm
Location: Sweden
x 1

Post by Lee04 »

well actually... you could use Open Steer to get an easy crowd simulation without too much trouble
I don't think so for 2000+ agents. thats why we do it on the GPU.

It's just to much work for the CPU.
Ph.D. student in game development
klauss
Hobgoblin
Posts: 559
Joined: Wed Oct 19, 2005 4:57 pm
Location: LS87, Buenos Aires, República Argentina.

Post by klauss »

Ok... just to document the original idea I had on its interface (which, IMO, would be the most useful interface) (and sorry if it's late): none.

Well... some rendersystem/manager/whatever controlling API would be great, like Something::setInstancingBatchLength() or Something::setInstancingEnable() or Something::setInstancingMethod()... but otherwise, everything could be automatic.

It's not that hard to make multiple instances of a same mesh (multiple entities) render at once with a specific, special-purpose algorithm, without requiring special handling by the application.

Each entity would, when populating the render queue, add an "instance entry" into a shared per-mesh pool, and when the queue gets executed, multiple such entries could be rendered at once if the batch isn't broken by material changes or render queue boundaries. It isn't hard at all... at least not in principle (perhaps implementation details get in the way - like who the hell is responsible for choosing the method? the RenderSystem? would make sense... but it's the scene manager the one populating and executing render queues... really messy in that aspect).

Furthermore, it uses this controlling API for hints, but it's ultimately Ogre the one deciding how to do things. So... if a driver reports no instancing capability (no shaders, no nothing), it can "emulate" instancing through batching (in some cases - ie: static geometry). It's a lot to code, but can be done progressively (the very basic functionality doesn't need that much "intelligent behavior"). Also, if a driver is deemed inappropriate for instancing (poor performance and/or bugs), it can be disabled from the rendersystem (since it's the rendersystem the one deciding), rather than have the application have to recognize those situations.

I think... design-wise... that's the best option generally speaking: making it an automatic but hintable thing.
Oíd mortales, el grito sagrado...
Hey! What is it with that that?
Wing Commander Universe
Crashy
Google Summer of Code Student
Google Summer of Code Student
Posts: 1005
Joined: Wed Jan 08, 2003 9:15 pm
Location: Lyon, France
x 49
Contact:

Post by Crashy »

Just some news: now it works with quite every non animated .mesh, although for the moment I'm only instancing the first submesh if the mesh has multple submesh.

But I've a little interrogation about the athene.mesh file.

Code: Select all

 <vertex>
                        <position x="-15.8713" y="7.52567" z="-20.2445" />
                        <normal x="-0.718988" y="-0.513476" z="-0.4684" />
                        <texcoord u="0.756245" v="0.630712" />
                        <texcoord u="0.9534" v="-0.0479685" w="-0.297871" />
            </vertex>
As you see, it has multiple texcoord set, why?(just a little interrogation :) ). Is it a frequent case?
Follow la Moustache on Twitter or on Facebook
Image
User avatar
BlasterN
Gnome
Posts: 378
Joined: Thu Mar 24, 2005 1:07 am
Location: Spain
Contact:

Post by BlasterN »

Is it a frequent case?
not much, but in nomal mapped meshes if you did it in differents programs maybe you will have multiple coord_set.
other cases are high texture for faces for example, and low res for body.



Are you planing to include optimizartion for animeted objects?
I recomend E3 video game: Ninty nine nights (really impresive)
Works:
MapToMesh | Bengine B9 @ www.sourceforge.net/projects/maptoogremesh/
3DWorldStudio exporter@ www.blastern.info
Crashy
Google Summer of Code Student
Google Summer of Code Student
Posts: 1005
Joined: Wed Jan 08, 2003 9:15 pm
Location: Lyon, France
x 49
Contact:

Post by Crashy »

not much, but in nomal mapped meshes if you did it in differents programs maybe you will have multiple coord_set.
other cases are high texture for faces for example, and low res for body.
Okay, so there will be a support for that in the implementation :)

There will be instancing for animated mesh, using hardware skinning for animation, and some methods to have for each character some different details.
Follow la Moustache on Twitter or on Facebook
Image
User avatar
BlasterN
Gnome
Posts: 378
Joined: Thu Mar 24, 2005 1:07 am
Location: Spain
Contact:

Post by BlasterN »

Good news. I allways want to do a massive 3rd person like 99nights. But right now i focus myself on a little fps with 4-5hours gameplay.
This is the video I told you.
Ninety-Nine Nights E3 2006 Trailer (HD Version)
http://www.gamershell.com/download_13775.shtml
Works:
MapToMesh | Bengine B9 @ www.sourceforge.net/projects/maptoogremesh/
3DWorldStudio exporter@ www.blastern.info
Crashy
Google Summer of Code Student
Google Summer of Code Student
Posts: 1005
Joined: Wed Jan 08, 2003 9:15 pm
Location: Lyon, France
x 49
Contact:

Post by Crashy »

Thanks for the video, impressive!

A little test with 4*100 athenas(with more than 14 Athenas, I needed to swicth from 16bits to 32 bits index buffer, because athene.mesh has 5338 vertices !! ).

The framerate is *correct* but it is really unplayable. I didn't tried with static geometry, but I think the result would just be awfull.

Image
Follow la Moustache on Twitter or on Facebook
Image
User avatar
Frenetic
Bugbear
Posts: 806
Joined: Fri Feb 03, 2006 7:08 am

Post by Frenetic »

Crashy: Holy crap!

What hardware is that running on?
User avatar
joshcryer
Gnome
Posts: 351
Joined: Wed Oct 13, 2004 8:22 am

Post by joshcryer »

Triangle count = 2.4 million, I'm suspecting some high end hardware. Well, at least, high end last year (which is when Crashy was showing shots of Antharia, which he said uses high end hardware)...
Image
User avatar
Frenetic
Bugbear
Posts: 806
Joined: Fri Feb 03, 2006 7:08 am

Post by Frenetic »

joshcryer wrote:I'm suspecting some high end hardware.
Gee, ya think? :lol:
Post Reply