[GSoC 2013 - accepted] Ogre 2.0

[Klaim]

Congrats!

Looks like there will be a LOT of merging work going on while this and resource manager separation will happen. Good thing Ogre uses a DSVC!!!

[dark_sylinc]

Great ideas and congratulations! I have a few comments:

Implement unique ID system to replace unique name system.

By that do you mean using integers instead of strings to identify and look for objects? That's gonna be a nice performance improvement indeed. While you're at it, I think some maps (such as the MovableObjectMap in SceneManager) could be changed to HashMap. That would also improve look up time a lot.

Yes, strings are going away, an integer replaces them (strings will be kept for usability, i.e. custom names, but they could be stripped away for Release ships, and don't have to be unique).
As for MovableObjectMap, I'll be switching it to a plain old array (an std::vector). It has it's trade offs, but will work very nice if you follow LIFO order (as does the rest of the memory model in ArrayMath).
Arrays are the best for iteration, and we'll use swap & pop method for O(1) insertion & removals; and lookups will take O(N), which are the least used operation (however a removal will probably involve a lookup depending on how we code it, I may be able to slip the index in the array into the MovableObject and update it when dirty). To amortize this, we'll do a reverse linear search, so if you respect LIFO order, it would takes much less than O(N) on average. And frankly I'm not concerned about lookup-then-removal performance when iteration is the giant elephant in the room, not to mention there's the chance of avoiding the lookup entirely when removing.

Write a ThreadManager abstraction layer with a Windows implementation and thread FrustumCulling and UpdateAllTransforms.

Instead of implementing platform-specific code here, I think OpenMP might be a nice cross-platform solution. It doesn't require a separate package such as Boost or TBB and, unlike C++11 threads, it's available on older compilers.

I find OpenMP very limited. And I'm not worried about platform specific code because we'll just need the basics for those functions FrustumCulling & UpdateAllTransforms (I just need to be able to launch threads and use sync barriers). That's basically one solution for Windows, and another for almost everything else using POSIX threads. It's something I can code in one night as I'm experienced with it.

And thanks for everyone for the support.

[holocronweaver]

I wouldn't leave 2.0 and/or 2.1 to students, unless they're really good, for example.

In a moment of uncertainty, you must have looked in the mirror and thought to yourself, "Yeah...I am just that good."

In all seriousness though, I am glad to be working alongside you this summer.

Could you clarify what changes, if any, you are planning to make to the RenderSystem class? I intend to work on the GL3+ render system in the 2.0 branch, but most of my core changes should be left to supporting the additional shaders (tessellation, geometry, compute).

[dark_sylinc]

I wouldn't leave 2.0 and/or 2.1 to students, unless they're really good, for example.

In a moment of uncertainty, you must have looked in the mirror and thought to yourself, "Yeah...I am just that good."

In all seriousness though, I am glad to be working alongside you this summer.

Could you clarify what changes, if any, you are planning to make to the RenderSystem class? I intend to work on the GL3+ render system in the 2.0 branch, but most of my core changes should be left to supporting the additional shaders (tessellation, geometry, compute).

Touché. Tbh refer to the part "Didn't you say Ogre 2.0 was too big for GSoC?". The more I looked into it, the more 2.0 looked convincing, and I wasn't seeing myself in GSoC, then I said... why not? it was like a mental barrier or prejudgment I had.
But yeah.. touché

We're in the bonding period, and we should iron this out. Technically speaking I won't be touching the RS (give or take whatever minor changes that break compilation) in this GSoC, and in the wiki it isn't planned until 2.3; but we should get this straight to avoid effort going to waste later.

First things first: I see BSP is marked as broken in your gsoc post. If it's the BSP demo as in "the demo", then the BSP manager won't work unless ported to 2.0; and I doubt someone will take the effort of doing it. BSP scene managers are really outdated.
If your intention is to fix it because "it has visual artifacts and any other situation could have them too", then I've got nothing to say

* The idea is that 2.3 should be shader-only, as long as we bundle something that removes the hassle of writting & managing shaders for trivial stuff (i.e. 2D guis, basic 3D materials). A fixed function 'emulator' (if that's the right term, i.e. RTSS) can be used for those who prefer avoiding shaders completely, and actually uses FF in architectures where shaders aren't supported, if there's any arch alive at the time with that requirement (i.e., right now it's the Wii)
So probably all ff related stuff could potentially end up being discarded or moved into another place.

* Whether shader based or FF, at a higher level we will be managing 'states' to minimize redundant API calls; and the RS will be given the new states and make the changes, so all API calls get together in one place (or at least few).
Right now we're full of "glSetXX( Y ) doSomething() glSetXX( NULL )" sometimes being necessary, sometimes not so much, and same with D3D.
Edit: To name a few, this includes scissors, alpha blending, cw. vs ccw. culling; global shader/FF settings like fog settings, ambient light colour, etc.
We'll also try to see if we can group by constant buffers (frequency of update, size, etc).

* Something that IS relevant are texture buffers & constant buffers support. DX10+ & GL4 will use texture buffers to pass some important data (morph targets, bone matrices, user-custom). D3D9 will use VTF, which is basically the same with texture coordinates.
But Murat took to my attention that texture reads from vertex shaders in GLES 2.0 are optional (and most mobile gpus don't support it); therefore we'll have to support good ol' constant registers as well

* HW gamma. We support it in GL; the only change I pretend is that we can have .texture files where textures are flagged as "convert to linear space" and "leave in it's own space"; because we're following the more restrictive criteria of GL & D3D10 (as oposed to d3d9) that textures with hw gamma have to be requested on creation (d3d9 allows toggling hw gamma on & off) and right now it's a race to see which material gets loaded first and the one who says "it's gamma" wins, overriding everyone else.
In other words, HW gamma correction should work and should never assume it's enabled unless explicitly told so (that last one probably doesn't work consistently across Ogre)

* Something I often talk lightly is the new Compositor planned for 2.1, and it's probably the one that will impact your work the most (well, it will impact everything).
In 2.1 it will be the most important component in the engine. A Compositor script (similar syntax to current one) will tell how the scene should be rendered. Ogre will not be able to render without an active compositor. Of course we'll provide a basic default one that just renders everything sequencially into the backbuffer.
For example let's suppose the user requested for 7 render queues (RQ for short) at startup, and provides this custom script:

1. render_scene from render queue 2 to rq 5
2. Now render_scene from rq 0 to rq 1 inclusive

Now this compositor is telling Ogre to first render RQs 2 to 5, THEN render 0 to 1 and not to render rq 6 (assuming there's not another compo node using it). It's not that crazy, it's possible and most importantly, very flexible.

First the Compositor will parse all compositor nodes and see that RQs 0 to 5 are being used. Had the user not provided explicitly 7 RQs, Ogre would've created 6 RQs automatically, since that's the amount needed. May be the user knows the 7th one will be needed later for his own stuff.
Second the Compo will see that only 6 RQs are in use; so it will request the High Level Culler entities from RQs 0 to 5. Now Ogre will do it's update functions (HighLevelCull -> UpdateAllAnimations -> UpdateAllTransforms -> LowLevelCull -> Compositor Manager) and in the end it gets to the Compo Manager again.
The Compo Mgr. will dispatch threads (when API allows threading, GL for example doesn't...) and update all render targets in the right order*.

*This sounds easy but is rather very hard, because at loading time the Compo Mgr. will have to compile the operations to process in order, and resolve and assert dependency issues: from simple valid dependencies, to invalid circular deps, to implicit deps (i.e. two render_scenes with shadow mapping enabled should use their own depth texture or can they share and reuse because it will be overwritten? Is the shadow texture from a pass being used as input to a compositor? Should Ogre render to shadow maps what's in the entire scene or what's in this render_scene alone or what's in all render_scenes from a compo node? etc; You'll see I haven't sorted out everything yet)

The rest of the flow is in the slides.

Now, what may be relevant for you is that the Compo Mgr. may be able of:

• Rendering a whole render_scene into stream out rather than a render target
• Encapsulate this stream out into a "Stream Out Container" and being able to use it in another pass, ie streamout_pass and output to a RenderTarget; so the RenderSystem takes the Stream Out Container and renders it.
• Have nodes used for GPGPU that take RenderTarges, Stream Out, Vertex Buffers or whatever is possible by the api as input and run a Compute shader on it (what will the output consist of, if any, I don't know yet, since gpgpu is too flexible. Probably GPGPU nodes will only enforce "execute me before node X" because node X uses something you know the GPGPU node touches)

IIRC D3D11 & GL3 allow using a geometry shader deciding which texture the GPU should output to (i.e. selecting across 6 cube faces). It doesn't always work very fast depending on the operation, but it might be handy to have.

Well, I left a few details, but that's pretty much all I have to say now about Render System changes and the Compositor Manager. May be you'll have more questions. To be honest, I feel we're far enough from 2.3 that I've been concentrated on the very low level details from 2.0 for this GSoC.
Probably the biggest one is that AFAIK we don't have any Stream Out encapsulation, and if we do, it may not fit the requirements that the Compo Mgr. will need (it isn't much though). Probably we'll just have something like this:

Code: Select all

class StreamOut
{
void setMaterial( MaterialPtr mat );
virtual void render( RenderTarget *target ) = 0;
/*...*/
};
class D3D11StreamOut : public StreamOut { /*...*/ };
class GL3PlusStreamOut : public StreamOut { /*...*/ };

Cheers
Matias

[holocronweaver]

...I wasn't seeing myself in GSoC, then I said... why not? it was like a mental barrier or prejudgment I had.

I had a similar experience, having to get over the preconception that GSoC was something only done by undergraduates with free summers (something I never had). Thankfully this summer will be the first lull in my entire college career, and I am grateful for it.

I see BSP is marked as broken in your gsoc post. If it's the BSP demo as in "the demo", then the BSP manager won't work unless ported to 2.0; and I doubt someone will take the effort of doing it. BSP scene managers are really outdated.
If your intention is to fix it because "it has visual artifacts and any other situation could have them too", then I've got nothing to say

I meant the latter, fixing common visual artifacts. I actually had not considered fixing the BSP scene manager itself, though you are likely right that no one would be willing to port such a mammoth. I will miss it though. CSG was my introduction to computer graphics and game programming. Doom, Duke Nukem and Unreal Tournament, anyone?

The idea is that 2.3 should be shader-only

David already designed OpenGL 3+ with core functionality only - no fixed function pipeline. So...done!

...as long as we bundle something that removes the hassle of writting & managing shaders for trivial stuff (i.e. 2D guis, basic 3D materials).

Are you referring to using RTSS in these cases, or do you have something else in mind?

Whether shader based or FF, at a higher level we will be managing 'states' to minimize redundant API calls; and the RS will be given the new states and make the changes, so all API calls get together in one place (or at least few).
Right now we're full of "glSetXX( Y ) doSomething() glSetXX( NULL )" sometimes being necessary, sometimes not so much, and same with D3D.

That is an excellent idea! I always found that pattern to be one of the more tedious parts of graphics API. Sounds like states should be relatively easy to implement.

Something that IS relevant are texture buffers & constant buffers support...texture reads from vertex shaders in GLES 2.0 are optional (and most mobile gpus don't support it); therefore we'll have to support good ol' constant registers as well

So close. Maybe GLES 4.0 will get us out of the woods.

HW gamma correction should work and should never assume it's enabled unless explicitly told so (that last one probably doesn't work consistently across Ogre)

I am glad this will be a priority because I plan to use it extensively in my personal projects.

The rest of the flow is in the slides.

I have read your slides about a dozen times. Thank you for making them. I learned allot! Especially about cache management.

Now, what may be relevant for you is that the Compo Mgr. may be able of:

• Rendering a whole render_scene into stream out rather than a render target
• Encapsulate this stream out into a "Stream Out Container" and being able to use it in another pass, ie streamout_pass and output to a RenderTarget; so the RenderSystem takes the Stream Out Container and renders it.
• Have nodes used for GPGPU that take RenderTarges, Stream Out, Vertex Buffers or whatever is possible by the api as input and run a Compute shader on it (what will the output consist of, if any, I don't know yet, since gpgpu is too flexible. Probably GPGPU nodes will only enforce "execute me before node X" because node X uses something you know the GPGPU node touches)

Handling render streams sounds easy enough from the render system viewpoint.
Interesting approach to handling compute shaders. I need to flesh out how compute shaders will be implemented a bit better. I will probably come back to bounce some ideas off you, if you don't mind.

IIRC D3D11 & GL3 allow using a geometry shader deciding which texture the GPU should output to (i.e. selecting across 6 cube faces). It doesn't always work very fast depending on the operation, but it might be handy to have.

You are correct about this being a feature of GL3. No doubt it could be handy. I am considering how I might use it for some of my samples.

To be honest, I feel we're far enough from 2.3 that I've been concentrated on the very low level details from 2.0 for this GSoC.

I had expected a reply mostly focused on Ogre 2.0, but am glad you decided to talk up to 2.3. You have given me many things to keep in mind while working on the GL3+ rendering system.

[Xavyiy]

Hi all!

I've been absent for the last month but finally I've quasi-finished my exams (still one more next week, but easy!), so I'm back to development very soon.

This project is crucial for the long-term future of Ogre, and personally I want to be an early-adopter in my engine & product, so I'll follow very close your progresses Matías

Related to your last post, some thoughts:

* The idea is that 2.3 should be shader-only

Agree, but I think it's a good idea to keep the FF code somewhere in the case in the future FF comes to be popular again (which is unlikely, but we never know). The problem is that if we want to be FF compatible we should keep the material related params which is in the Ogre core and not going to be used by default (assuming that 2.3 is going to be shader-only), so I don't know what's the best way to follow if finally we want to keep FF compatibility.

* HW gamma. We support it in GL; the only change I pretend is that we can have .texture files where textures are flagged as "convert to linear space" and "leave in it's own space"

Are you sure about this? I mean, maybe it's enough the way it works right now: you define if the texture unit must be gamma or linear (gamma corrected) and at loading time it's loaded being or not gamma correcte.

Problems with this approach is mainly inconsistency, since if multiple texture units refers to the same texture and some of them are flagged as gamma corrected while others not, the behaviour is more or less undefined (depending of the order the scripts are parsed).

So on this side the .texture files or similar looks like a good idea, but dunno if the "added complexity" (although not much, one more script format) is worth.

* Something I often talk lightly is the new Compositor planned for 2.1

That's is going to be a tedious work

But that's indeed very needed - the current way Ogre render scenes is far from being a good solution, SceneManager::_renderScene is not at all efficient at all (appart from being a huge headache...) in complex pipelines since it lacks info from the main compositor chain.

I like the RQ system you're propossing, and also you're considering even at this early stage the implication of shadow maps, let's see how this develops when the time comes!

Btw, the current render_pass first_render_queue and last_render_queue attributes are buggy somehow (at least with complex compositor chains), in my engine I've had to use viewport visibility flags instead to render group of objects since first_render_queue and last_render_queue didn't do their job. I had done some debugging but nothing obvious so I finally left it.

Btw, what do you think about the future of Ogre::RenderObjectListener::notifyRenderSingleObject ? I'm saying it because the core of my engine relies on being able to change the pass texture units and constants depending of the object (so with one single material you are able to render different objects using different textures and/or parameters), something similar to Ogre::Renderable::setCustomParameter but more high level.

Xavier

[Owen]

* HW gamma. We support it in GL; the only change I pretend is that we can have .texture files where textures are flagged as "convert to linear space" and "leave in it's own space"

Are you sure about this? I mean, maybe it's enough the way it works right now: you define if the texture unit must be gamma or linear (gamma corrected) and at loading time it's loaded being or not gamma correcte.

How about leaving things as they are now, but emitting a diagnostic whenever inconsistent gamma properties are detected for a texture?

[dark_sylinc]

* HW gamma. We support it in GL; the only change I pretend is that we can have .texture files where textures are flagged as "convert to linear space" and "leave in it's own space"

Are you sure about this? I mean, maybe it's enough the way it works right now: you define if the texture unit must be gamma or linear (gamma corrected) and at loading time it's loaded being or not gamma correcte.

How about leaving things as they are now, but emitting a diagnostic whenever inconsistent gamma properties are detected for a texture?

Well there's a gigantic problem with that. See, if I have 2 Materials using the same texture, I have to declare the texture unit as using hwGamma explicitly
(which means typing more and a very big chance of screwing it up).

Otherwise it's a race of who gets loaded first. But it gets worse. If it ended up using hw gamma and during run time I unload the texture, and in the next
frame the non-hwgamma material is used first, the texture will be loaded again but this time not using hw gamma. Same viceversa. Every time I reload the texture I'm crossing my fingers.

For Distant Souls I ended up using a naming system, where at startup it looks for all materials that contains a particular string (i.e. "MyGammaName", also if it matches 100% the word "Diffuse") and sets the gamma flag for all of them. It slowdowns loading time for a bit, but I'm sure of getting what I wanted. If the texture was already loaded (which is an annoying habit of ParticleUniverse) with non-gamma, it's unloaded.

So, .texture files remove the problem, and you don't have to define it if you don't want to. .texture files are also useful for other stuff, for example
enforcing a particular pixel format (i.e. PF_L8) without having to do it each time in the material.If you don't define the texture in a .texture file, it's loaded with defaults (or what the material texture unit definition says).You don't have to deal with the complexity of another .texture file if you don't want to, but if you're using an WYSIWYG editor (your case Xavi!) it's really useful, because you get to store custom texture parameters in a centralized file.

It's also a lot easier to create cubemaps, 3D textures and 2D array textures from separate files, because you can define a an arbitrary texture name i.e.
"MyCubeMap" and construct if from 6 different files or "MyTextureArray" and just use that texture as input to the material.

* The idea is that 2.3 should be shader-only

Agree, but I think it's a good idea to keep the FF code somewhere in the case in the future FF
comes to be popular again (which is unlikely, but we never know). The problem is that if we want to be FF compatible we should keep the material related
params which is in the Ogre core and not going to be used by default (assuming that 2.3 is going to be shader-only), so I don't know what's the best way to
follow if finally we want to keep FF compatibility.

Hehe, we think very much alike. I haven't fully thought of the material settings, I'll probably want to leave them as they're now too; another alternative is
to create custom params and assign them automatic bindings (sounds familiar?) like "diffuse auto_diffuse"; so FF based systems get to know where to grab their data from; and if it's not present just use a default.

Using FF again shouldn't be thaaat hard, the system takes the parameters it can from the the material (i.e. if we have diffuse, emissive, fog, and
CookTorranceRoughness, it can take the first 3) and create an FF state that can represent the material. It may not look the same, of course.

I like the RQ system you're propossing, and also you're considering even at this early stage the implication of shadow maps, let's see how this
develops when the time comes!

I remembered something about shadow map rendering, I ended up realizing the best solution to the problem is to let the user create a compositor node that says what to render into the shadow map (the main question is what visibility flags and what RQs to use, and the user can know the answer to those) and use that node for shadow maps.
He can create multiple nodes if he wants shadow maps with different objects rendered in it.
For simplicity multiple lights and CSM/PSSM ought to be handled by the same node (i.e. avoid one node per cascade). Multiple nodes is useful if the user wants two shadow maps rendered with different visibility flags settings, or RQs (or even shadow map technique?)

For example (pseudo code):

Code: Select all

myShadowNode
{
technique pssm
num_shadow_maps 6
num_splits 3
num_lights_with_splits 1
shadow_map 0 PF_FLOAT_16 light 0 split 0 //Render light 0 (assuming directional) in split 0
shadow_map 1 PF_FLOAT_16 light 0 split 1 //Render light 0 (assuming directional) in split 1
shadow_map 2 PF_FLOAT_16 light 0 split 2 //Render light 0 (assuming directional) in split 2
shadow_map 3 PF_FLOAT_16 light 1 //Render 2nd closest light in the 3rd shadow map (can be directional, point, spot)
shadow_map 4 PF_FLOAT_16 light 2 //Render 3rd closest light in the 4th shadow map
shadow_map 5 PF_FLOAT_16 light 3 //Render 4th closest light in the 5th shadow map

//Render shadow maps #0, 1, 2, 3, and 5
shadow_map 0 1 2 3 5
{
	render_scene
	{
		visibility_flags 0xff00ff00
		rq_first 0
		rq_last max //Special value implying maximum
	}
}

//Render missing shadow map #4 with different settings
shadow_map 4
{
	render_scene
	{
		visibility_flags 0x0000000f
		rq_first 2
		rq_last 3
	}
}
}

That is a very flexible & easy way of rendering shadow maps, IMHO. Of course there's validation: If a node has 2 lights with splits and 3 splits per light, then you need to use all 6 splits, if you define 6 shadow maps, you have to specify render_scene to all of them.
And with a bit of tweaking we can use MRTs and reflective shadow maps can be created, which are useful for Global Illumination.

Btw, the current render_pass first_render_queue and last_render_queue attributes are buggy somehow (at least with complex compositor chains), in my engine I've had to use viewport visibility flags instead to render group of objects since first_render_queue and last_render_queue didn't do their job. I had done
some debugging but nothing obvious so I finally left it.

Yeah, I had the same frustration. It had to do with the order of Compositor and what RQs they were using IIRC.

Btw, what do you think about the future of Ogre::RenderObjectListener::notifyRenderSingleObject ? I'm saying it because the core of my engine relies on being able to change the pass texture units and constants depending of the object (so with one single material you are able to render different objects using different textures and/or parameters), something similar to Ogre::Renderable::setCustomParameter but more high level.

I have no idea.
Using a non-const renderable is dangerous (thread safety) so it's probably going const. I take your point is to avoid creating so many materials when they all share everything but the texture?
That listener may affect how Ogre wants to batch everything. But honestly I have no idea yet.

[Xavyiy]

So, .texture files remove the problem, and you don't have to define it if you don't want to. .texture files are also useful for other stuff, for example
enforcing a particular pixel format (i.e. PF_L8) without having to do it each time in the material.If you don't define the texture in a .texture file, it's loaded with defaults (or what the material texture unit definition says).You don't have to deal with the complexity of another .texture file if you don't want to, but if you're using an WYSIWYG editor (your case Xavi!) it's really useful, because you get to store custom texture parameters in a centralized file.

It's also a lot easier to create cubemaps, 3D textures and 2D array textures from separate files, because you can define a an arbitrary texture name i.e.
"MyCubeMap" and construct if from 6 different files or "MyTextureArray" and just use that texture as input to the material.

That convinced me much more, pixel file format specification and cube/array textures definition. It could also work to define animated textures (ogre already has this feature, dunno how much it's used but in Hydrax, for example, I use it for the water caustics animation).

In my case, to deal with gamma corrected textures I've two type of textures: texture and raw_texture (I'm using a custom material manager, which at the end creates all the needed stuff using the ogre one, so it's not an ogre modification but an external manager). texture -> gamma corrected (diffuse / color maps), raw_texture -> non gamma corrected (normal maps, specular, etc).

As you, I do the same, realod the texture if needed. In my case it's more explicit since different scenes could have linear or gamma lighting (well, altough not exposed in the editor anyomre, it used to be this way when it was focussed to game development), so after switching an scene some textures had to be reloaded.

Hehe, we think very much alike. I haven't fully thought of the material settings, I'll probably want to leave them as they're now too; another alternative is
to create custom params and assign them automatic bindings (sounds familiar?)

I would leave them as they're now, it makes sense since you can have access to them in shaders via auto params (ambient color, blah blah) and in the case it's needed in the future, the FF system can relie on these generic params.

That is a very flexible & easy way of rendering shadow maps, IMHO. Of course there's validation: If a node has 2 lights with splits and 3 splits per light, then you need to use all 6 splits, if you define 6 shadow maps, you have to specify render_scene to all of them.
And with a bit of tweaking we can use MRTs and reflective shadow maps can be created, which are useful for Global Illumination.

Looks like a very good idea. Simple and the user has a total control on it. =)

Note: the current implementation LiSPSM is buggy too, sometimes some casting geometry is not taken account resulting in incorrect results, dunno if you've experimented it but I did with a "vertical scene" (big cliffs), so I'm using a custom cascaded shadow mapping impl instead.

Yeah, I had the same frustration. It had to do with the order of Compositor and what RQs they were using IIRC.

Glad to know I'm not the only affected. Yes, it had to do with the comp order and min/max RQs, anyway since this part will be redesigned it's not worth spending time fixing it for 1.9.

I have no idea.
Using a non-const renderable is dangerous (thread safety) so it's probably going const. I take your point is to avoid creating so many materials when they all share everything but the texture?
That listener may affect how Ogre wants to batch everything. But honestly I have no idea yet.

Well, the idea is avoid creating so many materials but I've taken the concept further: in my engine's materials, you have 3 type of "custom parameters": spatial_custom_param, node and scene. (appart from constant params and auto_params)

The spatial custom param will use the value of the custom param defined in this spatial (~Movable object, but spatial params can be specified per subspatial ~Renderable too) or if not defined, use the default one (specified in the material script). Custom params can be of any type: float, int, matrices, quat, vectors, textures, etc (all this done visually in the editor, nothing to code)

Node custom params are more or less the same, but for a group of spatials. For example, you've a group of 6 rocks which you want to use the same texture, and another 10 rocks with another texture, so you simply have to attach them in two different nodes, use a material which texture param is defined as node_custom_param and add a custom param of type texture to each node, assigning the desired texture. (If a parent node doesn't have this custom param then we look in the parent of parent, until reach the root node).

Scene custom params are the same, but at scene level (in my engine you have projects which can contain any number of scenes, related or not, just different worlds), so maybe in one scene you want the water-materials to look red instead of blue, so you just use a scene_custom_param

About the const-renderable, no problem, I'm doing some "dirty" things in order to update textures and shader params at the last time, like forcing the update via the RS (this listener is not invoked at queueing time, but at rendering time -> So it does not affect to batching). Anyway, these dirty hacks are a cost I'm willing to pay since this system allows me to do really good things in a really simple way So if this listener is avaiable in 2.3 I would be more than happy

Xavier

[smisra]

Well am a novice user for Ogra. What i can see is Ogre has some SSE libraries, shared pointer etc. So as it will move to 2.0, why not use the functionality of Boost library more so as to simplify the tasks.I am not sure regarding the amount of work, but just a suggesting.

[Transporter]

Congrats

Well am a novice user for Ogra. What i can see is Ogre has some SSE libraries, shared pointer etc. So as it will move to 2.0, why not use the functionality of Boost library more so as to simplify the tasks.I am not sure regarding the amount of work, but just a suggesting.

Search the forum, we had a lot of discussions how to get rid of dependencies.

[spacegaier]

Just for reference: Mathias posted about about some upcoming changes on his blog:

Ogre 2.0: O(1) insertion, removal and contiguous memory iteration

[Klaim]

Nice reading!

Just two quick notes:

1. could have the efficient remove function have been implemented using the remove-erase idiom? It looks like it's doing the same thing without the index manipulations;
2. suggestion: use non-const references as function attributes instead of pointers if it is not valid to provide nullptr;

About 2), I suppose you're used to tons of Ogre code using this convention of using pointers instead of references, certainly because most of the user code will manipulate pointers anyway.
But in my experience it generate user code that is harder to read than the equivalent using non-const references.
I suspect that a big pass in Ogre all interfaces to fix this could help identify dumb manipulations and clarify user code passing pointers objects to functions.
But it's not really an important change, so...

[dark_sylinc]

1. could have the efficient remove function have been implemented using the remove-erase idiom? It looks like it's doing the same thing without the index manipulations;

The whole point of efficientVectorRemove is to avoid using std::vector::erase. Erase() will preserve the order of the container when removing an element in the middle hence takes O(N); efficientVectorRemove takes O(1) but doesn't preserve such order.

2. suggestion: use non-const references as function attributes instead of pointers if it is not valid to provide nullptr;

std::vector<SceneNode&> is not legal, and writing "&" each time on the input parameter to compare against the vector's content (or push into it) makes it worse/harder to read.
I agree normally prefer references to pointers for many reasons, but here is not the case.

[Klaim]

The whole point of efficientVectorRemove is to avoid using std::vector::erase. Erase() will preserve the order of the container when removing an element in the middle hence takes O(N); efficientVectorRemove takes O(1) but doesn't preserve such order.

Ah yes I see.

std::vector<SceneNode&> is not legal,

Obviously I was not suggesting this

and writing "&" each time on the input parameter to compare against the vector's content (or push into it) makes it worse/harder to read.

I disagree [...], but I guess that's really a minor detail, don't bother.

[Lee04]

What stage are 2.0 in right know?

Have you used Intels Vtune or simular, to actually see what eating up the CPU time?

[drwbns]

I think he already knows what's eating up CPU time, hence the GSOC project. He's refactoring at the moment - https://bitbucket.org/dark_sylinc/ogre2-gsoc

[Lee04]

I figure better to ASK then too be sorry, since I seen people optmizing 3D engines before and got little to no results if they don't profile....

[Klaim]

Shouldn't profiling be done when you finish the most important changes?

[spacegaier]

Shouldn't profiling be done when you finish the most important changes?

Both I think. First upfront before you start so that you know where to actually optimize (where you will get the most gain by the fewest amount of work) and then afterwards to check whether your assumptions about the optimization held true and you actually gained a boost.

Mathias definitely got an idea what needs to be done as others pointed out, no small part thanks to the investigation and analysis he made and released publicly.

[sajty]

Just for reference: Mathias posted about about some upcoming changes on his blog:

Ogre 2.0: O(1) insertion, removal and contiguous memory iteration

This technique looks very similar to the VectorSet class used here: https://bitbucket.org/sinbad/ogre/src/c ... v1-9#cl-89
If you base it on Ogre::SmallVector, you can optimize out extra allocation for the buffer, but you need to tell the suggested size as a template parameter. In Ogre::ProgressiveMeshGenerator this is very important because we have thousands of very small vectors.

[FrameFever]

Does the Ogre build under windows still uses the security buffer check (/GS)?
This will cost a lot when you have many function calls. It should be disabled.

Anyway, what is the current state?
After every change, you (we) should see how the performance improves.
How do you measure the performance of Ogre?

[masterfalcon]

It may or may not currently compile. There are many huge changes so performance checks after each checkin aren't realistic. As for the current state, check the mercurial log and this forum.

Unless it is added by default, /GS is not added explicitly by Ogre.

[AshMcConnell]

/GS is set by default in VS 2012 at least (I just checked). It is early days, from what I've seen things still don't compile yet. A lot of ripping out needed before the awesome happens

@FrameFever : is there a list of recommended optimisation flags for a recent version of VS anywhere?

[FrameFever]

It may or may not currently compile. There are many huge changes so performance checks after each checkin aren't realistic. As for the current state, check the mercurial log and this forum.

Unless it is added by default, /GS is not added explicitly by Ogre.

I know it is set by default, you have to disable it with /GS-

Does Ogre will not compile without this "security check"?