[GSoC 2013] Topic Proposals from Ogre3D Dev Team

[spacegaier]

General

We internally have come up with a list of topics that we consider especially important and would therefore like to see covered in this year's GSoC 2013. That doesn't necessarily mean that we will not accept other topics (just try to convince us ), but focusing on the areas we list below will probably increase your chances of getting picked.

But remember: There is no sense in now trying to force yourself into one of these topics, just for the sake of it, if you don't have the necessary skills, knowledge and motivation to actually properly tackle those...and believe us: we will find out, most likely already in the early discussion before GSoC actually starts, so please don't test us !

Topics

Note: The order has no particular meaning other than that was the order we discussed the ideas!

1. Automated nightly builds / continuous integration / completion of visual testing
Picking up the previous GSoC project that set the foundation for visual test cases. Polish it and implement a productive version for Ogre, combined with continous integration / nightly builds for all supported platforms, e.g. in combination with JIRA via the Atlassian product "Bamboo". Also, expansion of the functional unit tests.

2. Modularization of the Animation system + refactoring and extending
As part of our effort to make Ogre more modular, the animation system should be moved into an own component and at the same time various bugs should get fixed and new features implemented.
The problem with the actual impl is that the core is polluted with animation members regardless you need animation or not. Decoupling means here to remove all members which for example exists in the Node class which is just there for the node track animation and move that into a own animation component.
After that the student should rework the software skinning part using OpenCL or make an addition to the current implementation which make use of many cores. This work could lead into Ogre 2.0 direction because mentor and student could start with an actual problem.
We could also split this to two students.

3. Ogre Footprint Profiling / FPS hickups / Ogre String ID removal
Further investigation into Ogre's bottlenecks as an addition to Mathias' powerpoint slides. Also optimization of Ogre's footprint, e.g. by getting rid of all the string identifiers which are a huge issue on mobile devices since they get backed into the DLLs and executables and bloat up the result.
So removing all runtime string constructions / comparisons throughout the whole library - which also covers the material parser.

4. Coverity Development Testing/Analysis
The project will be to run Coverity (we have an open-source license) on our code base, create a "how to" that will save us time to recreate the test in the future, create virtual machine with the needed tools. Then fix the issues found by Coverity, perhaps create bugs in ours bug tracking system for all the issues. Optionally, also fix the issues the PSV Visual Studio plugin reports for which we also have a team license.

5. Improve RTSS performance
RTSS has still some potential for performance improvements e.g. by adding a better state identification.
RTSS is really important on mobile devices, but we would need to target a major rework how shaders are generated.
First modify RTSS to generate shaders without the usage of sub-functions (this a killer for most drivers on Android). The thing would be to change the shader library instead of functions we should create some kind of snippets - in this part we should be thinking a bit in the direction of unity how it can define shaders at a higher level.

6. Ogre 2.0 Refactoring
Similar to idea #3, tackling first parts of the redesign, e.g. the SceneManager traversal. This would be an R&D project were the student tries to refactor the SceneManager into more maintainable code + documentation. This is more a refactoring task but it could be used as base because I (Murat) do not thing we can merge this back 1:1.

7. Texture Streaming
A modern technique, that would help things like the terrain system, loading times, etc. See the Help Requested page for more info.

PS: More ideas can be drawn from the links in the "Sources" section in the general GSoC 2013 information thread!

[masterfalcon]

#1 can be significantly scaled back to just expansion of the visual and functional unit tests. I've been working on OS X and iOS support for the visual tests in the last couple days and I already have Bamboo set up...though dedicated hardware for agents would be nice.

[Klaim]

I believe #6 is not, in the end, "refactoring". My understanding is that it's a deep architectural change, isn't it? Even if it is isolated into OgreMain (or maybe not only).

[Wolfmanfx]

Refactoring means to restructure the existing code in way we could isolate the behaviors/feature in a better way but still keeping the same code around just better structured. Atm the scenemanager needs such a restructuring.

[Lee04]

Texture streaming as already been done for Ogre, two years ago if I don't remember wrong.
And many, many other cool stuff as well that hasn't made it into Ogre and are now absolete, because there where not maintained.

However they are not always made as an Ogre permanent feature, rather looked at extravagant features not necessary for the great Ogre.
Just years later the Ogre team wants it because the feature has become mainstream in so many other engines and games.

Like wise Optimization has been contributed to Ogre that never made it into core, even if it was submitted. My work mine was excepted by the Ogre team, but the optmization was ripped out.

And now are we suppose to those optimizations back in for version 2.0??

Why not just accept code that are optimized to start with, and start adding new cool stuff to Ogre as the community members makes them.
Instead the extras feature modules get outdated, because Ogre moves on and no one makes sure the that the extra cool extra add ons breaks.

[jacmoe]

What I am disappointed to learn is that PCZSM is not going to be maintained, which is a pity.
It is really a cool scene manager.
I wish there were community members who were contributing fixes to it, at the very least.

[bstone]

Yeah, but once we have a decent occlusion culling implemented for Ogre it will be less of a pity.

[jj]

Hi!

Not sure if this is already possible (not easily from what I know), but related to the Scene Manager refactoring or maybe as a separate area, it would be amazing to have the possibility to "enable" something like immediate rendering in Ogre (meaning the possibility to draw things "as we go" as opposed to having it all rendered at once by the scene manager. I know this kills the optimizations that we gain by having scene managers, but for some kinds of projects it's really useful. Let me explain a bit more:

In other libraries like Processing (http://www.processing.org) or OpenFrameworks (http://www.openframeworks.cc) this is generally used as it leads to easier handing of drawing with "layers". So for example if I have a couple quad with textures, I can do (this is just pseudocode)

• Draw quad1 (would be on the bottom layer, rendered first)
• Draw quad2 (would be on top of quad1, as it's render call is made afterwards)
• Draw quad1 (finally on top of the other 2, even if it is the same object... as render is called last).

And if they are drawn in the same are from the camera's viewpoint, they will be drawn in that order, like "painting layers" so you can easily reuse an object to draw it multiple times. In a scene based system like Ogre, for a situation like this you need to have 3 different instances (not 2), but more importantly, ensuring who is rendered in what order is very problematic. Beyond using different Z values, which could not be an option if you want the 3 objects to be in the same plane and can give Z-fight issues if the Z values are too close... the only possibility we have in Ogre is to use the render queue. But this is very limited (as there is only about 100 render queues) and it is not optimal if we are doing this for thousands of objects.

We are building a library to create interactive projetions and such that uses Ogre as the graphics engine, but non being able to render in "immediate mode" is quite problematic for situations like the one I described, which happen very often in quick prototypes or full projects like those created with these tools.

Thoughts?

Thanks!
jj

[Lee04]

"What I am disappointed to learn is that PCZSM is not going to be maintained, which is a pity.
It is really a cool scene manager.
I wish there were community members who were contributing fixes to it, at the very least."


Figures.....

What is the "new" hardware occlusion scene manager system?? Or is it going to be a software aproche like Dice uses?

[bstone]

What is the "new" hardware occlusion scene manager system?? Or is it going to be a software aproche like Dice uses?

"New"? "Hardware/software"? Where does that come from? As long as it's decent I don't care

[dark_sylinc]

What is the "new" hardware occlusion scene manager system?? Or is it going to be a software aproche like Dice uses?

Ogre 2.x will be in phases, until we reach feature complete we want in 3.0

Ogre 2.0 won't have any occlusion culling other than Frustum Culling. Ironically, we estimate this will already be several factors faster than current 1.9 branch because of extremely innefficient scene traversing.

Ogre 2.1 or 2.2 will include a software rasterizer for occlusion culling (ala Dice) and an octree-like high level system.
Starting 2.x, we'll have two kinds of culling: Low level & High level culling.
High level culling needs to follow very strict rules in how they output their occluded entities; but they're algoritmically very efficient (i.e. a 2D or 3D Grid, a tree-based scenegraph, portals, etc). What we know as OctreeSceneManager & Co. are examples of it (though the interface will change drastically, I doubt any 1.x scene manager will even compile for 2.x)

Low level culling doesn't follow those strict rules, but it is usually tuned down for performance at instruction level and is all about Raw Power. A software rasterizer occlusion culling system is an example of it. A hybrid solution like Crytek's (Coverage buffer) is also possible, or even full GPU HW occlusion culling (except for predicated rendering because it's more tied with the API after sending the commands, rather than culling before we need to send the commands)

In theory both culling systems are pluggable: You will be able to choose between combinations of High & Low level culling systems (or even null, like 2.0), so you could have portals and hw gpu occlusion culling, or 2D grids & sw occlusion culling, etc.
We plan to maintain a sort of tree/3D Grid based solution for High Level Culling & a software rasterizer for Low Level Culling. Any other variant the community is invited to write & maintain their own solution (unless they turn out so superior we end up adopting it as default, of course) just like today.
Of course, we cannot guarantee that all combinations of cullers will work i.e. a Low Level Culling system specifically designed to be tied with the High Level Culling system. I can't think of any example for that and we won't encouraging it, but it's worth mentioning

There will be happy users, angry users; we hope to please most. The reason we're moving is because it's current state is hard to maintain and the performance we get from the engine overall is lame.

I hope this clears your doubts. If you're curious about the strict rules High Level Cullers need to follow, you can look at my slides (or wait for the full documentation after GSoC).

[drwbns]

Are the proposed occlusion culling techniques not currently possible to implement with the current Ogre or is a complete redesign needed to implement occlusion culling? I'm curious as to what needs to change or if it is possible to implement a straight forward technique with Ogre 1.8.

[dark_sylinc]

It's not impossible, but we have little to gain for more aggressive culling. Right now we're extremely CPU bounded due to inefficient scene graph traversal, transformation, and frustum culling. Adding i.e. a software rasterizer would only make things worse.

We'll be now focusing on 2.0 to improve performance, and there's where we'll add the new culling techniques.

[drwbns]

Ah ok. makes perfect sense. one question i had about occlusion culling is whether or not the culling needs its own octree as opposed to using the scene manager directly. can u enlighten me?