[GSoC 2009] Ogre Scene Serializer.

[naruse]

Hi guys,
Im an undergraduate student from computer science at EAFIT University in Colombia, I'm in my last year .

I have previous Experience in 3D engines (not a lot in OGRE3D, just took a class or two inspecting OGRE) , and I have made some developments in Unity3D (http://www.unity3d.com), So I can say that I am in someway new to OGRE

Also I've worked developing the collision system of a laparoscopic surgery simulator that was being developed in the Virtual Reality Lab at EAFIT, In this project, they used plain OpenGL and C++, no 3D engines where used.

As a show of my portfolio I have worked in:

http://www.cloudwitch.com/ --> Where I Developed all the menus, the Input Management System and the scoring System, also I developed the desert Level.

http://forum.unity3d.com/viewtopic.php?t=14569 --> Fully developed this "Guitar Hero Clone"

Both projects participated in the Unity Awards 2008.

Also I have made some simulations in the same engine (Unity3D):

http://juanmunozar.blogspot.com/2008/06 ... ysics.html --> If you cant see the demo working, please let me know I'll send you a copy of the program working.

Also simulated a water surface in Unity3D

And at the moment Im working on my thesis wich is a visualization of the stocks market in 3D wich can be found at:
http://190.249.35.94:8080/

PD: To check these examples, you'll have to be in MacOS or in Windows and download the plugin to see them,Sadly there is not support already for Linux :'(


Project Proposal: Full scene serializing

Serialize a scene to a file and enable loading it from file; Work on a program that can Load and save a full scene from a file that has all the atributes of each component of the scene inside, (rotations, positions, scaling,...etc), and be as optimized as possible in order to achieve fast times in loading / saving scenes.


Schedule

I'm able to work all the days from monday to friday, and also sat - sun if necessary and if we are running out of time.

If Im not wrong, the project extends for 3 months, that's 12 weeks.

Week 1:

Read, and get information about the different objects that interact in a scene in ogre and how these objects can be checked in order to know which objects are going to be serialized, and also get documentation on how other 3D engines manage their scenes, and start thinking in the best way to serialize a scene in ogre without losing simplicity

Week 2:

Document myself about the different ways of serializing objects, study them carefully and choose the best one that can work in OGRE.

Week 3:

Start designing a scene loader and a scene serializer that loads and saves a scene in a plain file.

Week 4:

Implement the scene file serializer given a scene.

Week 5:

Continue implementing the scene file serializer.

Week 6:

Test the scene serializer and fix bugs that can appear.

Week 7:

Implement the scene loader given a scene

Week 8:

Continue Implementing the scene loader.

Week 9:

Test the scene loader and fix bugs that can appear

Week 10:

Merge the scene loader and the scene serializer and test them both on different scenes and fix bugs that can appear.

Week 11:

Continue testing the scene loader and serializer, and start optimizing if no bugs are found

Week 12:

Continue testing and optimizing the scene loader and serializer, and document what I have done.



Future Extensions

Make a GUI for the scene serializer / loader, in order to place objects/lights/particle systems, etc. more easily, so people dont loose time putting objects and setting atributes in code, and help people focusing on the game logic, instead of the position of the objects/setting attributes.

Why You're The Person For This Project

As my knowledge in 3D engines is still vague, I love a lot low level programming, and as I showed in the abstract, I have worked in some games, sadly they where not developed in OGRE3D, maybe because in the company I was working (http://www.c2estudio.com), they Use Unity3D as their engine, but working on this engine (Unity3D), helped me a lot understanding some fundaments of 3D Graphics, and how things work when you are doing a game.

I have a decent experience with C++(check the collision detection on the laparoscopic surgery simulator at http://code.google.com/p/laparoscopica/ ... wse/trunk/).

Also, I have a decent written english and I can read pretty much anything you throw at me.

Why OGRE?

Its simple, the first approach to a 3D engine I got was to OGRE, sadly I was on my 1rst year so I didnt understand a lot of things, I also compiled the sources on linux and I ran the examples, but I wasnt able to do things because I didnt understand a lot of concepts I do understand now.



Now, I want to work in OGRE3D, because Its really interesting being part of a team that developed a 3D engine, I know I will learn a lot of things that I dont know now, and also I want to work on a engine different from Unity3D, even if that engine is easier than OGRE

Anything Else

Feel free to ask me any question, or let me know any suggestion

[KungFooMasta]

I don't want to sound overly negative, but there is a dot scene format used to serialize scenes, and through using tinyXml library, serialization of scenes isn't too large a task, IMO.

[jacmoe]

I don't want to sound overly negative, but there is a dot scene format used to serialize scenes, and through using tinyXml library, serialization of scenes isn't too large a task, IMO.

It was added to the Help Requested Wiki Page on 31st of January this year. Which makes it a GSOC project. AFAIK, it's not old dotscene, but serialising a scene directly. A binary scene dump, if you like. (As opposed to the xml scene, or the xml mesh for intermediary exchange)

[naruse]

I don't want to sound overly negative, but there is a dot scene format used to serialize scenes, and through using tinyXml library, serialization of scenes isn't too large a task, IMO.

ui
Well, then if its not a big task as you say, then What about creating a UI for making scenes in Ogre, and placing objects in the scene, and then serialize the scene created with the Library that you are talking about? .

PD: I'm not sure if this UI Project for scenes serialization is too big for the GSoC tough :S, and Its a feature that I have not seen in OGRE .

[KungFooMasta]

As Jacmoe says, if its been added to the Help Requested Page, then its probably a feasible project. If the output is a binary file, like .mesh, that might be pretty cool. (I don't know much about scene seralization other than writing and reading a specified format, so I assumed it wasn't a major task. Hopefully the Ogre team will respond and clarify things)

[nullsquared]

I recommend Google Protocol Buffers. Much more efficient and simpler than using something like XML.

[naruse]

Now that you guys are mentioning it, I was thinking about the serializing, into a XML, or another way to save the scene, never tough about a binary file, would a binary file be better than a plain text file?, we would be getting performance?

Just saw the google Protocol Buffer, its pretty awesome!

[jacmoe]

Just saw the google Protocol Buffer, its pretty awesome!

Yes, it is.
I think this should be in Ogre, though. But I'm not sure.
Let's hope Assaf jumps to our rescue and explains the idea.

[Pyritie]

Now that you guys are mentioning it, I was thinking about the serializing, into a XML, or another way to save the scene, never tough about a binary file, would a binary file be better than a plain text file?, we would be getting performance?

It'd certainly be better for the times where you don't want users to easily edit the saved game files :P

[tuan kuranes]

You have to be a lot more detailed and specific, both to let us know you understand and that you'll be able to do it...

- Do you plan a scenemanager agnostic ?
- Do you plan per scene manager extension ?
- How do you plan to handle Zones ?
- How do you plan to handle pages ?
- How do you plan to design it so it can be still used in regards of Ogre 2.0 new scene management ?
- etc...

[sinbad]

For info, in trunk there is a class called StreamSerialiser which does everything an OGRE user really needs for binary serialisation. I've looked at Google Protocol Buffers before but unless you really need the language-neutrality it's a bit overkill. I'm using StreamSerialiser for the new paging system and it's really easy.

A non-paging serialiser is still useful as a component, and could be used by page content providers if necessary. But yes, more detail is needed on this application.

[naruse]

Hello,

I did some research on how OGRE 's SceneManager works.

Firs of all, I'll start by serializing the ST_GENERIC type of scene, which is the simplest one, and as I have seen in the documentation and in the book, the SceneManager works as a Tree (with the scene nodes) so its possible to go over all the scene simply by going over the tree, which leads me to think, that there shouldn't be a problem regardless the scene type for serializing.

I didnt think on Zone Handling / Pages handling, I only thought about the Scene and the objects that the scene contains, not how does the SceneManager Organizes itself to render more efficiently the scene.

You are right in one thing and makes me think now, how am I going to reconstruct the scene?, well. in the serialized file, at the beginning there can be a value that tells me how was this scene saved, so its just a matter of adding all the objects again to a new scene and set the type for the scene as the file says.

Now, Maybe things are not as easy as I see them (maybe I'm not aware of how things are in practice ) but I know that the mentor is going to show me which things are right and which things are wrong.

As for Version 2.0, I wasnt able to find how is the SceneManager going to change , but I hope that if it still behaves as a tree, there shouldnt be problems on porting it

If anyone can Give me a link to the 2.0 Specs, or can show me where to find Info about version 2.0, would be awesome!, and I will for sure give it a read!:D

[stealth977]

naruse:

About serializing the other scenes (Except PCZSM which I have no experience with):

In my opinion you dont have to save the WORLD_GEOMETRY specific to the Scene Manager. You just save the initialization configuration file name. And when loading the scene, you call SceneManager->setWorldGeometry(configfilename) and the scene manager auto loads its specific world geometry

There is another exception here: For the BSPSM, you actually need to bind the resources with an extra resource function, dont forget that special case.

And here comes the tricky part with no general purpose solutions. How will you serialize the PLUGIN COMPONENTS?
- Hydrax can save its config to a file, but you have to check if its present, you cant only save the mesh it generated
- Caelum can not save its configuration by itself, you have to detect and save it yourself, very very special case
- Physics?? How you gonna serialize it?
- Plugins add extra dynamic objects to scene, how will you detect and exclude them? (Hydrax have mesh and compositors, Caelum has many nodes/objects added to scene, Paged Grass has its page meshes in the scene, you cant just save them, they are dynamic)

So, to sum up, implementing special case for different plugin objects is not efficient, what if a new plugin is added that you are not aware of?

One possible solution is to provide serialization support for plugins, a function all plugins must implement with the same name (for example dllSerializePlugin() )

Also, you need to detect if an object is Plugin Dependant, like a function object->isPluginDependant(), but the problem here is that OGRE objects are not derived from an OgreObject base class, so you cant depend on overridden virtual functions...
If it was derived from a general base class, you could always use object->isPluginBased() or object->serialize() provided that the object properly overridden those virtual functions...

Thats my 50 cents

(I spent a lot of time trying to serialize scenes for Ogitor, and also I am sure Nauk too has a lot to say about the subject)

ismail,

[naruse]

naruse:

About serializing the other scenes (Except PCZSM which I have no experience with):

In my opinion you dont have to save the WORLD_GEOMETRY specific to the Scene Manager. You just save the initialization configuration file name. And when loading the scene, you call SceneManager->setWorldGeometry(configfilename) and the scene manager auto loads its specific world geometry

Yes, I was thinking in that

There is another exception here: For the BSPSM, you actually need to bind the resources with an extra resource function, dont forget that special case.

Have a question, why do I need to bind the resources to an extra resource function?, dont understand this quite well :S

And here comes the tricky part with no general purpose solutions. How will you serialize the PLUGIN COMPONENTS?
- Hydrax can save its config to a file, but you have to check if its present, you cant only save the mesh it generated
- Caelum can not save its configuration by itself, you have to detect and save it yourself, very very special case
- Physics?? How you gonna serialize it?
- Plugins add extra dynamic objects to scene, how will you detect and exclude them? (Hydrax have mesh and compositors, Caelum has many nodes/objects added to scene, Paged Grass has its page meshes in the scene, you cant just save them, they are dynamic)

So, to sum up, implementing special case for different plugin objects is not efficient, what if a new plugin is added that you are not aware of?

One possible solution is to provide serialization support for plugins, a function all plugins must implement with the same name (for example dllSerializePlugin() )

Well, to tell you the truth, I was only thinking on the standard things that uses Ogre, like entities and the world geometry etc. never thought about a a weather manager or a fluid system, wich is something I should have thought about .

But you have a great Idea that can solve the problem for those plugins, and its to implement an abstract method that every plugin has to implement to provide serialization for each plugin.

This is also a good thing, because the serialization system would be able to support any plugin that can be created , just implement a the serialization method for each plugin, and you should be good to go :p

Also, you need to detect if an object is Plugin Dependant, like a function object->isPluginDependant(), but the problem here is that OGRE objects are not derived from an OgreObject base class, so you cant depend on overridden virtual functions...
If it was derived from a general base class, you could always use object->isPluginBased() or object->serialize() provided that the object properly overridden those virtual functions...

Thats my 50 cents

(I spent a lot of time trying to serialize scenes for Ogitor, and also I am sure Nauk too has a lot to say about the subject)

ismail,

I quite dont understand this well again :S, I need to know if an object is plugin dependant for what reason?, I suppose that if you serialize a scene and then after serializing the scene you build up the scene again (lets say in saving/loading a game state), you still have the plugins in the code, so why should I need to know if an object is plugin dependant if I have the plugin already in the app?, maybe I didnt understand well what you are saying .

Anyways, If i get accepted I think I'll ask you a lot of questions

[stealth977]

1 - BSPSM uses pk3 files for the scene, the resources inside the pk3 should be mapped and there is a resource binding function which i forgot just check the bsp demo...

2 - Why detect if an object is plugin dependant? Because:
All plugins create some kind of meshes in the scene, if you serialize those meshes then you have a problem of duplicate objects, to make it clear:
ydrax creates a water mesh, if you serialize that mesh, it will be a static mesh representing the state it is saved at, but useless without its textures and compositors, and even worse, if you load hydrax too, it will crash because of name collision, or will have 2 water meshes at the same time, making the scene visually wrong. Again, Paged Geometry has many meshes (even the grass pages are so) in the scene, if you serialize them, you will again have useless duplicate meshes...etc.etc.

Thats why you need to somehow detect if an object is a result of a plugin, if so, you must not serialize it to avoid useless duplicate objects which even may not have materials (since they are dynamicly created and modified by their plugin, or depend on compositors maintained by plugin)

ismail,

[naruse]

Understood , thanks!