Page 1 of 1

(submitted) .fx to .material file converter proposal

Posted: Sat May 06, 2006 1:59 am
by gzmzhen
I've finished writing a rough draft of my proposal. This is the first time I've written such a thing and not completely sure what I should include, or perhaps remove. If people tend to think it lacks length or thickness, I do not mind going further into depth describing the .fx file format and the .material file format. I've found some helpful resources and have an idea of where I will start to go when I start working.

... Anyways, I'm looking for some constructive criticism. Thank you, Nate.

---
[EDIT] Here is a near-final draft of my proposal. Thank you for your feedback.

OGRE 3D Summer Of Code Proposal
Nathaniel Bauernfeind

nbauernfeind@cmu.edu

The Proposal: [UPDATED]

Create a command line program that converts DirectX .fx files to OGRE .material files. In addition to the tool, the way that it will be implemented will allow this parser to work as an add-on to OGRE.

Why?

Many people learn about OGRE while they are in the middle of a project, or after they have many resources already available to them. This tool allows those developers who have previously used DirectX, or have access to previously used models that were made specifically to be used with DirectX, to export their data to something that OGRE can use without recreating the .material files which is a time-demanding activity.

In addition to converting files, it is an easy door for developers who are used to creating .fx files learn the .material file format by simply converting files and noting the differences. They can learn what types of things that they could do with a .fx file that isn’t supported with a .material file (if there are such things).

Why Me?

I am going to be a sophomore at Carnegie-Mellon University in Pittsburgh, PA. I am currently aiming for a BS degree in Computer Science with an additional major in mathematics. I have been working with Systems Scientist Dr. Yang Cai who is currently doing research at CMU. I convinced him to let me use OGRE for some of the software he has asked me to create during this past school year. I, therefore, have a basic understanding of OGRE and how it works. This project is a perfect opportunity for me to dive behind the scenes a little more and understand a little bit of how the underlying API works. I aspire to create a top-down “Crazy Taxi” style game at some point in time before I graduate using OGRE, and this is one more project that will get me closer to that goal.

[INSERTED]
Here is a website that has previous projects I’ve worked on and completed along with descriptions and source. http://www.andrew.cmu.edu/~nbauernf/

Methodology To Tackling This Project: [BIG UPDATE]

In the OGRE engine there is already a nice parser called OgreMaterialSerializer that already has the capabilities to import a .material file and to export an already created material to a file. This makes the job of creating a 1 to 1 correspondence even easier, because now I need to just map the .fx file format to the material class itself. As if it were a separate serializer.

Basically, I will create a new serializer called OgreFxMaterialSerializer which will attempt to load the .fx file into a Material object. This can be added to the OGRE SDK as an add-on for anyone who would like to use it in lieu of using just the command line.

Although this project requires no new research, I will need to learn .fx file format well. I want to create a list of features for both .fx file-format and the Material object and attempt to create a one to one correspondence for as much as possible between the two of them. The parser will basically start with supporting none of the .fx features. Then features will be added one by one to make sure that each .fx function is fully supported before adding another. This reduces the number of run-time failures and greatly reduces the amount of debugging time required to fix those run-time failures (because we know exactly where the bug is located).

During the initial stage, in addition to creating the map, I will also attempt to collect a library of .fx files that span the entire set of available features that are supported by DirectX. Hopefully some, not necessarily all, of these files will come with something that allows the files to double as a secondary check to see if the parser is working correctly. It would also be extremely helpful to find someone in the community that would be willing to test their .material files versus ours parsed from the tool (given they still have the original .fx files).

The entire implementation will be written in C++ as is OGRE.

Time Break Up:

Starts May 22nd.
2 Weeks to gather .fx files and create a map between .fx and .material files.
3 Weeks to create first generation map between .fx and .material files and make last major modifications to the map.
1 Week to prepare mid-program progress document (basically a start on the final document).
Approximately June 30th
2 Weeks to produce a “rough draft” of the parser. Entire functionality should be supported in this version. Also submit to public for testing.
2 Weeks to reproduce (or produce) any initial problems and fix them.
1 Weeks to finalize debugging any problems and publish completed .fx parser.
Approximately August 7th
2 Weeks to prepare and complete final document for Google which is due 7/21 @ 5pm.
Total: 13 weeks

[COMMENT] I was off on the number of weeks before... for some reason I missed that the student deadline is August 21st [/COMMENT]


Challenges:

I think that the major challenge will be finding an exact map between the .fx and .material files. I know that as I write the parser that the map will not be complete and may have ambiguities that may not have an appropriate answer. The challenge will be to find what the correct thing is to do when these ambiguities arise.

Conclusion:

I have written several data mining tools for Dr. Yang Cai and others who work for him. I am confident that this is simply another tool where I need to parse the data and ‘reinterpret’ it such that OGRE knows how to use the data. To be successful I need to focus on separating the file format into sections and support one new feature at a time. I know I can create this tool and do so within the time frame.

Posted: Sat May 06, 2006 2:39 am
by Lioric
In your application please include some links to a code sample of past projects you have made

Im not sure about adding a .fx format parser into the core engine, in any case you might want to consider doing this part as an optional addon module

Posted: Sat May 06, 2006 2:53 am
by gzmzhen
What's better, a small project that has been entirely completed? Or a larger project that never quite was finished?

Also, is it appropriate to have Java code as an example? The Data parsers I've made for Dr. Cai were written in Java, but are rather relevant to the concept of transforming things.

Thanks again, Nate.

Posted: Sat May 06, 2006 3:43 am
by Lioric
Any functional code where you can show your code habits (completed projects or not, just your code), how you comment code and a few of your programing skills, the code should be in the language your application targets, but i recommend to use c++ as this is the language used by Ogre engine

Posted: Sat May 06, 2006 11:43 am
by sinbad
Proposal looks good, I like the level of detail you've included about your proposed approach and the basic principles of the tool. Your assertion that the mapping is likely to be the trickiest part is on the money.

My suggestion on the external tool / incorporation question is that the converter should basically be a set of classes which will parse a .fx file and load it into in-memory Material/Technique/Pass/TextureUnitState/GpuProgram etc classes. An external tool could then use these classes, plus MaterialSerializer to write the data back out to a .material script. However if a user of OGRE wanted to load .fx directly they could use the classes at runtime too. We wouldn't add this to the core, but the structure would allow it should the user choose to go that route.

Posted: Sat May 06, 2006 12:14 pm
by gzmzhen
Wow, that's a great way to create the tool. This would basically allow anyone who always makes their materials during runtime to export them to files to perhaps speed up the run time or to simply get a better grip of the material files themselves =).

I really appreciate your help so far =).

Nate

Posted: Sat May 06, 2006 12:16 pm
by sinbad
gzmzhen wrote:Wow, that's a great way to create the tool. This would basically allow anyone who always makes their materials during runtime to export them to files to perhaps speed up the run time or to simply get a better grip of the material files themselves =).
Not sure if you realised, but MaterialSerializer already exists, and several tools already use it to write material scripts from in-memory materials. This is our general approach to all file interfaces - use in-memory classes and use serializers to get data in/out from whatever source required. It makes exporters much easier to write.

Posted: Sat May 06, 2006 12:28 pm
by gzmzhen
I just brought up that class... and all I can say is Wow! If there's a better definition for how to write a material file (better than the manual section 3.1) it's this piece of code.

Thank you Sinbad!

Posted: Sat May 06, 2006 5:45 pm
by gzmzhen
I've made a lot of changes to my proposal (see above) as well as have added a link to an external site with a few pieces of work. The first two are in c++ and the last is in java.

If there are more suggestions just let me know.

Thanks again,
Nate