fxogretool here

Threads related to Google Summer of Code
buddy
Google Summer of Code Student
Google Summer of Code Student
Posts: 78
Joined: Tue Mar 29, 2005 3:35 pm
Location: USA

fxogretool here

Post by buddy »

Hi folks.

I just completed editing wiki page on fxogretool (that is FxOgreExporter part of the SoC project).

http://www.ogre3d.org/wiki/index.php/SoC2006_RmExporter

The tool is ready for review, I made an archive of binaries available for download here, however you can compile from source in addons/RmOgreExporter fxogretool project (you'll need msvc80 to do that though).

http://sukhoy.public.iastate.edu/soc200 ... tool10.zip

You can try running it against FX files you have (I tried it on several examples from DirectX SDK). The parsing itself works on all files I could get (except DirectX 10 FX files, but that's just another superset language!) - e.g. you can get XML parse or AST tree of it. Material output should also work for any script, however I am not so sure of that as about parsing. It does work fine against 10-20 example files that I tried.

Running is as simple as fxogretool file.fx file.material however there are more options available. You can get progress messages with --verbose and request help with --help.

We are stripped off CVS access as of 8:00 PST Aug 21st, it seems, but I think I will be able to put patches here or elsewhere if the need arises.
User avatar
Jerky
Orc Shaman
Posts: 791
Joined: Wed Mar 02, 2005 4:13 am
Location: Springville, Utah
Contact:

Post by Jerky »

Fantastic news, I'll let you know how it goes. Great work.
Erik Briggs (Jerky)
My Blog
Project Wish
Image
User avatar
spookyboo
Silver Sponsor
Silver Sponsor
Posts: 1141
Joined: Tue Jul 06, 2004 5:57 am
x 151
Contact:

Post by spookyboo »

This is very needed and appreciated by many people. I haven't tried it yet, but I definitely use it. Should be part of the Ogre tools section in the download.
User avatar
Wretched_Wyx
Orc
Posts: 498
Joined: Thu Mar 16, 2006 5:27 pm
Contact:

Post by Wretched_Wyx »

Very cool- I've only played with it a little bit so far. Made a batch file for easy drag and drop. I have a big collection of FX files, and out of the 10 or so I tested, only 1 worked- a fur shader. I haven't tested it in my engine, but it did convert. The others gave parsing errors on line XXX. I assume it's the shader, as I can't view them in the FX viewer as well.

Anyhow, keep up the good work on this, I really like the tools you've been pumping out.
buddy
Google Summer of Code Student
Google Summer of Code Student
Posts: 78
Joined: Tue Mar 29, 2005 3:35 pm
Location: USA

Post by buddy »

Wretched_Wyx wrote: The others gave parsing errors on line XXX. I assume it's the shader, as I can't view them in the FX viewer as well.
could you post one or two of those that had parse errors so that I can have a look? I actually batch tested everything in DirectX SDK to see if there were parse errors, and could not find any.

The reason for possible parse errors is that boost::spirit is LL, whereas grammar in DX docs that I used was designed for bottom-up YACC-style parser, so I had to do some adaptation. Those kind of things may backtrack in very subtle way, so the only way to make sure it is fine is to actually test a bunch of files. Spirit, however, is much easier to deploy and tweak, so it was chosen because of the limited timeframe on the project.

If you used DirectX fx viewer app, then it requires shader to be SAS-compliant to display. It may be another reason for non-display.
Shoko
Gnoblar
Posts: 7
Joined: Mon Jun 26, 2006 7:00 am
Location: Togo, Africa

Post by Shoko »

Sorry buddy but where do I actually run this code:

fxogretool source.fx result.material
buddy
Google Summer of Code Student
Google Summer of Code Student
Posts: 78
Joined: Tue Mar 29, 2005 3:35 pm
Location: USA

Post by buddy »

Shoko wrote:Sorry buddy but where do I actually run this code:

fxogretool source.fx result.material
First, you will need MS Windows (I use XP, however it should also work on 2000 or 2003, less likely on 9x however it may happen so too).
Second, you have to download that binary pack or compile the source code from CVS.
That should give you a file "fxogretool.exe" and its dependencies (a bunch of DLL files - they are required to run fxogretool.exe). The file fxogretool.exe is a binary executable intended to be used from command line (it is a command line tool).

Therefore you will need a shell, the default one on Windows machines is cmd.exe. You can try other options, like Windows Powershell or bash.

Once you are in a shell, you will have to change your directory to where fxogretool is and also copy your FX file there. Once you have everything set up, you can run "fxogretool source.fx source.material" at your shell prompt.

Good luck.
big_o
Goblin
Posts: 279
Joined: Sun Feb 19, 2006 1:08 am

Post by big_o »

You run it from the command line. Scary isn't it.

I don't have the fx composer on my comp right now, but I do have rendermonkey, and it appears to completely ignore the texture units. I don't know if its a problem with rm's fx exporter or what. Its not that big of a deal, but it can be confusing.
Shoko
Gnoblar
Posts: 7
Joined: Mon Jun 26, 2006 7:00 am
Location: Togo, Africa

Post by Shoko »

buddy wrote:
Shoko wrote:Sorry buddy but where do I actually run this code:

fxogretool source.fx result.material
First, you will need MS Windows (I use XP, however it should also work on 2000 or 2003, less likely on 9x however it may happen so too).
Second, you have to download that binary pack or compile the source code from CVS.
That should give you a file "fxogretool.exe" and its dependencies (a bunch of DLL files - they are required to run fxogretool.exe). The file fxogretool.exe is a binary executable intended to be used from command line (it is a command line tool).

Therefore you will need a shell, the default one on Windows machines is cmd.exe. You can try other options, like Windows Powershell or bash.

Once you are in a shell, you will have to change your directory to where fxogretool is and also copy your FX file there. Once you have everything set up, you can run "fxogretool source.fx source.material" at your shell prompt.

Good luck.
And the material file will be created automatically right?

But on the command prompt I get this error:

The system cannot execute the specified program

Thank you soo much for your help. Looking forward to using your program :D
buddy
Google Summer of Code Student
Google Summer of Code Student
Posts: 78
Joined: Tue Mar 29, 2005 3:35 pm
Location: USA

Post by buddy »

big_o wrote: I don't have the fx composer on my comp right now, but I do have rendermonkey, and it appears to completely ignore the texture units. I don't know if its a problem with rm's fx exporter or what. Its not that big of a deal, but it can be confusing.
Well, if you can suggest the way to handle them... Say you have something like "texture blahblahblah" in fx file - ok, so then what?
What RM does when exporting texture is append annotation with texture location. But there is no standard on annotations. MS uses SAS (Shader Annotation Semantics), RM uses no SAS, but homegrown approach.

A similar regexp approach like for variable names seems feasible, some thinking is required though :).

As for sampler states.. It is doable, but a little bit hard. And I am not so much sure that it is needed at all, since they are handled by shader code anyway. However if you do not intend to use produced .material with the FX code it may make some sense.

FX uses sampler state in much different way comparing to OGRE. Basically they are not part of pass state, they are declared explicitly but used implicitly within shader body. So to handle them, one has to look into shader body, look for sampler being involved and behave appropriately.

That sounds much easier than it is. But doable.
buddy
Google Summer of Code Student
Google Summer of Code Student
Posts: 78
Joined: Tue Mar 29, 2005 3:35 pm
Location: USA

Post by buddy »

Shoko wrote:And the material file will be created automatically right?
yes, but it may not be the material file you need. There may be some editing required on top of the result.
Shoko wrote: The system cannot execute the specified program
What kind of system do you have? You can try filemon or depends.exe (I think you can get latter at http://www.dependencywalker.com/) to see what is missing on your system.
User avatar
Wretched_Wyx
Orc
Posts: 498
Joined: Thu Mar 16, 2006 5:27 pm
Contact:

Post by Wretched_Wyx »

Buddy,

I hadn't tested the DX samples, as I took your word that they worked. What I was testing were FX files made by the general public. I can post a link to a zip file with the ones that didn't work if you'd like.

Also, I can post a batch file, so that those of you scared of the command line can simply drop files onto the batch file, and it will do all the fancy work for you. There would be a simple edit though, the path to the fxogretool. Actually, it's as simple as this:

Make a new text file. Add these lines to it:

c:\path\to\fxogretool\fxtool.exe %1 %1.material
pause

Save it, and rename the file to convert.bat

If you have spaces in the file path, put quotes around the path, like so:

"c:\path\to\fxogretool\fxtool.exe"

If you want to view all the commands, you can make a seperate batch file, or modify this one to look like this:

c:\path\to\fxogretool\fxtool.exe --help

When you want to convert an FX file, drag a file onto your newly created batch file, and voila! Make sure that the batch file, and the FX files are in the same folder as the tool. You can also get crazy and add an environment variable for the path to the FX tool, but this is just a quick and easy way. Hope it helps some people.
buddy
Google Summer of Code Student
Google Summer of Code Student
Posts: 78
Joined: Tue Mar 29, 2005 3:35 pm
Location: USA

Post by buddy »

Wretched_Wyx wrote: I hadn't tested the DX samples, as I took your word that they worked. What I was testing were FX files made by the general public. I can post a link to a zip file with the ones that didn't work if you'd like.
If it's not hard. To fix the grammar I need an example of nonparsable input :)
Shoko
Gnoblar
Posts: 7
Joined: Mon Jun 26, 2006 7:00 am
Location: Togo, Africa

Post by Shoko »

Thank you, I have just managed to get through.

But I have one more question please :oops:

how do you deal with RenderColorTarget ?

can I mail you a .fx file as an example?
buddy
Google Summer of Code Student
Google Summer of Code Student
Posts: 78
Joined: Tue Mar 29, 2005 3:35 pm
Location: USA

Post by buddy »

Sorry for late response, was kind of busy around here.
Shoko wrote: how do you deal with RenderColorTarget ?
Oh, it is a semantic.
Actually if I understand correctly you have something like
texture blabla : RENDERCOLORTARGET.
Well, it means that blabla is a render-target texture. You see, the real problem is that semantically FX and OGRE material framework are somewhat close, but not at all equivalent. In Ogre it does not seem like you can do render targets through material framework, you have to setup them via C++ and your application code. FX is semantically more complex than ogre materials, so it is virtually impossible to devise "full" mapping, one can only hope to do "subset" mapping - e.g. map some subset of FX capabilities into OGRE. The same holds for semantics - it is possible to process some simple one's and map it to predefined variables, etc. but it is impossible to map everything given the way OGRE material framework works.

Hope that what I wrote here makes some sence :).
buddy
Google Summer of Code Student
Google Summer of Code Student
Posts: 78
Joined: Tue Mar 29, 2005 3:35 pm
Location: USA

nonworking examples

Post by buddy »

@Wretched_Wyx

I received your examples. Many thanks!

I figured out the source of most problems with the scripts:

1. Inline asm code within technique is not supported right now. Besides everything, it is a grammar flaw.
2. Closures.. things like PixelShader = f2(lighDir); "f2(lightDir)" is actually a closure. This is a flaw in processing of parse tree - one can get parse/ast tree of those files - it means that spirit part is fine.
I will investigate what can be done, but for now a simple walkaround might be just wrap closures in separate function in FX file before you feed them to the tool.

Both are fixable. I'll post updates here.
User avatar
Wretched_Wyx
Orc
Posts: 498
Joined: Thu Mar 16, 2006 5:27 pm
Contact:

Post by Wretched_Wyx »

Okay cool, It'll be a test for me to see if I can't fix some of those ;), I'm not so great with shaders.
User avatar
AekMuldoon
Halfling
Posts: 74
Joined: Mon Sep 25, 2006 3:44 am
Location: Perth, Australia
Contact:

Post by AekMuldoon »

We really need this tool, however we have issue with our .fx files.
Can I post my .fx files to you to see what the problem might be?
They contain HLSL code inside the Pass, I am wondering if that is the problem?

Thankyou for you efforts so far with this tool
User avatar
Evak
Orc Shaman
Posts: 707
Joined: Sun Apr 02, 2006 7:51 pm
Location: Sacramento, CA
x 1
Contact:

Post by Evak »

Hmm I tried some fx shaders exported from a new FX authoring plugin for 3dsmax called shader FX by lumonix.

I'm not a coder at all, but making shaders with the tool is pretty simple.
eg of the interface

Image

Anyway the two simple sample shaders I tried failed to parse. Both in a similar place in the source.


Test 1 .fx file was at line 40 [float4 light1Pos : POSITION ]

Code: Select all

/************** light info **************/ 

float4 light1Pos : POSITION 
< 
	string UIName = "Light 1 Position"; 
	string Object = "PointLight"; 
	string Space = "World"; 
	#ifdef DCC_MAX 
		int refID = 1; 
	#endif 
> = {100.0f, 100.0f, 100.0f, 0.0f}; 

Test 2 was at a similar place at line 40.

again with

float4 light1Pos : POSITION

I'll send you the files in PM so you can look at them if you wish, let me know if I can send you anything else or if you require media. Would be great if it were possible to get these FX files to work.

Knowing my luck its probably something really simple.
User avatar
Evak
Orc Shaman
Posts: 707
Joined: Sun Apr 02, 2006 7:51 pm
Location: Sacramento, CA
x 1
Contact:

Post by Evak »

been talking to the author of the above plugin, and they wanted to know if you use the Microsoft DXSAS format?

Code: Select all

 However, if you hear from the Ogre guys that they only support 'DXSAS" fx files, you'll know you'll be able to create those in the near future with shaderFX. 
Vectrex
Ogre Magi
Posts: 1266
Joined: Tue Aug 12, 2003 1:53 am
Location: Melbourne, Australia
x 1
Contact:

Post by Vectrex »

Good work Evak, I've checked out their demo and it looks fan-freaking-tasic.
I will definately buy this for our project if it works.
ps remember me from the days of the Aurora debarcle? :D
User avatar
Evak
Orc Shaman
Posts: 707
Joined: Sun Apr 02, 2006 7:51 pm
Location: Sacramento, CA
x 1
Contact:

Post by Evak »

Yeah I remember Aurora lol. Now have a very talented and much more experienced and reliable coder on the team now.

The pluging for 3dsmax is fantastic. I've been playing with it quite a bit and quite a few shaders that don't rely on the lights parse just fine.

Here's something cool Daniel_c did with it in his early tests with the plugin.

Image

CG support is in the works too, so you won't be stuck with HLSL which is DX9 only.

Really hoping buddy will take a look at the files, would be great if it was a fairly simple modification to get the shaders to work from ShaderFX
buddy
Google Summer of Code Student
Google Summer of Code Student
Posts: 78
Joined: Tue Mar 29, 2005 3:35 pm
Location: USA

Post by buddy »

Evak wrote: Really hoping buddy will take a look at the files, would be great if it was a fairly simple modification to get the shaders to work from ShaderFX
Here I am, duh.
DXSAS.. well.. too bad but it won't be a simple modification - i.e. SAS support should be coded within the tool from scratch (I bet the grammar handling logic should be significantly changed), i'm not sure if all of SAS can be supported w/o changes in OGRE, anyway.
sorry for this late reply, was busy with my own stuff.
these shaders, however, do look pretty damn good.
I'm going to look at the files this weekend, perhaps.
bcloward
Gnoblar
Posts: 2
Joined: Wed Feb 07, 2007 6:59 am
Contact:

Post by bcloward »

Buddy,

Kees Rijnen and I (the creators of ShaderFX) are interested in creating an export option in ShaderFX that will output FX shaders that are more friendly with your conversion tool and that will be easier to bring into Ogre. We'd like to try to help smooth out this process so that it will be easier for people to use our tool in Max and then get their shaders into Ogre. We would really like to hear any input you'd like to give us on the topic.

We haven't used Ogre before and we're not familiar with its material system, but we've had lots of people ask us if our shaders are compatible - so we'd like to add that feature. Any help you could give would be greatly appreciated!

-Ben
Ben Cloward
Technical Artist
Lumonix.com
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Post by sinbad »

buddy will be more up on the specifics here, but I'm always keen to see more tools targetting Ogre. Here are a couple of things to bear in mind:

- Ogre needs mul(matrix, vertex) and not mul(vertex, matrix) because of the way matrices are passed
- Using globals as uniforms is awkward because Ogre explicitly identifies inputs to shaders (and tracking where a global is used is tough). So try to use proper uniform inputs to shader entry points not globals.

In many ways OGRE materials have more features than FX, at least at the real-time integration end since we support material schemes, material LOD levels, specification of shadow caster and receiver shaders, more flexible pass iteration, material subclassing/extension, and a few other things. FX is just a little better at tool integration.
Post Reply