Advanced compilers for Ogre's scripts

Threads related to Google Summer of Code
Post Reply
User avatar
Praetor
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3335
Joined: Tue Jun 21, 2005 8:26 pm
Location: Rochester, New York, US
x 3
Contact:

Post by Praetor »

That looks a change in the overall syntax. Function call syntax like that would be quite a change. But you can achieve the same effect in other ways. If you wanted to handle the "emission_rate" node yourself, you could allow for extra parameters in the header. Just drop the parentheses like this:

Code: Select all

emission_rate cameraDependency 40
{
    ...
}
In language like lua i believe function calls don't even require the parentheses.

Now, do you want this script to actually call the function cameraDependency in C/C++ and put in the value of 40? Or do you want some sort of function concept in the scripting language itself?
Game Development, Engine Development, Porting
http://www.darkwindmedia.com
User avatar
Kencho
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 4011
Joined: Fri Sep 19, 2003 6:28 pm
Location: Burgos, Spain
x 2
Contact:

Post by Kencho »

I don't think it's possible to do a multipurpose script programming language compiler in this time frame. If it has enough strength, would be a nice addition for the post-SoC period, but not now in my opinion.

PS: Neither Lua or Python that I know. But Qbasic did :P
Image
User avatar
Praetor
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3335
Joined: Tue Jun 21, 2005 8:26 pm
Location: Rochester, New York, US
x 3
Contact:

Post by Praetor »

yeah if it was the first case (call something in C++ from a custom handler) that is quite possible, but case 2 is really out of scope of not just this summer, but I think the scope of Ogre's scripting language in general. It really seems like it should be a declarative language, not a functional or procedural one.
Game Development, Engine Development, Porting
http://www.darkwindmedia.com
User avatar
spookyboo
Silver Sponsor
Silver Sponsor
Posts: 1141
Joined: Tue Jul 06, 2004 5:57 am
x 151
Contact:

Post by spookyboo »

I have 2 'entities', emission_rate and cameraDependency. The cameraDependency changes the emission rate, but it may also change other entities. So, cameraDependency is not really 'part' of the emission_rate. Fow now I solved it with the syntax:

Code: Select all

emission_rate 40 << cameraDependency
{
   ...
}
This requires some extra code in my current system. I was just wondering how flexible your system is.
User avatar
Praetor
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3335
Joined: Tue Jun 21, 2005 8:26 pm
Location: Rochester, New York, US
x 3
Contact:

Post by Praetor »

I'm afraid I still don't fully understand. What exactly is cameraDependency? Can you show me what kind of thing it is? Is it another production in the script somewhere, and if so, how does "passing in" 40 affect it?
Game Development, Engine Development, Porting
http://www.darkwindmedia.com
User avatar
spookyboo
Silver Sponsor
Silver Sponsor
Posts: 1141
Joined: Tue Jul 06, 2004 5:57 am
x 151
Contact:

Post by spookyboo »

Normally the emission rate is constant (40), but you can also make it dependent of the mutual distance between the Particle System and the camera. If the camera is near, the emission rate is 40. If the camera moves away, the emission rate decreases at a certain rate (to be specified within the brackets). You can do the same with the particles´ dimensions. If the camera is near, the dimensions decrease. If the camera moves away, the dimensions will get their normal width, height and depth. This is to prevent that particles that are in front of the camera are huge (for instance raindrops in front of the camera look gigantic with the current particle plugin).
User avatar
Praetor
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3335
Joined: Tue Jun 21, 2005 8:26 pm
Location: Rochester, New York, US
x 3
Contact:

Post by Praetor »

I'm sorry, I'm looking for information about the reference to it in the script. It sounds like it is just another production in the script. A sort of property of the emission rate. You can custom handle the emission_rate token and allow it any properties you want. You can even use "cameraDependency(40)" as the syntax. The parser just isn't going to do any of the extra work for you. It will pass cameraDependency(40) as a single token that you can then figure out what to do with any way you like. You "<< 40" syntax may be even better, since it'll be tolerant of newlines and spaces. In that case the parser will give you cameraDependency << and 40 all as separate tokens one after the other. What you do with them is up to you.
Game Development, Engine Development, Porting
http://www.darkwindmedia.com
User avatar
Kencho
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 4011
Joined: Fri Sep 19, 2003 6:28 pm
Location: Burgos, Spain
x 2
Contact:

Post by Kencho »

Spooky, as I understand it, what you say is that the emission rate and particle size, are directly bound to the camera distance. I think you might do things such as, through a listener, pick the emission_rate and size nodes as they are parsed, and pick some extra parameters to apply the changes yourself. For instance, adding this line

Code: Select all

emission_rate 40 camera_distance_bound 1.5
might mean that, when the camera is at 1 unit of distance, the emission rate is 40, but decreases exponentially with distance with a factor of 1.5. Is that fine for your purposes? I don't really see practical to officially extend Ogre's scripts without the core Ogre adding the changes. However, the flexibility of this compiler really allows some neat customisation :)
Image
User avatar
spookyboo
Silver Sponsor
Silver Sponsor
Posts: 1141
Joined: Tue Jul 06, 2004 5:57 am
x 151
Contact:

Post by spookyboo »

Kencho. you are right, that is how it works. I know how to do this, but its more that I'm struggling with a script syntax that I'm comfortable with. I will stick with the << solution for now and take a look at Praetors' compiler code when it is more or less 'finished'.
User avatar
Praetor
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3335
Joined: Tue Jun 21, 2005 8:26 pm
Location: Rochester, New York, US
x 3
Contact:

Post by Praetor »

At some point this month I'll try to work up a tutorial/example of customizing the compilers. Once I post it you should be able to check out if it will work for you and see how to do it. I think it will be able to accomplish what you want.
Game Development, Engine Development, Porting
http://www.darkwindmedia.com
User avatar
Kencho
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 4011
Joined: Fri Sep 19, 2003 6:28 pm
Location: Burgos, Spain
x 2
Contact:

Post by Kencho »

Praetor wrote:At some point this month I'll try to work up a tutorial/example of customizing the compilers.
Very good idea. One of the most important features of Ogre in my opinion is that it's intensively documented and there's plenty of info about how it works and how to use it.
Image
User avatar
Praetor
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3335
Joined: Tue Jun 21, 2005 8:26 pm
Location: Rochester, New York, US
x 3
Contact:

Post by Praetor »

Update time.

I've actually already started the material compiler. I just wanted to clarify why I was expecting it to take longer. Particle system use a setParameter interface which allows for a fairly generic compilation procedure. With the material compiler I will be explicitly handling every property. So, it will be tedious going, but not too difficult to do.

After that I will make a material demo using the new compiler just like i did with the particle compiler. I will also create a separate demo showcasing the use of the compiler listeners to both receive feedback and modify and add behavior to the compilers.
Game Development, Engine Development, Porting
http://www.darkwindmedia.com
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Post by sinbad »

Great stuff. And I apologise, I've been adding more stuff to the material script in the last few weeks than I have in ages ;) But as we agreed before that can be caught up on when merge time arrives.
User avatar
Praetor
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3335
Joined: Tue Jun 21, 2005 8:26 pm
Location: Rochester, New York, US
x 3
Contact:

Post by Praetor »

I've been keeping up-to-date with HEAD, so I'll cross-reference the current compiler and i should be able to pick up all the additions.
Game Development, Engine Development, Porting
http://www.darkwindmedia.com
User avatar
Praetor
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3335
Joined: Tue Jun 21, 2005 8:26 pm
Location: Rochester, New York, US
x 3
Contact:

Post by Praetor »

It's been a bit since I updated.

I'm plugging away at the material compiler. I'm into the bulkiest part: the pass. Once through is the second bulkiest: the texture_unit. Then I'll do the relatively light gpu programs. I still need to set up a local test demo like I did for the particle compiler to get the kinks out of it, but overall the work is easy but tedious. Seeing all the linear iterator manipulation and string comparisons I'm starting to worry about performance, but I can't do anything about that right now. I'll have to profile it when I at least partially finish the material compiler.

I'm starting to think that August will get very hectic for me. I've made a lot of commitments that are going to kick in around then. Because of this I've decided to accelerate development during July, instead of waiting to try scrounging up time in August. I want to finish at least the material and compositor compilers and have time to document and demo them before SoC ends.

I'll check in the compiler tonight so you can look over it. Nothing special there, except maybe the material compiler's listener has more methods than the particle's. You will be able to control how it allocates the material, the gpu programs, the textures, hook into the texture aliases before they are applied, as well as customize individual node compilation.
Game Development, Engine Development, Porting
http://www.darkwindmedia.com
User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US
x 22

Post by xavier »

Ya, and when Sinbad is done messing with your stuff, I have an idea for generating GPU programs on the fly from template code that will probably build off this project. ;)
Do you need help? What have you tried?

Image

Angels can fly because they take themselves lightly.
User avatar
Praetor
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3335
Joined: Tue Jun 21, 2005 8:26 pm
Location: Rochester, New York, US
x 3
Contact:

Post by Praetor »

Like Sh? That kind of generating? I've always wanted that, but I feel like sh has just been lagging for a long time. That would be an awesome project!
Game Development, Engine Development, Porting
http://www.darkwindmedia.com
User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US
x 22

Post by xavier »

Haven't really looked into Sh much -- if it fits then maybe some integration with it. Either way the idea is to eliminate the combinatorial management nightmare.
Do you need help? What have you tried?

Image

Angels can fly because they take themselves lightly.
User avatar
Praetor
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3335
Joined: Tue Jun 21, 2005 8:26 pm
Location: Rochester, New York, US
x 3
Contact:

Post by Praetor »

Yep, that's what Sh is good for. You can use actual C++ to generate your shaders. It exactly eliminates (or at least mitigates) the combinatorial explosion of shaders. As I said, last time I looked Sh seemed great for odd "showcase" shader, but didn't seem ready for real primetime usage. An integration might be a good idea (if it's ready), maybe with a few xavier touch-ups and additions.
Game Development, Engine Development, Porting
http://www.darkwindmedia.com
User avatar
jacmoe
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 20570
Joined: Thu Jan 22, 2004 10:13 am
Location: Denmark
x 179
Contact:

Post by jacmoe »

xavier wrote:Haven't really looked into Sh much -- if it fits then maybe some integration with it. Either way the idea is to eliminate the combinatorial management nightmare.
:wumpus: did some time ago - even had a dialog with the author.
I believe I downloaded an early demo of OgreSH.
/* Less noise. More signal. */
Ogitor Scenebuilder - powered by Ogre, presented by Qt, fueled by Passion.
OgreAddons - the Ogre code suppository.
User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US
x 22

Post by xavier »

According to a post on the Sh website from August 2006:
At this point we are not actively continuing the development of Sh, as it is our responsibility to focus on building our commercial products.
Plus, whereas Sh is intended for general-purpose GPU programming, I'd focus instead on the needs of Ogre and real-time rendering GPU programs, specifically working a generic framework into the Ogre material system.
Do you need help? What have you tried?

Image

Angels can fly because they take themselves lightly.
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Post by sinbad »

I recently heard of Mental Mill, from the guys that do Mental Ray. Looks rather interesting as a way to bridge the offline / real-time graphics. A cut-down version of it went into FX Composer 2, which is now looking like the 'new' standard for editing shaders.
User avatar
Praetor
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3335
Joined: Tue Jun 21, 2005 8:26 pm
Location: Rochester, New York, US
x 3
Contact:

Post by Praetor »

They mention generating and manipulating ASTs of the shaders, which is exactly what I figured a system like this is used for. But they're product seems to be more about what they built on top of it, not the underneath. I take we are all talking about something done in-code, like a template meta-programming framework. Either way, I think it is a good idea xavier. Too bad about sh, but the abstract syntax tree manipulation seems to be a good place to start.
Game Development, Engine Development, Porting
http://www.darkwindmedia.com
User avatar
jacmoe
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 20570
Joined: Thu Jan 22, 2004 10:13 am
Location: Denmark
x 179
Contact:

Post by jacmoe »

Are we talking about something like CUDA ?
I thought about SH the moment I heard of it.
/* Less noise. More signal. */
Ogitor Scenebuilder - powered by Ogre, presented by Qt, fueled by Passion.
OgreAddons - the Ogre code suppository.
User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US
x 22

Post by xavier »

CUDA, apart from being nVidia-only, is again a GPGPU-targeted system -- it makes more sense to me in this case, to create something specific to how Ogre handles its materials, in a way that prevents combinatorial explosion of shader management, while maintaining flexibility and GPU code reuse. Given what I understand about GPGPU systems, in this case using CUDA or Sh here would be analogous to using full SGML when XML will do fine.

This really should be broken off into another thread, I think -- the only reason I brought it up in this thread in the first place was that it would be integrated directly into the material compiler (at least the way I understand it at this point).
Do you need help? What have you tried?

Image

Angels can fly because they take themselves lightly.
Post Reply