shiny - a shader and material management library [v 0.3]

A place to show off your latest screenshots and for people to comment on them. Only start a new thread here if you have some nice images to show off!
scrawl
OGRE Expert User
OGRE Expert User
Posts: 1119
Joined: Sat Jan 01, 2011 7:57 pm
x 2

Re: shiny - a shader and material management library for OGR

Post by scrawl » Sat Jul 28, 2012 11:57 pm

But, are you really sure that everything is supposed to work with just these lines? (Taken from the doxy page)
This is what the current version says (pretty much the same as your code)

Code: Select all

sh::Factory* factory = new sh::Factory(platform);
factory->setCurrentLanguage(sh::Language_GLSL);
factory->loadAllFiles();
Maybe you need to refresh your browser cache?

Besides, thanks a bunch for pointing out the issues with the docs. And no, you are not annoying or boring me in any way :lol:
0 x

Flayer
Gnoblar
Posts: 7
Joined: Mon May 21, 2012 4:20 pm

Re: shiny - a shader and material management library for OGR

Post by Flayer » Sun Jul 29, 2012 12:08 am

scrawl wrote:Maybe you need to refresh your browser cache?
Yeah, that was it. Thought it would have refreshed the cache as I closed the browser, but apparently not :P
Now I can see the current version so to speak =)
scrawl wrote: Besides, thanks a bunch for pointing out the issues with the docs.
I'm glad I could help =)

Anyways, I was taking a look into the materials from openmw and saw this in the moon.shader

Code: Select all

#define MRT @shGlobalSettingBool(mrt_output)
Is that used for deferred shading, or is it for something else?
Just curious as I have been trying to wrap my head around how to implement deferred shading to support tons of lights, but I might be going a little too fast for my own good here.
Have taken a good look at the example that comes with Ogre, but that demo is scaled differently and just scaling things did not just solve the problems with directional lights (or spotlights for that matter),
but I still have a lot to learn about shaders as it is with a forward renderer and shaders in general :P

Well, we'll see how it goes in the end =)
Thanks again
0 x

lilljohan
Halfling
Posts: 82
Joined: Tue Jan 03, 2006 4:05 pm
Location: Växjö, Sweden
Contact:

Re: shiny - a shader and material management library for OGR

Post by lilljohan » Tue Aug 28, 2012 2:02 pm

Hi,
I just wanted to let you know that there is a type error in the documentation at http://scrawl.bplaced.net/sh_doxy/defin ... aders.html

Code: Select all

shOutputColor(0) = shSample(diffuseMap, UV);
should be

Code: Select all

shOutputColour(0) = shSample(diffuseMap, UV);
Took me a while to figure out what it was complaining about =)

It would be nice if it could log an error if an included file (like "Core.h") is not found instead of just throwing an exception.

I would also like to thank you for your awesome work on this library.
0 x

User avatar
teodron
Halfling
Posts: 78
Joined: Fri Jul 22, 2011 11:49 am
Location: Bucharest, Romania

Re: shiny - a shader and material management library for OGR

Post by teodron » Wed Sep 12, 2012 3:42 pm

Hi,
This library sounds like the secret weapon many people needed. I tried to use it, but it appears that it needs to link statically against boost. Is this the case? If yes, what could I do to make it link against the shared library variant of boost?
0 x
T

scrawl
OGRE Expert User
OGRE Expert User
Posts: 1119
Joined: Sat Jan 01, 2011 7:57 pm
x 2

Re: shiny - a shader and material management library for OGR

Post by scrawl » Wed Sep 12, 2012 3:49 pm

You can link statically or dynamically. If you are linking dynamic version of wave, make sure to not compile the source in Preprocessor/* (which is basically an optimized, shipped version of wave).
0 x

User avatar
thebluefish
Gremlin
Posts: 170
Joined: Tue Apr 05, 2011 1:55 am

Re: shiny - a shader and material management library for OGR

Post by thebluefish » Thu Sep 13, 2012 12:19 am

teodron wrote:Hi,
This library sounds like the secret weapon many people needed. I tried to use it, but it appears that it needs to link statically against boost. Is this the case? If yes, what could I do to make it link against the shared library variant of boost?
You may get this if you don't build the library and your game/project with /MD (Multi-Threaded DLL) or /MDd (Multi-Threaded Debug DLL). Boost automatically generates #pragma commands to link the "correct" version of its libraries. If your project isn't specified as DLL, then boost tries to link statically instead of dynamically.

Great library though, I'm really loving it. Currently I'm keeping my project as Shaders [Off, Basic, Advanced] as an option. Do you have any way of specifying a configuration for a specific object rather than a viewport? Say I want to add the ability to turn off effects on my terrain but keeping my characters with full effects or vice-versa.
0 x
During the code inspection, a couple of minor points were noticed: -

Function inlining was critical to performance.
For MSVC, at least, a "delete 0" caused execution of 11 assembly instructions, including a function call. So in cases where performance is at an absolute premium it can be worth inserting the extra manual test.

scrawl
OGRE Expert User
OGRE Expert User
Posts: 1119
Joined: Sat Jan 01, 2011 7:57 pm
x 2

Re: shiny - a shader and material management library for OGR

Post by scrawl » Thu Sep 13, 2012 10:09 am

thebluefish wrote:Do you have any way of specifying a configuration for a specific object rather than a viewport? Say I want to add the ability to turn off effects on my terrain but keeping my characters with full effects or vice-versa.
Yes, that is what global settings are intended for.

Code: Select all

mFactory->setGlobalSetting("terrain_parallax", "true")

Code: Select all

#define TERRAIN_PARALLAX @shGlobalSettingBool(terrain_parallax)

#if TERRAIN_PARALLAX
...
#endif
0 x

User avatar
Tubulii
Gnoblar
Posts: 5
Joined: Tue Jul 19, 2011 11:52 am

Re: shiny - a shader and material management library for OGR

Post by Tubulii » Tue Sep 25, 2012 7:16 pm

Hi,

Is it possible to replace the microcode cache with a different approach to support ogre < 1.8 ?
Ogre 1.7 does not have a shader chache.
0 x

scrawl
OGRE Expert User
OGRE Expert User
Posts: 1119
Joined: Sat Jan 01, 2011 7:57 pm
x 2

Re: shiny - a shader and material management library for OGR

Post by scrawl » Tue Sep 25, 2012 7:59 pm

Just remove the offending code. It will work without.
0 x

scrawl
OGRE Expert User
OGRE Expert User
Posts: 1119
Joined: Sat Jan 01, 2011 7:57 pm
x 2

Re: shiny - a shader and material management library for OGR

Post by scrawl » Sun Sep 30, 2012 2:38 pm

I'm looking to add 2 more features. Feel free to add your comments or make suggestions.

Saving functionality

Saving functionality for materials. Realtime editing is already possible, so this is the last obstacle for seamlessly integrating a material editor into your game. The workflow could look like this:

- Write the shaders and base materials in a text editor.
- In the in-game material editor, start creating derived materials and tweaking them by overriding the properties of the base material.
- Once you're satisfied, hit the "save" button and enjoy the exported .mat file. Then repeat from step 2 on :)

Also nice to have would be a "Reload shaders" functionality, because it's kinda tedious to restart your game every time you've made a shader change. EDIT: After thinking about it again, this is already possible by just deleting and re-creating the factory :)

Caching improvements

You might notice the current caching functionality (which you can turn on, and is disabled by default) is a bit annoying - it requires you to delete the cache manually every time you change a shader. Of course, this could be "fixed" by not using the cache until the final release of your software is ready - but that's not really optimal. A nice solution would be to save a timestamp of the shader files' "Last modified" time and wipe the corresponding cache if that timestamp changes. :)

As a side note, we are currently integrating this library into StuntRally - stay tuned, I will post more when it's done.
0 x

scrawl
OGRE Expert User
OGRE Expert User
Posts: 1119
Joined: Sat Jan 01, 2011 7:57 pm
x 2

Re: shiny - a shader and material management library for OGR

Post by scrawl » Tue Oct 30, 2012 1:19 pm

As promised, we integrated this library into StuntRally 1.8. The main advantage was better compatibility due to providing a GLSL version, as well as getting rid of hardcoded shaders which made editing tedious.
The materials can also be used as references for:
- Integration with paged-geometry (wind effect, grass fading)
- Triplanar texturing for Ogre::Terrain
- Soft particles
- Multi render target output for SSAO, DoF, etc
Last edited by scrawl on Sun Jan 13, 2013 7:45 am, edited 1 time in total.
0 x

BTolputt
Greenskin
Posts: 121
Joined: Thu Feb 18, 2010 8:05 am

Re: shiny - a shader and material management library for OGR

Post by BTolputt » Thu Nov 01, 2012 10:39 am

That's awesome scrawl!
0 x

noorus
Halfling
Posts: 75
Joined: Wed Apr 20, 2011 9:55 pm
Location: Helsinki, Finland

Re: shiny - a shader and material management library for OGR

Post by noorus » Thu Nov 08, 2012 9:07 pm

Can I suggest prefixing the header file names with "sh" or something?
The names are so generic that they might collide with other libraries or people's projects rather easily.
0 x
Creator of Nice Input Library, for your advanced input needs.
Image

bstone
OGRE Expert User
OGRE Expert User
Posts: 1920
Joined: Sun Feb 19, 2012 9:24 pm
Location: Russia

Re: shiny - a shader and material management library for OGR

Post by bstone » Fri Nov 09, 2012 5:14 am

Just put them in their own folder. It's better that way because you can't persuade every author to rename his library files and you'll be working with lots of libs eventually.
0 x

noorus
Halfling
Posts: 75
Joined: Wed Apr 20, 2011 9:55 pm
Location: Helsinki, Finland

Re: shiny - a shader and material management library for OGR

Post by noorus » Fri Nov 09, 2012 10:20 am

bstone wrote:Just put them in their own folder. It's better that way because you can't persuade every author to rename his library files and you'll be working with lots of libs eventually.
That would work fine, except the shiny headers themselves refer to each other without a path. So I'd have to add the same path prefix to shiny's header-to-header includes, which isn't a nice thing to maintain. Or am I missing something..?
0 x
Creator of Nice Input Library, for your advanced input needs.
Image

scrawl
OGRE Expert User
OGRE Expert User
Posts: 1119
Joined: Sat Jan 01, 2011 7:57 pm
x 2

Re: shiny - a shader and material management library for OGR

Post by scrawl » Fri Nov 09, 2012 2:29 pm

When you're compiling it as a static or dynamic lib you don't need any other includes besides boost, so that shouldnt be a problem.
0 x

bstone
OGRE Expert User
OGRE Expert User
Posts: 1920
Joined: Sun Feb 19, 2012 9:24 pm
Location: Russia

Re: shiny - a shader and material management library for OGR

Post by bstone » Fri Nov 09, 2012 2:42 pm

The standard search order for the header files is to look for includes in the same folder so as long as you keep all shiny headers in their own folder they should work fine. You will reference them in your code as #include "shiny/xyz.h" for e.g. and they should get their includes from there. External dependencies might be a problem but they are usually easy to work around using linked folders. Prefixed file names are one of the very few things I don't like in Ogre - makes my life slower b/c of the extra typing :)
0 x

efaj
Gnoblar
Posts: 21
Joined: Tue Dec 27, 2011 2:00 pm

Re: shiny - a shader and material management library for OGR

Post by efaj » Fri Mar 08, 2013 1:50 pm

Shiny needs 1 absolute path on construction, but in my project I use "res/creatures/crit1" and so on, and "res/materials" for common shaders. How could this problem be approached with shiny?

To get this working in Ogre, I use:

Code: Select all

tempVector=cf->getMultiSetting("MatPath",secName);
			for (unsigned int ii = 0; ii < tempVector.size(); ++ii) {
				boost::filesystem::path materialFilename=tempVector.at(ii);
				if(!Ogre::MaterialManager::getSingleton().resourceExists(materialFilename.stem().string())){
					//Load Material
					Ogre::DataStreamPtr dataStream =Ogre::ResourceGroupManager::getSingleton().openResource(
							(indexPath.parent_path()/materialFilename).string(), groupName );
					Ogre::MaterialManager::getSingleton().parseScript(dataStream, groupName);
				}
			}
Which manually loads each material before use (cf is a configfile that represents a complete asset, with MatPath representing the Path to the Material, so I load it before the mesh is loaded).
Which would be the shiny way?

My main reason to attempt to use shiny is that I'm trying to enable/disable some shaders at whim, be it the artist, the end user, or me. For example, I use a Rim Lightning shader, but, this effect may not look appropriate in a cave area, so I want to enable it/add it depending on which stage you are. RTSS doesn't do this, right? (since I haven't found how to load a .cg with RTSS or something like that, just functions that seem to cover only basic material stuff) And shiny can do this, right?
0 x

scrawl
OGRE Expert User
OGRE Expert User
Posts: 1119
Joined: Sat Jan 01, 2011 7:57 pm
x 2

Re: shiny - a shader and material management library for OGR

Post by scrawl » Fri Mar 08, 2013 2:35 pm

shiny doesn't use Ogre's resource system at this point. It would be nice if it would (even though I personally don't need it).
Another thing it doesn't support is on the fly loading from disk, which I personally haven't needed either because loading materials is extremely fast and shaders are compiled on demand anyway.
So yeah, both of this should be relatively easy to add and I'll gladly accept patches.
0 x

efaj
Gnoblar
Posts: 21
Joined: Tue Dec 27, 2011 2:00 pm

Re: shiny - a shader and material management library for OGR

Post by efaj » Fri Mar 08, 2013 10:06 pm

scrawl wrote:shiny doesn't use Ogre's resource system at this point. It would be nice if it would (even though I personally don't need it).
Another thing it doesn't support is on the fly loading from disk, which I personally haven't needed either because loading materials is extremely fast and shaders are compiled on demand anyway.
So yeah, both of this should be relatively easy to add and I'll gladly accept patches.
Personally, now, after much trial and error, I've learned how the system works, and I, too, don't see the need for either :) I can keep my file structure by adding the directories to the name of stuff (source materials/rimLightning/RimLight.shader), and realized that "#include" is relative to the Platform's path (which I can settle to use "res", so I'll be using: #include "materials/core.h")

Though, I think it would be better if the loading of files in both cases was relative to the file-being-parsed's path, this is something so little important and arguable, that I don't care.

Kudos for the library. IMO, this is something that could even be part of Ogre.
0 x

scrawl
OGRE Expert User
OGRE Expert User
Posts: 1119
Joined: Sat Jan 01, 2011 7:57 pm
x 2

Re: shiny - a shader and material management library for OGR

Post by scrawl » Sat Mar 09, 2013 12:00 am

efaj wrote: Though, I think it would be better if the loading of files in both cases was relative to the file-being-parsed's path
Agreed, and fix commited.
0 x

bstone
OGRE Expert User
OGRE Expert User
Posts: 1920
Joined: Sun Feb 19, 2012 9:24 pm
Location: Russia

Re: shiny - a shader and material management library for OGR

Post by bstone » Sat Mar 09, 2013 7:34 am

efaj wrote:Kudos for the library. IMO, this is something that could even be part of Ogre.
I agree, that would be cool. But it works the other way too. It's funny though that shiny gets little attention overall, but I guess the problem is that not many Ogre users reach the level where you absolutely need something like shiny to move forward. Or maybe RTSS is all hot these days? :D
0 x

User avatar
Zonder
Ogre Magi
Posts: 1133
Joined: Mon Aug 04, 2008 7:51 pm
Location: Manchester - England
x 22

Re: shiny - a shader and material management library for OGR

Post by Zonder » Sat Mar 09, 2013 11:05 am

bstone wrote:
efaj wrote:Kudos for the library. IMO, this is something that could even be part of Ogre.
I agree, that would be cool. But it works the other way too. It's funny though that shiny gets little attention overall, but I guess the problem is that not many Ogre users reach the level where you absolutely need something like shiny to move forward. Or maybe RTSS is all hot these days? :D
As soon as I read about it I added into my important library list :)
0 x
There are 10 types of people in the world: Those who understand binary, and those who don't...

scrawl
OGRE Expert User
OGRE Expert User
Posts: 1119
Joined: Sat Jan 01, 2011 7:57 pm
x 2

Re: shiny - a shader and material management library for OGR

Post by scrawl » Sat Mar 09, 2013 3:37 pm

It's been 8 months since I tagged the 0.1 "version" and a lot has happened so I figured I'd call it 0.2 now. Everyone's probably just using the trunk anyway but it's nice to have some kind of changelog besides the commit logs:
- Material LOD support.
- Shader caching now detects changed shader files.
- A few macros added to core.h.
- Lots of misc fixes.
0 x

scrawl
OGRE Expert User
OGRE Expert User
Posts: 1119
Joined: Sat Jan 01, 2011 7:57 pm
x 2

Re: shiny - a shader and material management library [v 0.2]

Post by scrawl » Sun Mar 17, 2013 8:38 pm

A new convenience macro has been added.

Previously, when using an optional texture unit, one would need a property for the texture itself, plus a property to enable / disable the texture unit.
This huge redundancy can now be avoided with the new macro, which checks if a property name (in this case the texture name) is empty. It also integrates with the permutation handler.

Also, I'm currently in the planning stage of adding a realtime material editor in QT. It won't be a standalone application, but instead will be available from a simple 1 line call in the user's application, which will open an editing window. That way, all editing will have an immediate visual effect on the game window without a restart.
0 x

Post Reply