Hlms + JSON now in 2.1

Discussion area about developing with Ogre-Next (2.1, 2.2 and beyond)


User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5446
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1348

Hlms + JSON now in 2.1

Post by dark_sylinc »

I've just updated OgreDeps to include rapidjson.

I also updated Ogre to use a CMake option OGRE_CONFIG_ENABLE_JSON. It was supposed to be mandatory but after much meditation I decided to make it optional.

What does it do?

Well, that's the juicy part: When enabled, Ogre will look for *.material.json files containing JSON definition of our favourite Hlms materials. The code is easier to maintain, the format is human readable and editable, and there's a lot of tools for handling JSON.
It supports all the features/options that PBS materials can handle, which is something the old implementation was lagging behind (it was becoming increasingly difficult to handle all the features).
The main goal of this system is to allow material editors to have something to work with that can be loaded and saved easily, while quick hand-edits with notepad are still possible.

This is an example of a valid json material:

Code: Select all

{
    "samplers" :
	{
        "unique_name" :
        {
            "min" : "linear",
            "mag" : "linear",
            "mip" : "linear",
            "u" : "wrap",
            "v" : "wrap",
            "w" : "wrap",
            "miplodbias" : 0,
            "max_anisotropic" : 1,
            "compare_function" : "disabled",
            "border" : [1, 1, 1, 1],
            "min_lod" : -3.40282347E+38,
            "max_lod" : 3.40282347E+38
        }
    },

    "macroblocks" :
    {
        "unique_name" :
        {
            "scissor_test" : false,
            "depth_check" : true,
            "depth_write" : true,
            "depth_function" : "less_equal",
            "depth_bias_constant" : 0,
            "depth_bias_slope_scale" : 0,
            "cull_mode" : "clockwise",
            "polygon_mode" : "solid"
        }
    },

    "blendblocks" :
    {
        "unique_name" :
        {
            "alpha_to_coverage" : false,
            "blendmask" : "rgba",
            "separate_blend" : false,
            "src_blend_factor" : "one",
            "dst_blend_factor" : "zero",
            "src_alpha_blend_factor" : "one",
            "dst_alpha_blend_factor" : "zero",
            "blend_operation" : "add",
            "blend_operation_alpha" : "add"
        }
    },

    "pbs" :
    {
        "material_name" :
        {
            "macroblock" : "unique_name",
            "blendblock" : "unique_name",
            "alpha_test" : ["disabled", 0.5],
            "shadow_const_bias" : 0.01,

            "workflow" : "specular_fresnel",

            "transparency" :
            {
                "value" : 1.0,
                "mode" : "None",
                "use_alpha_from_textures" : true
            },

            "diffuse" :
            {
                "value" : [1, 1, 1],
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "specular" :
            {
                "value" : [1, 1, 1]
            },

            "roughness" :
            {
                "value" : 1,
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "fresnel" :
            {
                "mode" : "coeff",
                "value" : [1, 1, 1],
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "normal" :
            {
                "value" : 1,
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "detail_weight" :
            {
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "detail_diffuse0" :
            {
                "mode" : "NormalNonPremul",
                "offset" : [0, 0],
                "scale" : [1, 1],
                "value" : 1,
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "detail_normal0" :
            {
                "offset" : [0, 0],
                "scale" : [1, 1],
                "value" : 1,
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "reflection" :
            {
                "texture" : "cubemap.png",
                "sampler" : "unique_name"
            }
        }
    }
}
First, a series of macro-, blend- and samplerblocks are defined, and a name is assigned to them.
Then these names can be referenced from the pbs materials.

For reference, these is an example of all available options. Note it is not valid JSON, nor a valid material (i.e. you shouldn't define a metallic texture when the workflow isn't metallic, it will assert in debug mode):

Code: Select all

{
    "samplers" :
	{
        "unique_name" :
        {
            "min" : "point" "linear" "anisotropic",
            "mag" : "point" "linear" "anisotropic",
            "mip" : "none" "point" "linear" "anisotropic",
            "u" : "wrap" "mirror" "clamp" "border",
            "v" : "wrap" "mirror" "clamp" "border",
            "w" : "wrap" "mirror" "clamp" "border",
            "miplodbias" : 0,
            "max_anisotropic" : 1,
            "compare_function" : "less" "less_equal" "equal" "not_equal" "greater_equal" "greater" "never" "always" "disabled",
            "border" : [1, 1, 1, 1],
            "min_lod" : -3.40282347E+38,
            "max_lod" : 3.40282347E+38
        }
    },

    "macroblocks" :
    {
        "unique_name" :
        {
            "scissor_test" : false,
            "depth_check" : true,
            "depth_write" : true,
            "depth_function" : "less" "less_equal" "equal" "not_equal" "greater_equal" "greater" "never" "always",
            "depth_bias_constant" : 0,
            "depth_bias_slope_scale" : 0,
            "cull_mode" : "none" "clockwise" "anticlockwise",
            "polygon_mode" : "points" "wireframe" "solid"
        }
    },

    "blendblocks" :
    {
        "unique_name" :
        {
            "alpha_to_coverage" : false,
            "blendmask" : "rgba",
            "separate_blend" : true,
            "src_blend_factor" : "one" "zero" "dst_colour" "src_colour" "one_minus_dst_colour" "one_minus_src_colour" "dst_alpha" "src_alpha" "one_minus_dst_alpha" "one_minus_src_alpha",
            "dst_blend_factor" : "one" "zero" "dst_colour" "src_colour" "one_minus_dst_colour" "one_minus_src_colour" "dst_alpha" "src_alpha" "one_minus_dst_alpha" "one_minus_src_alpha",
            "src_alpha_blend_factor" : "one" "zero" "dst_colour" "src_colour" "one_minus_dst_colour" "one_minus_src_colour" "dst_alpha" "src_alpha" "one_minus_dst_alpha" "one_minus_src_alpha",
            "dst_alpha_blend_factor" : "one" "zero" "dst_colour" "src_colour" "one_minus_dst_colour" "one_minus_src_colour" "dst_alpha" "src_alpha" "one_minus_dst_alpha" "one_minus_src_alpha",
            "blend_operation" : "add" "subtract" "reverse_subtract" "min" "max",
            "blend_operation_alpha" : "add" "subtract" "reverse_subtract" "min" "max"
        }
    },

    "pbs" :
    {
        "material_name" :
        {
            "macroblock" : "unique_name" ["unique_name", "unique_name_for_shadows"],
            "blendblock" : "unique_name" ["unique_name", "unique_name_for_shadows"],
            "alpha_test" : ["less" "less_equal" "equal" "not_equal" "greater_equal" "greater" "never" "always" "disabled", 0.5],
            "shadow_const_bias" : 0.01,

            "workflow" : "specular_ogre" "specular_fresnel" "metallic",

            "transparency" :
            {
                "value" : 1.0,
                "mode" : "None" "Transparent" "Fade",
                "use_alpha_from_textures" : true
            },

            "diffuse" :
            {
                "value" : [1, 1, 1],
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "specular" :
            {
                "value" : [1, 1, 1],
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "roughness" :
            {
                "value" : 1,
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "fresnel" :
            {
                "mode" : "coeff" "ior" "coloured" "coloured_ior",
                "value" : [1, 1, 1],
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "metallness" :
            {
                "value" : 1,
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "normal" :
            {
                "value" : 1,
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "detail_weight" :
            {
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "detail_diffuse0" :
            {
                "mode" : "NormalNonPremul" "NormalPremul" "Add" "Subtract" "Multiply" "Multiply2x" "Screen" "Overlay" "Lighten" "Darken" "GrainExtract" "GrainMerge" "Difference",
                "offset" : [0, 0],
                "scale" : [1, 1],
                "value" : 1,
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "detail_normal0" :
            {
                "offset" : [0, 0],
                "scale" : [1, 1],
                "value" : 1,
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "reflection" :
            {
                "texture" : "cubemap.png",
                "sampler" : "unique_name"
            }
        }
    }
}
At the moment, only PBS materials support JSON. Soon Unlit will follow. I also plan in writing a function that will allow exporting all existing datablocks to JSON.
Also note when an option isn't provided, a default is used.

There could be one or two bugs to iron out.
What about the old scripts? I don't plan on removing them, but I don't plan on improving it either. It was not scaling well.
dermont
Bugbear
Posts: 812
Joined: Thu Dec 09, 2004 2:51 am
x 42

Re: Hlms + JSON now in 2.1

Post by dermont »

The FindRapidjson.cmake file appears to be missing.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5446
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1348

Re: Hlms + JSON now in 2.1

Post by dark_sylinc »

Thanks. Fixed.
dermont
Bugbear
Posts: 812
Joined: Thu Dec 09, 2004 2:51 am
x 42

Re: Hlms + JSON now in 2.1

Post by dermont »

Thanks, now a problem compiling, OgreLwConstString.h appears to be missing?

Code: Select all

/media/sdb5/Libraries/OGRE/src/v2-1/OgreMain/include/OgreHlmsJson.h:36:31: fatal error: OgreLwConstString.h: No such file or directory
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5446
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1348

Re: Hlms + JSON now in 2.1

Post by dark_sylinc »

Ouch. Sorry about that. Fixed again.
Transporter
Minaton
Posts: 933
Joined: Mon Mar 05, 2012 11:37 am
Location: Germany
x 110

Re: Hlms + JSON now in 2.1

Post by Transporter »

Just to complete the information: You can find rapidjson under http://rapidjson.org/ (code: https://github.com/miloyip/rapidjson/)
al2950
OGRE Expert User
OGRE Expert User
Posts: 1227
Joined: Thu Dec 11, 2008 7:56 pm
Location: Bristol, UK
x 157

Re: Hlms + JSON now in 2.1

Post by al2950 »

Amazing, so much easier to use for both 'users' & 'developers' :). Can we add json support for the compositors as well?

I dont think there is much need to support the old style, to that point I think it should be removed otherwise it will confuse new starters. However we should also try and standardise other areas, ie compositors, low lever materials & particles. The old system was good and designed when things like Json did not exist, so I think this would be a good time to make a transition to a new script system.

Creating parsers for the different areas wont be difficult but will require a bit of time, to that end I suggest you pass the responsibility to the rest of the community, I would be happy do a lot of it.

One last thing, regarding HLMS materials. Would it be a good idea to allow a dev to set default sampler, blend block values? IE Allow a dev to set anisotropic filtering by default? Or would this cause some potential undesired issues with a HLMS assuming specific default values.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5446
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1348

Re: Hlms + JSON now in 2.1

Post by dark_sylinc »

al2950 wrote:Amazing, so much easier to use for both 'users' & 'developers' :). Can we add json support for the compositors as well?
Haha, Let's not get carried away! shall we? (new shiny object syndrome)

I had the same thought, however JSON is not a magic bullet. It has cons:

Pros:
  • Standardized syntax with lots of tools, online and offline.
  • Easy to maintain and extend from C++
  • Great for containing information generated by a machine (like an editor) in a human readable format
Cons:
  • Noisy syntax (when compared to Ogre's AST). {Brackets} "quotation marks" Double points : and commas , everywhere.
  • Very sensible to syntax error. For example the following is invalid JSON:

    Code: Select all

    {
     "hello" : "world",
     "hello again" : "from JSON",
    }
    What's the error? there is an extra comma after fromJSON. It needs to be removed. When you're writing an entire JSON from scratch by hand, these subtleties can drive you crazy (and right now we don't have any Compositor node editor).
  • No comments in JSON, though you can add an extra entry that will be ignored by the parser and use it as an entry.
I dont think there is much need to support the old style, to that point I think it should be removed otherwise it will confuse new starters.
Point taken, I agree. Let's see what the other 2.1 adopters think about it.
One last thing, regarding HLMS materials. Would it be a good idea to allow a dev to set default sampler, blend block values? IE Allow a dev to set anisotropic filtering by default?
This would add unnecessary complexity C++ side that can easily be solved via this new JSON. The following example creates four PBS materials with a diffuse texture and anisotropic filtering:

Code: Select all

{
    "samplers" :
   {
        "DefaultAniso" :
        {
            "min" : "anisotropic",
            "mag" : "anisotropic",
            "mip" : "linear",
            "max_anisotropic" : 16
        }
    },

    "pbs" :
    {
        "material_name" :
        {
            "diffuse" :
            {
                "texture" : "texture.png",
                "sampler" : "DefaultAniso"
            }
        },
        "mySecondMaterial" :
        {
            "diffuse" :
            {
                "texture" : "texture.png",
                "sampler" : "DefaultAniso"
            }
        },
        "myThirdMaterial" :
        {
            "diffuse" :
            {
                "texture" : "texture.png",
                "sampler" : "DefaultAniso"
            }
        },
        "myFourthMaterial" :
        {
            "diffuse" :
            {
                "texture" : "texture.png",
                "sampler" : "DefaultAniso"
            }
        }
    }
}
"DefaultAniso" will take the default parameters of a samplerblock and we only override the ones we're interested. Then we reference it in the material. I believe there is no need for anything fancier than that.

Sidenote: There is a plan to create HlmsManager::overrideAll*Blocks to change all blocks to a given setting at once (i.e. if the samplerblock says use bilinear filtering but you override to anisotropic, it will use anisotropic despite that the block still says "FO_LINEAR"). The only blocks that will not be overridden will be the ones that have their BasicBlock::mAllowGlobalDefaults member set to 0.
xrgo
OGRE Expert User
OGRE Expert User
Posts: 1148
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 169

Re: Hlms + JSON now in 2.1

Post by xrgo »

dark_sylinc wrote:
I dont think there is much need to support the old style, to that point I think it should be removed otherwise it will confuse new starters.
Point taken, I agree. Let's see what the other 2.1 adopters think about it.
I only use C++ for everything so I am not planning on using JSON, but it definitely looks more powerful than the old .materials
dark_sylinc wrote: Sidenote: There is a plan to create HlmsManager::overrideAll*Blocks to change all blocks to a given setting at once (i.e. if the samplerblock says use bilinear filtering but you override to anisotropic, it will use anisotropic despite that the block still says "FO_LINEAR"). The only blocks that will not be overridden will be the ones that have their BasicBlock::mAllowGlobalDefaults member set to 0.
I like, I have my own "overrideAll*Blocks" for toggling Wireframe/solid that I use all the time :P
User avatar
Thyrion
Goblin
Posts: 224
Joined: Wed Jul 31, 2013 1:58 pm
Location: germany
x 8

Re: Hlms + JSON now in 2.1

Post by Thyrion »

dark_sylinc wrote:For reference, these is an example of all available options. Note it is not valid JSON, nor a valid material (i.e. you shouldn't define a metallic texture when the workflow isn't metallic, it will assert in debug mode):

Code: Select all

{
    "samplers" :
	{
        "unique_name" :
        {
            "min" : "point" "linear" "anisotropic",
            "mag" : "point" "linear" "anisotropic",
            "mip" : "none" "point" "linear" "anisotropic",
            "u" : "wrap" "mirror" "clamp" "border",
            "v" : "wrap" "mirror" "clamp" "border",
            "w" : "wrap" "mirror" "clamp" "border",
            "miplodbias" : 0,
            "max_anisotropic" : 1,
            "compare_function" : "less" "less_equal" "equal" "not_equal" "greater_equal" "greater" "never" "always" "disabled",
            "border" : [1, 1, 1, 1],
            "min_lod" : -3.40282347E+38,
            "max_lod" : 3.40282347E+38
        }
    },

    "macroblocks" :
    {
        "unique_name" :
        {
            "scissor_test" : false,
            "depth_check" : true,
            "depth_write" : true,
            "depth_function" : "less" "less_equal" "equal" "not_equal" "greater_equal" "greater" "never" "always",
            "depth_bias_constant" : 0,
            "depth_bias_slope_scale" : 0,
            "cull_mode" : "none" "clockwise" "anticlockwise",
            "polygon_mode" : "points" "wireframe" "solid"
        }
    },

    "blendblocks" :
    {
        "unique_name" :
        {
            "alpha_to_coverage" : false,
            "blendmask" : "rgba",
            "separate_blend" : true,
            "src_blend_factor" : "one" "zero" "dst_colour" "src_colour" "one_minus_dst_colour" "one_minus_src_colour" "dst_alpha" "src_alpha" "one_minus_dst_alpha" "one_minus_src_alpha",
            "dst_blend_factor" : "one" "zero" "dst_colour" "src_colour" "one_minus_dst_colour" "one_minus_src_colour" "dst_alpha" "src_alpha" "one_minus_dst_alpha" "one_minus_src_alpha",
            "src_alpha_blend_factor" : "one" "zero" "dst_colour" "src_colour" "one_minus_dst_colour" "one_minus_src_colour" "dst_alpha" "src_alpha" "one_minus_dst_alpha" "one_minus_src_alpha",
            "dst_alpha_blend_factor" : "one" "zero" "dst_colour" "src_colour" "one_minus_dst_colour" "one_minus_src_colour" "dst_alpha" "src_alpha" "one_minus_dst_alpha" "one_minus_src_alpha",
            "blend_operation" : "add" "subtract" "reverse_subtract" "min" "max",
            "blend_operation_alpha" : "add" "subtract" "reverse_subtract" "min" "max"
        }
    },

    "pbs" :
    {
        "material_name" :
        {
            "macroblock" : "unique_name" ["unique_name", "unique_name_for_shadows"],
            "blendblock" : "unique_name" ["unique_name", "unique_name_for_shadows"],
            "alpha_test" : ["less" "less_equal" "equal" "not_equal" "greater_equal" "greater" "never" "always" "disabled", 0.5],
            "shadow_const_bias" : 0.01,

            "workflow" : "specular_ogre" "specular_fresnel" "metallic",

            "transparency" :
            {
                "value" : 1.0,
                "mode" : "None" "Transparent" "Fade",
                "use_alpha_from_textures" : true
            },

            "diffuse" :
            {
                "value" : [1, 1, 1],
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "specular" :
            {
                "value" : [1, 1, 1],
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "roughness" :
            {
                "value" : 1,
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "fresnel" :
            {
                "mode" : "coeff" "ior" "coloured" "coloured_ior",
                "value" : [1, 1, 1],
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "metallness" :
            {
                "value" : 1,
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "normal" :
            {
                "value" : 1,
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "detail_weight" :
            {
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "detail_diffuse0" :
            {
                "mode" : "NormalNonPremul" "NormalPremul" "Add" "Subtract" "Multiply" "Multiply2x" "Screen" "Overlay" "Lighten" "Darken" "GrainExtract" "GrainMerge" "Difference",
                "offset" : [0, 0],
                "scale" : [1, 1],
                "value" : 1,
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "detail_normal0" :
            {
                "offset" : [0, 0],
                "scale" : [1, 1],
                "value" : 1,
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },

            "reflection" :
            {
                "texture" : "cubemap.png",
                "sampler" : "unique_name"
            }
        }
    }
}
This is awesome! it's the best material documentation i've ever seen!

Isn't it worth a news entry on the homopage?
User avatar
Crystal Hammer
Orc
Posts: 402
Joined: Sat Jun 23, 2007 5:16 pm
x 112

Re: Hlms + JSON now in 2.1

Post by Crystal Hammer »

IMO doesn't look much like a documentation just a snippet, at least not when compared to all what Ogre 1.9 has.
Yeah, some news would be cool since the last was around June and it's December :lol: (one might think the project is dead :( ).
Well if gathering all, makes actually enough to say something has changed since then, right?
al2950
OGRE Expert User
OGRE Expert User
Posts: 1227
Joined: Thu Dec 11, 2008 7:56 pm
Location: Bristol, UK
x 157

Re: Hlms + JSON now in 2.1

Post by al2950 »

dark_sylinc wrote:
al2950 wrote:Amazing, so much easier to use for both 'users' & 'developers' :). Can we add json support for the compositors as well?
Haha, Let's not get carried away! shall we? (new shiny object syndrome)
What me.... :roll: !
dark_sylinc wrote:This would add unnecessary complexity C++ side that can easily be solved via this new JSON.
I thought this might be the case. The only reason I mentioned it was because I have many different materials files (one for each model) and I am lazy and dont want to have to define an anisotropic filtering sample block in every file!
hydexon
Gremlin
Posts: 164
Joined: Sun Apr 14, 2013 8:51 pm
x 10

Re: Hlms + JSON now in 2.1

Post by hydexon »

Just saying some things....

I'm pretty sure you tested the new HLMS JSON in Linux i have to guess, because i had no luck of how CMake detected my RapidJSON package which i downloaded in GitHub, in your FindRapidJSON.cmake file, what paths looks first to find RapidJSON because i opened that file Rapidjson_HOME and i set it as envionrment variable, and nothing.

Also i have some precompiler header errors regarding LwOgreString.cpp forgot to add OgreStableHeaders.h, probably is an compiler side-effect error when RapidJSON headers wasn't found.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5446
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1348

Re: Hlms + JSON now in 2.1

Post by dark_sylinc »

hydexon wrote:Also i have some precompiler header errors regarding LwOgreString.cpp forgot to add OgreStableHeaders.h, probably is an compiler side-effect error when RapidJSON headers wasn't found.
Nope, plain bug. I added LwString pretty late and forgot to test on Windows. Didn't realize it broke quite a lot of things on Windows. Fixed. Thanks.
hydexon wrote:I'm pretty sure you tested the new HLMS JSON in Linux i have to guess, because i had no luck of how CMake detected my RapidJSON package which i downloaded in GitHub, in your FindRapidJSON.cmake file, what paths looks first to find RapidJSON because i opened that file Rapidjson_HOME and i set it as envionrment variable, and nothing.
RapidJSON was added to OgreDeps. If you use it, it will work. Does for me on Windows.
Basically CMake script expects on Dependencies\include\rapidjson.
As for why Rapidjson_HOME didn't work I have no idea. Sometimes (always) CMake sets me on my nerves. **sigh**
If someone wants to fix the script be my guess. Maybe the script expected it to be in Rapidjson_HOME/include/rapidjson or Rapidjson_HOME/rapidjson. Just a theory.
hydexon
Gremlin
Posts: 164
Joined: Sun Apr 14, 2013 8:51 pm
x 10

Re: Hlms + JSON now in 2.1

Post by hydexon »

Works perfectly in Linux OpenSUSE 42.1 Leap (Downloaded RapidJSON via GitHub and installed via CMake/Makefiles) but in Windows 10/VS2015 reports me another compiler error, this time is the D3D11RenderSystem: Ogre::v1::HardwarePixelBufferSharedPtr*: unknown size when compiling OgreD3D11RenderWindow.cpp, from the vector STL header, adding #include "OgreHardwarePixelBuffer.h" solved my problem.
User avatar
Kojack
OGRE Moderator
OGRE Moderator
Posts: 7157
Joined: Sun Jan 25, 2004 7:35 am
Location: Brisbane, Australia
x 535

Re: Hlms + JSON now in 2.1

Post by Kojack »

I'm not overly fond of json (such as the lack of comments), but if it makes it easier for tools and devs then that's cool.
(looking over the code, I'm getting an urge to make a Lua version too. I wish there was a header-only version of Lua so it's easier to add to projects)
User avatar
Zonder
Ogre Magi
Posts: 1172
Joined: Mon Aug 04, 2008 7:51 pm
Location: Manchester - England
x 76

Re: Hlms + JSON now in 2.1

Post by Zonder »

dark_sylinc wrote: [*]Very sensible to syntax error. For example the following is invalid JSON:

Code: Select all

{
 "hello" : "world",
 "hello" : "from JSON",
}
What's the error? there is an extra comma after fromJSON. It needs to be removed. When you're writing an entire JSON from scratch by hand, these subtleties can drive you crazy (and right now we don't have any Compositor node editor).
You shouldn't duplicate the Object member name either really as it's undefined behaviour for parsers.
RFC 7159 wrote: 4. Objects

An object structure is represented as a pair of curly brackets
surrounding zero or more name/value pairs (or members). A name is a
string. A single colon comes after each name, separating the name
from the value. A single comma separates a value from a following
name. The names within an object SHOULD be unique.

object = begin-object [ member *( value-separator member ) ]
end-object

member = string name-separator value

An object whose names are all unique is interoperable in the sense
that all software implementations receiving that object will agree on
the name-value mappings. When the names within an object are not
unique, the behavior of software that receives such an object is
unpredictable. Many implementations report the last name/value pair
only. Other implementations report an error or fail to parse the
object, and some implementations report all of the name/value pairs,
including duplicates.

JSON parsing libraries have been observed to differ as to whether or
not they make the ordering of object members visible to calling
software. Implementations whose behavior does not depend on member
ordering will be interoperable in the sense that they will not be
affected by these differences.
There are 10 types of people in the world: Those who understand binary, and those who don't...
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5446
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1348

Re: Hlms + JSON now in 2.1

Post by dark_sylinc »

Zonder wrote:
dark_sylinc wrote: [*]Very sensible to syntax error. For example the following is invalid JSON:

Code: Select all

{
 "hello" : "world",
 "hello" : "from JSON",
}
What's the error? there is an extra comma after fromJSON. It needs to be removed. When you're writing an entire JSON from scratch by hand, these subtleties can drive you crazy (and right now we don't have any Compositor node editor).
You shouldn't duplicate the Object member name either really as it's undefined behaviour for parsers.
Oops. That wasn't the point to show. Fixed the original post. Thanks.
Kojack wrote:I'm not overly fond of json (such as the lack of comments), but if it makes it easier for tools and devs then that's cool.
Yes, I have plans on making Dergo spit out JSON code for all the materials in the scene. Dergo needs that, and a nice installation, to make it usable for other users.
It should boost productivity many times fold.
User avatar
only_a_ptr
Halfling
Posts: 65
Joined: Sun Apr 26, 2009 8:43 pm
x 3

Re: Hlms + JSON now in 2.1

Post by only_a_ptr »

dark_sylinc wrote:Yes, I have plans on making Dergo spit out JSON code for all the materials in the scene. Dergo needs that, and a nice installation, to make it usable for other users.
It should boost productivity many times fold.
Yes! :D

That partially answers my questions in the Dergo thread
Rigs of Rods is alive and kicking!
User avatar
Zonder
Ogre Magi
Posts: 1172
Joined: Mon Aug 04, 2008 7:51 pm
Location: Manchester - England
x 76

Re: Hlms + JSON now in 2.1

Post by Zonder »

dark_sylinc wrote:
Zonder wrote:
dark_sylinc wrote: [*]Very sensible to syntax error. For example the following is invalid JSON:

Code: Select all

{
 "hello" : "world",
 "hello" : "from JSON",
}
What's the error? there is an extra comma after fromJSON. It needs to be removed. When you're writing an entire JSON from scratch by hand, these subtleties can drive you crazy (and right now we don't have any Compositor node editor).
You shouldn't duplicate the Object member name either really as it's undefined behaviour for parsers.
Oops. That wasn't the point to show. Fixed the original post. Thanks.
NP
There are 10 types of people in the world: Those who understand binary, and those who don't...
dermont
Bugbear
Posts: 812
Joined: Thu Dec 09, 2004 2:51 am
x 42

Re: Hlms + JSON now in 2.1

Post by dermont »

How to I disable this? DOGRE_CONFIG_ENABLE_JSON... only appears work when the Rapidjson headers are found/installed, otherwise the CMake build fails with:

Code: Select all

CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
Rapidjson_INCLUDE_DIR (ADVANCED)
   used as include directory in directory /media/sdb5/Libraries/OGRE/src/v2-1Test/OgreMain
   used as include directory in directory /media/sdb5/Libraries/OGRE/src/v2-1Test/RenderSystems/GL3Plus
   used as include directory in directory /media/sdb5/Libraries/OGRE/src/v2-1Test/RenderSystems/NULL
   used as include directory in directory /media/sdb5/Libraries/OGRE/src/v2-1Test/PlugIns/ParticleFX
   used as include directory in directory /media/sdb5/Libraries/OGRE/src/v2-1Test/Components/Hlms/PbsMobile
   used as include directory in directory /media/sdb5/Libraries/OGRE/src/v2-1Test/Components/Hlms/UnlitMobile
   used as include directory in directory /media/sdb5/Libraries/OGRE/src/v2-1Test/Components/Hlms/Pbs
   used as include directory in directory /media/sdb5/Libraries/OGRE/src/v2-1Test/Components/Hlms/Unlit
   used as include directory in directory /media/sdb5/Libraries/OGRE/src/v2-1Test/Components/MeshLodGenerator
   used as include directory in directory /media/sdb5/Libraries/OGRE/src/v2-1Test/Components/Overlay
   used as include directory in directory /media/sdb5/Libraries/OGRE/src/v2-1Test/Tools/MeshTool
   used as include directory in directory /media/sdb5/Libraries/OGRE/src/v2-1Test/Docs
There are a couple of typos:

Code: Select all

/media/sdb5/Libraries/OGRE/src/v2-1/Components/Hlms/Pbs/include/OgreHlmsJsonPbs.h:52:33: error: extra qualification ‘Ogre::HlmsJsonPbs::’ on member ‘parseBrdf’ [-fpermissive]
//static PbsBrdf::PbsBrdf HlmsJsonPbs::parseBrdf(const char *value);
static PbsBrdf::PbsBrdf parseBrdf(const char *value);
FindRapidjson.cmake

Code: Select all

###use_pkgconfig(Rapidjson_PKGC Rapidjson)
use_pkgconfig(Rapidjson_PKGC RapidJSON)
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5446
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1348

Re: Hlms + JSON now in 2.1

Post by dark_sylinc »

**sigh**

Thanks for reporting. All should be fixed now (OGRE_CONFIG_ENABLE_JSON doesn't appear because CMake is supposed to force it off when rapidjson isn't found, instead of failing to generate the project).
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5446
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1348

Re: Hlms + JSON now in 2.1

Post by dark_sylinc »

It may be a little WIP, but saving to JSON was added (PBS-only for now).

All you need to do is:

Code: Select all

mHlmsManager->saveMaterials( Ogre::HLMS_PBS, "/path/myFilename.material.json" );
It will save ALL pbs materials to JSON. I plan on adding a function to only save individual datablocks per file; and also output to string and not just file.
paroj
OGRE Team Member
OGRE Team Member
Posts: 2126
Joined: Sun Mar 30, 2014 2:51 pm
x 1141

Re: Hlms + JSON now in 2.1

Post by paroj »

as HLMS is now also in 1.10, I think that one should also consider the upgrade path when talking about replacing material scripts by JSON for HLMS. (note that the HLMS file format in 1.10 is still pending)

I agree that if we want to drop the material scripts, the step to HLMS and 2.1 is the right time to do it, however I also think that there should be some transition towards it.

This might be the automatic JSON export. Another (additional) option would be delaying the removal of HLMS support from material scripts until 2.2. During 2.1 only a subset of JSON functionality would be supported and HLMS materials would be marked as deprecated. This would also give some time to gather experience how well the JSON materials work out, before leaving them as the only choice.
User avatar
GlowingPotato
Goblin
Posts: 211
Joined: Wed May 08, 2013 2:58 pm
x 10

Re: Hlms + JSON now in 2.1

Post by GlowingPotato »

HI!

Code: Select all

"diffuse" :
            {
                "value" : [1, 1, 1],
                "texture" : "texture.png",
                "sampler" : "unique_name",
                "uv" : 0
            },
The "UV" means that i can pass another diffuse map to the object if my object have another texture channel/UV set ? If true, how many UV sets can i have?

like

Code: Select all

 "value" : [1, 1, 1],
                "texture" : "texture2.png",
                "sampler" : "unique_name",
                "uv" : 1
?