Ogre Forums

the as_XX() functions only work if you defined them in the proto, if you are using dynamic attributes like one structure has position but the other one doesnt have and pass them to same function, to use the pbuffers you need to know the message type of the variable passed , for namevaluepairlist you dont need to know what the list contains anyway, i guess the argument goes out of scope, my main idea here can be summarized with a quote:

There are 3 types of newspaper readers.
Type 1: Believes everything he reads
Type 2: Doesnt believe anything he reads (because he was Type 1 before and got frusturated so many times that now doesnt believe a single word)
Type 3: Reads the newspaper, parses the information by filtering using his own past experience and believing whatever makes sense

So, about the PBs, dont make them myths, they are good for a set of applications and not so good for a set of other applications, what i debate here is that their set of usefull applications is smaller than the set of not so usefull applications, so just dont believe in the perfect announcements by google, evaluate them from all perspectives and use them wherever they may benefit and avoid wherever they fail.

ismail,

NOTE: For some of you who may know where the quote comes from, i just used it since the quote makes sense, I dont approve the crimes the author committed to humanity and personally think he was a disgrace of the human kind...

stealth977 wrote:what i debate here is that their set of usefull applications is smaller than the set of not so usefull applications, so just dont believe in the perfect announcements by google, evaluate them from all perspectives and use them wherever they may benefit and avoid wherever they fail.

My argument still stands. You tested speed/performance without having the protocol buffers in speed/performance mode.

I benchmarked a debug build of Ogre vs. my own rasterizer and Ogre was 50% slower; therefore, I can conclude Ogre is very slow.

stealth977 wrote:the as_XX() functions only work if you defined them in the proto, if you are using dynamic attributes like one structure has position but the other one doesnt have and pass them to same function, to use the pbuffers you need to know the message type of the variable passed , for namevaluepairlist you dont need to know what the list contains anyway

This is not a valid point, IMO.
Dynamic attributes in pbuffers are optional fields. I use optional for each and every field.
If you pass something to a function, no matter what you pass, you still need to know what it is you're passing, right?
Be it a namevaluepairlist or a pbuffer object.
You still need to know the type.

If I was in doubt before, I am definitely going to use Protocol Buffers now!

jacmoe:

I guess you misunderstood what i meant

What i mean is, the pbuffer messages are actually C structures, so you need to know what members they have, so the has_xxx() calls are only used to check if any value assigned to an optional field that YOU KNOW THAT EXISTS.

so if you have a

Message {
optional x;
optional y;
optional z;
}

you can use has_x() to check if any x value defined for that message.

BUT YOU CANT use lets say has_Z(), because Z is unknown to the message and there is no corresponding function created

but in a namevaluepairlist you dont know which value pairs exist in it, you can query for the ones you need and operate on them

Why am i always making examples using namevaluepairlists? because they are easily serialized to XML format as i stated before.

And nulsquared:

man, either you dont know what you are talking about, or you are trying to make a faulty move

First, using the optimize for speed hint in pbuffers can never be compared to debug vs release builds of OGRE, if you really want to make sense, supply some data showing how much speed difference it makes when the optimize for speed hint is used. You know there is no magic behind pbuffers, the functions it creates are there in the code, what it does when optimize for speed is used is to implement the common optimization tricks that most compilers use, like ignoring some checks, passing by vareference when possible, inlining etc. etc. (and most of them are already used even when you dont specify the optimize for speed hint) which would probably make it a few times faster but not 1000x faster.
Also consider that the XML version i used during comparison was not optimized for speed at all.

What annoys me is that, while i am trying to say consider it well with cons and pros before using something, you are still behaving as if there is a pissing race and thats probably due to your younger age

Whatever is just fun for you is business to others, jumping in everything blindly costs you just some fun time, while it costs many people valuable work time / profit / credibility.

Also dont answer as if i am against PBs, all i am trying to say is that they are designed to be perfect for some uses, but not all uses and i stated at above posts in which situations they rock and in which they are doomed to fail.
The gains Google mentions while advertising it is misleading and can only represent the extreme cases. And, especially in my personal case, the gains of using it would not compensate the flexibility i lost using them.

ismail,

First, using the optimize for speed hint in pbuffers can never be compared to debug vs release builds of OGRE

No but i don't think he was comparing it like that. I think he meant that you can't assume that PB's are slower than the equivelent XML if you haven't tested them in their correct configurations. Once they've been tested correctly you can then compare them.

I'm definately looking into PB's for any of my projects. Even if they aren't useful for scene formats could they be useful for configuration files instead? Is it possible to read in using the text format or does it only output in text?

I've never liked using XML because of the large amount of code that has to be written to check for values and do conversions.

you can use has_x() to check if any x value defined for that message.
BUT YOU CANT use lets say has_Z(), because Z is unknown to the message and there is no corresponding function created

But your XML parsing code would need to know to check for certain values, so either way you need code that knows about the possible values that can be provided. With PB's its an actual function you can call.

I think we are talking about two very different things here: Stealth tried to use Protocol Buffers (Sorry, Ismael, for talking you into it) for Ogitor, but it didn't work out.
Because Ogitor deals with a flexible, open scene format, where few assumptions can be made.
It is an editing framework.

Nullsquared, Klaim, Chris Jones and me are talking about the use of pbuffers in a totally different context: a fixed scene format.
In that context Protocol Buffers wins over XML, IMO. Maybe even over a custom binary format (I think it does).
If we plan and optimise for pbuffers, we will see dramatic improvements in both size and speed, I am sure of it.

It is, however, not suited as yet another scene interchange format.
In that particular context, it is a waste of effort.

So, if we do a pissing contest, we better make sure we're pissing in the same direction..

stealth977 wrote: but in a namevaluepairlist you dont know which value pairs exist in it, you can query for the ones you need and operate on them

Why am i always making examples using namevaluepairlists? because they are easily serialized to XML format as i stated before.

What jacmoe, Chris, I, and others are saying, is this:

OK, consider that you have some custom property "foo"="123". You easily save it to XML. However, when you read that same XML - you need to know that there is indeed a "foo" custom property that you need to read. Instead of using strings and finding this property and then converting to an int, you can just a compile a format using protocol buffers that already has a hard-coded foo() that you can use as an integer.

Now, this is your next point: but my scene editor contains game-specific properties that it will not know how to save, but the game engine will know to find.

This dives into unknown territory for a scene editor. A scene editor has clearly defined properties - a vector3 position, a quaternion rotation, etc. This is perfect for google protocol buffers. Oh, and your argument of nesting... here's what my scene format idea would look like: Quick testing of nested messages of the same type work fine.

which would probably make it a few times faster but not 1000x faster.

And where exactly do you see a claim of a 1000x speedup?

What annoys me is that, while i am trying to say consider it well with cons and pros before using something, you are still behaving as if there is a pissing race and thats probably due to your younger age

So, once all proof is gone, we resort to the "it must be your younger age."

I believe we are talking about slightly different things here.
Stealth tried it in the context of Ogitor, where it proved to be more hassle than XML.
It makes sense to me, nobody claims that pbuffers is the new collada, dotscene, fbx or ogscene.
All he's saying is that we should take pbuffers with grains of salt.

However, for my own scene format, pbuffers rock.
Because I am not dealing with a great number of variables here. I know what a scene specific to my 'engine' is like.

Man, you guys are making me feel bad for 'rolling my own level format' now.

jacmoe wrote:I believe we are talking about slightly different things here.
Stealth tried it in the context of Ogitor, where it proved to be more hassle than XML.
It makes sense to me, nobody claims that pbuffers is the new collada, dotscene, fbx or ogscene.
All he's saying is that we should take pbuffers with grains of salt.

It would make sense to me, as well, but some of his claims aren't quite correct.

nikki wrote:Man, you guys are making me feel bad for 'rolling my own level format' now.

If you've already got something going well, keep it

To be precise : I don't have a "scene" format, only some kind of level graph (and other game entities instance data) that is really less complex than a graphic scene graph. So I guess the performances will be there anyway. The interesting part being the network messaging usage part in my case.

Ok, i must admit that at some points i kinda feel insulted and may have written things that may be called as an insult, am not proud of that.
I guess this time jacmoe exactly understood what i meant and it is even a shame for me since i could express myself with a lot less words if i knew and written "we should take pbuffers with grains of salt." because thats exactly what i was trying to say but my knowledge of English limited me

Also of course it is true that my particular way of implementing it may have caused the problems i mentioned, but i am almost sure that the problems are not limited to the implementation. Why i am saying that is:
1 - Google claims that it is 3x-10x smaller, but your typical way of using it will probably get an avarage 2x-3x size improvement
2 - Google claims that it is alot faster, now that depends how you use it or for what purpose you use it, is the 500ms gain so much important while loading a level file? (but of course its important if you have thousands of PB files to parse in small time, it will be hella faster than parsing a thousand XML files)
3 - And for those benefits in 1 and 2, what does it demand? It demands you to restrict your format as much as possible, thats where you should decide whether it worths or not? Not to mention it becomes not human readable (of course you can save it as text, its built in in PBs, but do you want to save both versions at the same time, if so whats the purpose, if you use binary version how will your user convert it to human readable and how will he/she convert it back after changes?)

thats all Sorry if i offended anyone, its not actually intended since i know everyone here tries to state his/her views for the benefit of the community,

ismail,

No worries, Stealth.

Protocol Buffers was made with a specific purpose in mind, efficient RPC protocol data:

Protocol Buffers are a way of encoding structured data in an efficient yet extensible format. Google uses Protocol Buffers for almost all of its internal RPC protocols and file formats.

But, notice the and file formats after RPC protocols.
Which means that we are not totally off with using it as a scene file format.
But that's just a side effect of its intended use: data encoding/decoding across the wire.

Sry for resurrecting this thread, but as it seems a new star is rising:

Flatbuffers:
https://google.github.io/flatbuffers/md ... marks.html

the only thing i miss is the C# support..
But:
C# support is one of the most popular requests, and might get supported in the next release

I have also been following Capt'n'Proto by the same original author of Protobuf: http://kentonv.github.io/capnproto/

He also posted a (biased but honest) comparison of the different solutions: http://kentonv.github.io/capnproto/news ... s-sbe.html

Yesterday, some engineers at Google released FlatBuffers, a new serialization protocol and library with similar design principles to Cap’n Proto. Also, a few months back, Real Logic released Simple Binary Encoding, another protocol and library of this nature.

It seems we now have some friendly rivalry.

Unfortunately Cap'n'Proto can't handle Visual Studio right now so they are waiting for some C++11 support missing (or backporting but it's way too much work) so I can't replace protobuf yet.