Future of Math in Ogre

[lunkhound]

Yea, wrapping the current x, y, z would be hard. It sounds nice to have it as x(), y() and z(). Probably also have the options of v[0], v[1] and v[2]?

EDIT: Seems like we already have that. It's just a matter of whether or not to use that format, I guess.

We would also need to take away the non-const operator[] stuff. Code like this:

Code: Select all

v[0] = 0;

Isn't going to be compatible with SIMD. const operator[] is fine though.

[vitefalcon]

Are you looking at the same version of Eigen? I'm looking at the latest, 3.1.2.
With the version I'm looking at, you certainly CAN'T just include Core/Matrix.h.
The file that includes all the stuff you need is called "Eigen/Core" (no .h).

Probably we are looking at the same version. Eigen::Vector3f is a typedef of Matrix<3,1> as seen here: http://eigen.tuxfamily.org/dox/group__m ... ce1f998096
You only need to include Core/Matrix.h to include Vector3f.

[lunkhound]

I was also looking into Bullets' math library. It it smaller than Eigen but seems fast enough (at least enough for the Playstation). Maybe also worth considering.

I like the Bullet math better than most. I haven't seen the latest version, latest I've seen is 2.80. But the code is nice and simple, easy to read. It gets the interface mostly right.
It has a few things I don't like:

• Exposes the data members. As an array called m_float. At least the naming is likely to discourage direct access. Also accessor functions are provided.
• btVector4 is derived from btVector3. That seems like trouble. For example, it looks like the dot product of btVector4 is using btVector3::dot.

Bullet also comes with another math lib, its in the Extras/vectormathlibrary folder (no, not the Extras/simdmathlibrary folder, that's something different!). This one is from Sony and supports SSE and AltiVec. It's a bit Altivec-centric since it was made for their PS3 hardware. License reads:

Code: Select all

 Vector Math library for 3-D linear algebra (vector, matrix, quaternion)
   SIMD support for SSE, PowerPC (PPU) and the SPU.
   Also includes generic multi-platform scalar version. 

   Copyright (C) 2006, 2007 Sony Computer Entertainment Inc. 
   All rights reserved.

   Redistribution and use in source and binary forms,
   with or without modification, are permitted provided that the
   following conditions are met:
    * Redistributions of source code must retain the above copyright
      notice, this list of conditions and the following disclaimer.
    * Redistributions in binary form must reproduce the above copyright
      notice, this list of conditions and the following disclaimer in the
      documentation and/or other materials provided with the distribution.
    * Neither the name of the Sony Computer Entertainment Inc nor the names
      of its contributors may be used to endorse or promote products derived
      from this software without specific prior written permission.

   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 
   AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
   IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
   ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
   LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
   CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
   SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
   INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
   CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
   ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
   POSSIBILITY OF SUCH DAMAGE.
 

Actually I think the the fact it was designed for Altivec is a plus since the Xbox360 and PS3 platforms aren't open and aren't as accessible. Also the Altivec platforms are the ones that benefit the most from use of SIMD, since they lack the fancy out-of-order instruction scheduling that we have on x86 platforms.

[lunkhound]

Are you looking at the same version of Eigen? I'm looking at the latest, 3.1.2.
With the version I'm looking at, you certainly CAN'T just include Core/Matrix.h.
The file that includes all the stuff you need is called "Eigen/Core" (no .h).

Probably we are looking at the same version. Eigen::Vector3f is a typedef of Matrix<3,1> as seen here: http://eigen.tuxfamily.org/dox/group__m ... ce1f998096
You only need to include Core/Matrix.h to include Vector3f.

I tried it, it doesn't work to just include the Matrix.h header. I get tons of errors:

Code: Select all

eigen-eigen-5097c01bcdc4\Eigen/src/Core/Matrix.h(106): error C2143: syntax error : missing ';' before '<'
eigen-eigen-5097c01bcdc4\Eigen/src/Core/Matrix.h(106): error C2059: syntax error : '<'
eigen-eigen-5097c01bcdc4\Eigen/src/Core/Matrix.h(107): error C2143: syntax error : missing ';' before '{'
eigen-eigen-5097c01bcdc4\Eigen/src/Core/Matrix.h(107): error C2447: '{' : missing function header (old-style formal list?)
eigen-eigen-5097c01bcdc4\Eigen/src/Core/Matrix.h(128): error C2143: syntax error : missing ',' before '<'
eigen-eigen-5097c01bcdc4\Eigen/src/Core/Matrix.h(348) : see reference to class template instantiation 'Eigen::Matrix<_Scalar,_Rows,_Cols,_Options,_MaxRows,_MaxCols>' being compiled

My test code:

Code: Select all

//#include <Eigen/Core>
#include <Eigen/src/Core/Matrix.h>

using namespace Eigen;

int main(int argc, char *argv[])
{
    Vector3f v;
    return 0;
}

When I uncomment the first line it compiles. Matrix.h does not include the files it depends on. In fact, it doesn't #include anything at all!

[vitefalcon]

Yup. I just took a closer look at Matrix.h and couldn't find out where those macros were defined. If Eigen inherently tries to include all files just to make use of one feature, I wouldn't opt for it. That's just plain wrong by design for a library. You should be able to choose what you want to use rather force people include everything. They might have a perfectly good reason for that. But that doesn't suit Ogre3D's use-case. So in that case, it's time to shift to the next best option available; bullet math.

[Kojack]

Some others to consider:
GLM (MIT license, replicates GLSL math syntax as closely as possible, has SSE and can be used inside of Cuda kernels). http://glm.g-truc.net
CML (Boost license, no cpu specific optimisations like SSE). http://cmldev.net

There's a simple benchmark and comparison of Eigen, GLM and CML here: http://www.mfoot.com/2012/01/choosing-a ... evelopment
In pure C++ (no SSE) GLM is 3 times faster than CML, which is in turn a tiny bit faster than Eigen.
SSE changes the order, Eigen is almost 5 times faster than GLM, with CML coming in last.
Both Arm v7a and Arm v7a NEON builds follow the order of GLM fastest, then Eigen, then CML last.

I haven't looked into either much, so there might be a ton of problems. Plus I'm a bit distracted, my Novint Falcon just arrived an hour ago.

[vitefalcon]

LOL. I did suggest GLMbut you replied that you wouldn't go for it as it was slow. But you did mention it was interesting.

BTW, I was working with creating a 3D modelling package for Novint Falcon in 2007. I did showcase it here at that time: CRE8. It's now called Cloud9. That was a fun and stressful time. Haptics had just come out and so did Novint Falcon. Have fun!

[Kojack]

That was 20 days ago, I can't remember that far back!
I'm 99% sure I've never looked at CML though.

There's also DirectXMath, it has SSE and NEON optimisations.
But I have no idea what the license is (probably not good), it needs C++11 and it may not work on other platforms (it's mainly intended for windows and windows 8 rt).
Plus it seems to use a C style interface (all standard functionality is via global functions that take two parameters, instead of methods).

(Not really in the running, just thought I'd mention it for the sake of completeness)


(I'm downloading CRE8 now. I'd be playing with getting the Falcon into ogre myself right now except only 25 people can download the sdk per day and I missed out today. But this is getting off topic)

[lunkhound]

I think the main hurdle we're facing is getting the current OGRE math classes into a form where they can wrap some of these other math libs. The main problem being publicly accessible members like x, y, z in Vector3 for instance.
Making those private, adding accessor functions, and then patching up all the code that accesses those members would go a long way towards being "wrappable". Once we reach the point of being able to wrap some of these libraries, the work of wrapping is pretty small, we could try out a few and decide then which is the best. Or we could leave the wrapping as an exercise for the user.

My point is simply this: there is no need to choose a specific library now. We can't use any of them until we get our code in order.

[masterfalcon]

Here's some more food for thought:

Should we keep our existing math API as intact as possible(which is basically Wild Magic) or adopt some API changes to more closely fit another library?

It's likely that we'll need to reimplement some functions. I'm working on bullet right now and I see that Quaternion doesn't have To/FromAngleAxis for example. And there are numerous functions that are not static or class members, like normalize. Which just has several function signatures for whatever data type we wish to use.

[Wolfmanfx]

If functions are reentrant than the should be static - regarding the api i would try to provide as much as possible as long it makes sense.

[lunkhound]

Here's some more food for thought:

Should we keep our existing math API as intact as possible(which is basically Wild Magic) or adopt some API changes to more closely fit another library?

It's likely that we'll need to reimplement some functions. I'm working on bullet right now and I see that Quaternion doesn't have To/FromAngleAxis for example. And there are numerous functions that are not static or class members, like normalize. Which just has several function signatures for whatever data type we wish to use.

My personal preference is a math API that tries to follow Cg/GLSL syntax to a certain degree. However the swizzling stuff is a bit much to shoehorn into C++, so I'd say skip the swizzling.
On the other hand, I'm not sure it's a good move to make any more changes than necessary to the API because it will make migration harder for people.

[vitefalcon]

On the other hand, I'm not sure it's a good move to make any more changes than necessary to the API because it will make migration harder for people.

But this is the only time when we can justify making breaking changes for the better. 2.0 is a major version change and if we need to break existing code for the better, seize the moment and let this happen. Otherwise, who knows how long it would be before 3.0? I'm pretty sure that if this math interface doesn't change for 2.0, it wouldn't till 3.0.

[PhilipLB]

To throw this one in: It would be great, if the Math-library could invert a matrix of any dimension. When the matrix is underdetermined, it should return a pseudoinverse like in http://www-evasion.imag.fr/people/Franc ... om2000.pdf .
Then I could implement having sharp and thin features in the volume rendering code some day.

[spacegaier]

To throw this one in: It would be great, if the Math-library could invert a matrix of any dimension. When the matrix is underdetermined, it should return a pseudoinverse like in http://www-evasion.imag.fr/people/Franc ... om2000.pdf .
Then I could implement having sharp and thin features in the volume rendering code some day.

Then please add it as a feature request on JIRA (with a link back to this post), so it doesn't get lost.

[lunkhound]

On the other hand, I'm not sure it's a good move to make any more changes than necessary to the API because it will make migration harder for people.

But this is the only time when we can justify making breaking changes for the better. 2.0 is a major version change and if we need to break existing code for the better, seize the moment and let this happen. Otherwise, who knows how long it would be before 3.0? I'm pretty sure that if this math interface doesn't change for 2.0, it wouldn't till 3.0.

Agreed on that. I'm just saying there's an argument to be made that making changes which are for the sole purpose of beautifying the math code isn't the best use of resources. Every minute spent making your game's math code more beautiful doesn't make your game more awesome. Same is true for OGRE.
The arguments for changing to a Cg-style math API would be:

• Makes it easier to move code snippets back and forth between GPU and CPU
• Less API to remember if you already know Cg
• Easier for new users to pickup if they already know Cg

I'm torn, but I think I'm leaning more towards no-beautification. I'd like to see our finite resources devoted to making OGRE better (performance and features), rather than making the code more beautiful.

[vitefalcon]

I'd like to see our finite resources devoted to making OGRE better (performance and features), rather than making the code more beautiful.

No one is talking about making the code more beautiful for Math. In fact, I prefer vector.x, vector.y over vector.x(), vector.y(). But that's just my personal opinion. What I want to change is to modify Math such that it would be possible to use SIMD, NEON instructions, when required. using .x, .y doesn't work with those if I am correct. Plus having it x(), y(), z() helps to hide Math library behind those functions if required. AFAIC, this change is a necessity than optional.

[lunkhound]

I'd like to see our finite resources devoted to making OGRE better (performance and features), rather than making the code more beautiful.

No one is talking about making the code more beautiful for Math. In fact, I prefer vector.x, vector.y over vector.x(), vector.y(). But that's just my personal opinion. What I want to change is to modify Math such that it would be possible to use SIMD, NEON instructions, when required. using .x, .y doesn't work with those if I am correct. Plus having it x(), y(), z() helps to hide Math library behind those functions if required. AFAIC, this change is a necessity than optional.

I agree. What I was referring to was the changes to the API above and beyond what is necessary to implement SIMD. I.e. changing the whole math API to look similar to Cg (except unlike Cg using accessors like v.x(), etc).
So instead of

Code: Select all

Vector3 c = a;
c.makeCeil( b );

Cg style would be:

Code: Select all

Vector3 c = max(a, b);

[vitefalcon]

Umm... did I miss something? Who said about making it Cg style?

[CABAListic]

I'd like to see our finite resources devoted to making OGRE better (performance and features), rather than making the code more beautiful.

No one is talking about making the code more beautiful for Math. In fact, I prefer vector.x, vector.y over vector.x(), vector.y(). But that's just my personal opinion. What I want to change is to modify Math such that it would be possible to use SIMD, NEON instructions, when required. using .x, .y doesn't work with those if I am correct. Plus having it x(), y(), z() helps to hide Math library behind those functions if required. AFAIC, this change is a necessity than optional.

In principle you could build a small wrapper class that acts like a float, but actually translates get/set operations into SIMD instructions on the actual data type. That way you could keep the interface intact (unless client code uses pointers/references on the vector members). But I really don't think it's important enough to bother.

In fact, if we do find a 3rd party lib that fits our needs, I would vote for using its interface directly. I don't see much point in wrapping it (or if there is, then the lib might not be suitable to begin with), given that 2.0 will probably break on more than just math usage.

[lunkhound]

Umm... did I miss something? Who said about making it Cg style?

Yes you seem to have missed a few posts. See MasterFalcon's post about API changes.

[vitefalcon]

Here's some more food for thought:

Should we keep our existing math API as intact as possible(which is basically Wild Magic) or adopt some API changes to more closely fit another library?

It's likely that we'll need to reimplement some functions. I'm working on bullet right now and I see that Quaternion doesn't have To/FromAngleAxis for example. And there are numerous functions that are not static or class members, like normalize. Which just has several function signatures for whatever data type we wish to use.

My personal preference is a math API that tries to follow Cg/GLSL syntax to a certain degree. However the swizzling stuff is a bit much to shoehorn into C++, so I'd say skip the swizzling.
On the other hand, I'm not sure it's a good move to make any more changes than necessary to the API because it will make migration harder for people.

Seems like you were talking about Cg/GLSL style syntax rather than masterfalcon. He was saying that bullet was missing some static functions and class members like normalize. Atleast the first instance of Cg/GLSL syntax was brought up by you. And you're contradicting yourself later.

[lunkhound]

I'd like to see our finite resources devoted to making OGRE better (performance and features), rather than making the code more beautiful.

No one is talking about making the code more beautiful for Math. In fact, I prefer vector.x, vector.y over vector.x(), vector.y(). But that's just my personal opinion. What I want to change is to modify Math such that it would be possible to use SIMD, NEON instructions, when required. using .x, .y doesn't work with those if I am correct. Plus having it x(), y(), z() helps to hide Math library behind those functions if required. AFAIC, this change is a necessity than optional.

In principle you could build a small wrapper class that acts like a float, but actually translates get/set operations into SIMD instructions on the actual data type. That way you could keep the interface intact (unless client code uses pointers/references on the vector members). But I really don't think it's important enough to bother.

In fact, if we do find a 3rd party lib that fits our needs, I would vote for using its interface directly. I don't see much point in wrapping it (or if there is, then the lib might not be suitable to begin with), given that 2.0 will probably break on more than just math usage.

I think wrapping makes sense if the goal is to enable SIMD performance with minimal effort. If the float-wrapper class works, that might be worth a try.

[lunkhound]

Here's some more food for thought:

Should we keep our existing math API as intact as possible(which is basically Wild Magic) or adopt some API changes to more closely fit another library?

It's likely that we'll need to reimplement some functions. I'm working on bullet right now and I see that Quaternion doesn't have To/FromAngleAxis for example. And there are numerous functions that are not static or class members, like normalize. Which just has several function signatures for whatever data type we wish to use.

My personal preference is a math API that tries to follow Cg/GLSL syntax to a certain degree. However the swizzling stuff is a bit much to shoehorn into C++, so I'd say skip the swizzling.
On the other hand, I'm not sure it's a good move to make any more changes than necessary to the API because it will make migration harder for people.

Seems like you were talking about Cg/GLSL style syntax rather than masterfalcon. He was saying that bullet was missing some static functions and class members like normalize. Atleast the first instance of Cg/GLSL syntax was brought up by you. And you're contradicting yourself later.

Yes MasterFalcon brought up the idea of making broader API changes, and I responded with my preferred API direction, but also raised the concern about too many API changes making migration hard for people. Sorry, I don't see any contradiction.

[Klaim]

On the point of adding SIMD instruction etc., I know that Boost isn't allowed in Ogre and I agree and all, I just wanted to point that Boost.Simd is in Beta and it's code might have been useful in case it is decided not to switch to another library.

See: http://boost.2283326.n4.nabble.com/Boos ... 40374.html

On the topic of wrapping or not, after some thinking I agree with Cabalistic that switching to a new lib would be weaker if done through the same interface than by using the lib itself.