The EA STL paper is posted

Anything and everything that's related to OGRE or the wider graphics field that doesn't fit into the other forums.
Post Reply
TwoBit
Kobold
Posts: 30
Joined: Sun Dec 17, 2006 5:48 am

The EA STL paper is posted

Post by TwoBit »

voxel
Gnome
Posts: 334
Joined: Wed Aug 02, 2006 9:27 am
Location: Toronto, Canada

Post by voxel »

Is this publicly available?

Most console developers write their own STL allocators (pretty widely known), but not too much beyond that.
User avatar
syedhs
Silver Sponsor
Silver Sponsor
Posts: 2703
Joined: Mon Aug 29, 2005 3:24 pm
Location: Kuala Lumpur, Malaysia
x 51

Post by syedhs »

If you follow the discussion link, it is a proprietary library - only used internally by EA).

Although not publicly available, it saves a lot of time for aspiring small studios from embarking into 'trial and error' kind of thing of performance tweaking (and also do & donts in games programming especially console development). I may be saying incoherently nonsense to some people here :lol:, but that is just what I feel. :wink:

The paper is clear to me, the motivation section is definitely motivating. And you can also see that in some implementation, EASTL is > 30x faster than STL. Definitely will keep this paper for me to keep telling myself - this is the way to go! 8) Suddenly I feel like a total n00b.
voxel
Gnome
Posts: 334
Joined: Wed Aug 02, 2006 9:27 am
Location: Toronto, Canada

Post by voxel »

syedhs wrote:The paper is clear to me, the motivation section is definitely motivating. And you can also see that in some implementation, EASTL is > 30x faster than STL. Definitely will keep this paper for me to keep telling myself - this is the way to go! 8) Suddenly I feel like a total n00b.
Aligning memory , using cache-friendly data structures, and the removal of deep function calls could get you near 30x on certain consoles. The problem is no two consoles are alike nor are even two compilers for the same console alike (i.e gcc vs. VectorC on the PS2).

Regardless, EASTL is a better STL for consoles, but probably has little performance benefit for PCs/Macs.
User avatar
Praetor
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3335
Joined: Tue Jun 21, 2005 8:26 pm
Location: Rochester, New York, US
x 3
Contact:

Post by Praetor »

voxel wrote:Regardless, EASTL is a better STL for consoles, but probably has little performance benefit for PCs/Macs.
Maybe so. But Ogre is trying to be accepting to the console crowd now... so we must be nice to our consoled brethren, and those who develop for them.
User avatar
pjcast
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 2543
Joined: Fri Oct 24, 2003 2:53 am
Location: San Diego, Ca
x 2
Contact:

Post by pjcast »

Praetor wrote:
voxel wrote:Regardless, EASTL is a better STL for consoles, but probably has little performance benefit for PCs/Macs.
Maybe so. But Ogre is trying to be accepting to the console crowd now... so we must be nice to our consoled brethren, and those who develop for them.
I'm not sure how Ogre can play nicely with something there is no code for? Unless there are plans to release EASTL of course.
Have a question about Input? Video? WGE? Come on over... http://www.wreckedgames.com/forum/
User avatar
Praetor
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3335
Joined: Tue Jun 21, 2005 8:26 pm
Location: Rochester, New York, US
x 3
Contact:

Post by Praetor »

pjcast wrote:I'm not sure how Ogre can play nicely with something there is no code for? Unless there are plans to release EASTL of course.
It has only the hint of a serious note. Interesting paper, but we can't really do much with it... I was attempting to be dry. Perhaps I was too dry.
User avatar
pjcast
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 2543
Joined: Fri Oct 24, 2003 2:53 am
Location: San Diego, Ca
x 2
Contact:

Post by pjcast »

Praetor wrote:
pjcast wrote:I'm not sure how Ogre can play nicely with something there is no code for? Unless there are plans to release EASTL of course.
It has only the hint of a serious note. Interesting paper, but we can't really do much with it... I was attempting to be dry. Perhaps I was too dry.
Or.. maybe it was too early in the morning for me to notice the hint of humor :) But, maybe a little dry also ;)
Have a question about Input? Video? WGE? Come on over... http://www.wreckedgames.com/forum/
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Post by sinbad »

Good paper anyway - obviously EA aren't going to release it as open source (pause to be surprised.......ok ;)), but if anyone wants to write their own this is a valuable reference guide to the design issues to consider. Knowing what to write is a fair amount of the battle.

Thanks for the link.
User avatar
Praetor
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3335
Joined: Tue Jun 21, 2005 8:26 pm
Location: Rochester, New York, US
x 3
Contact:

Post by Praetor »

Maybe this can trickle down and effect the "regular" standard library implementers.
Lord LoriK
Goblin
Posts: 254
Joined: Tue Feb 13, 2007 5:33 am

Post by Lord LoriK »

Praetor wrote:Maybe this can trickle down and effect the "regular" standard library implementers.
Let's hope so. STL libraries are one of the aspects that give so much power to C++, and having it further tweaked for speed is something we'd all wish.
TwoBit
Kobold
Posts: 30
Joined: Sun Dec 17, 2006 5:48 am

Post by TwoBit »

Disclaimer: I wrote both EASTL and the EASTL document.
Regardless, EASTL is a better STL for consoles, but probably has little performance benefit for PCs/Macs.
Well the benchmarks in the paper cover PC, Mac, and Linux and EASTL beats std STL those machines as much as it beats it on consoles. At least with those benchmarks. The benchmarks are about 2/3rds of the way into the doc. I can post the benchmark source code if anybody can provide a place to put it.

A primary purpose of the document is to explain the motivations and design considerations for EASTL to the outside world. In particular it is intended for the C++ standardization committee (of which some of them reviewed the document before publication) and intended for non-commercial game developers and open source people such as Ogre.

One of the motivating factors in producing this document was a discussion I participated in here a few months ago related to memory allocation. I was reviewing Ogre and suggesting that a future version change the way memory is controlled, and somebody posted a response to the effect of 'people don't need to overload operator new'. That comment and the urging of Scott Meyers were probably the two things that made the document happen. Writing these things is a lot of work and I actually consider it partially my contribution to Ogre and other similar projects. This document was a lot of work and was done off of work hours.
User avatar
Game_Ender
Ogre Magi
Posts: 1269
Joined: Wed May 25, 2005 2:31 am
Location: Rockville, MD, USA

Post by Game_Ender »

If I am not mistaken one of the reason for several features in the next C++ standard are the no strings attached reference implementations available from boost. I am sure your paper and its thoroughness will go a long way already but is there any consideration of a reference implementation of any of the techniques/new containers shown in the paper?

Oh and don't think your comments went completely unheard, there is Summer of Code project for Ogre that is introducing custom allocators for Ogre.
User avatar
syedhs
Silver Sponsor
Silver Sponsor
Posts: 2703
Joined: Mon Aug 29, 2005 3:24 pm
Location: Kuala Lumpur, Malaysia
x 51

Post by syedhs »

TwoBit wrote:Writing these things is a lot of work and I actually consider it partially my contribution to Ogre and other similar projects. This document was a lot of work and was done off of work hours.
A lot of people especially in open source areas will definitely appreciate your effort :) And I also agree the performance increase is also as great in PC.
voxel
Gnome
Posts: 334
Joined: Wed Aug 02, 2006 9:27 am
Location: Toronto, Canada

Post by voxel »

Lord LoriK wrote:
Praetor wrote:Maybe this can trickle down and effect the "regular" standard library implementers.
Let's hope so. STL libraries are one of the aspects that give so much power to C++, and having it further tweaked for speed is something we'd all wish.
I'd also like to see:

- More vectorizable-friendly data structures (SSE, Altivec, etc.)
- Tool-friendly data structure. i.e Ogre engine uses floats whereas tools use doubles and maybe interval arithmetic.
voxel
Gnome
Posts: 334
Joined: Wed Aug 02, 2006 9:27 am
Location: Toronto, Canada

Post by voxel »

TwoBit wrote:One of the motivating factors in producing this document was a discussion I participated in here a few months ago related to memory allocation. I was reviewing Ogre and suggesting that a future version change the way memory is controlled, and somebody posted a response to the effect of 'people don't need to overload operator new'. That comment and the urging of Scott Meyers were probably the two things that made the document happen. Writing these things is a lot of work and I actually consider it partially my contribution to Ogre and other similar projects. This document was a lot of work and was done off of work hours.
It's a good paper and you should be congratulated on writing in your spare time!

Did you think about just removing features from STL in EASTL? I'm thinking a smaller-more-efficient-predictably STL could be used for embedded systems...
User avatar
Game_Ender
Ogre Magi
Posts: 1269
Joined: Wed May 25, 2005 2:31 am
Location: Rockville, MD, USA

Post by Game_Ender »

It seems like that already did that, don't forget that the consoles EA targets are embedded systems. Lots of the changes in they made were exactly towards that goal: fixed size containers, pool allocators for std containers, etc.

They go into lots of detail about how carefully you can manage memory with EASTL, because apparently some games use within just few kilobytes(!) of available memory. That's on machines with no MMU or virtual memory, where un-handled out of memory errors yield a hard crash.
User avatar
raicuandi
Gargoyle
Posts: 1092
Joined: Wed Nov 09, 2005 12:56 pm
Location: Adelaide, Australia
Contact:

Re: The EA STL paper is posted

Post by raicuandi »

Post Reply