[perplexing] +9 bytes on heap obj. causes AABB assert crash

nullsquared · Post by **nullsquared** » Sat Mar 29, 2008 4:10 pm

Ok, this issue is even more confusing than my shadow mapping issues from the past:

Apparently, I'm running out of stack space, or something. Adding 9 or more bytes to a class (which I always allocate on the heap, mind you) causes Ogre's AABB assert of minCorner <= maxCorner to trigger and kill my application.

Originally it started as an Ogre::Vector3, and I BARELY caught it down - I "randomly" decided to try 12 raw bytes (for the vector's 3 floats), and I got the same issue - that's how I "narrowed" it down.

This issue is really killing me, and I have absolutely no idea what to do here

. It's happened before, but then I refactored my code (guess less variables?) and it disappeared. Now it's happening again. Good thing release mode still works fine, but worse off, when DEBUGGING WITH THE DEBUGGER, IT RUNS AS EXPECTED

.

What the heck am I supposed to think here? This object is always allocated on the heap. I have 658 MB of free physical memory available. Why does debugging run fine (though debug mode on its own, not) as well as release mode? AND WHY AM I GETTING OGRE's AABB ASSERTS INSTEAD OF SOME KIND OF CRASH?

Ok, sorry, but this is _really_ weird. Ideas, ideas, please?!?

Ogre 1.4.6 (mingw SDK)
XP SP2
Code::Blocks (and obviously mingw toolbox)

Post by **Kencho** » Sat Mar 29, 2008 4:14 pm

Some code to look at? I have issues with segment sets' intersections in another project, and something in the back of my head tells me this might be related somehow. I've always considered these computations to be always as exact as possible, but these errors really keep me wondering if I've got lazy transistors in my micro or something

nullsquared · Post by **nullsquared** » Sat Mar 29, 2008 4:17 pm

Kencho wrote:Some code to look at? I have issues with segment sets' intersections in another project, and something in the back of my head tells me this might be related somehow. I've always considered these computations to be always as exact as possible, but these errors really keep me wondering if I've got lazy transistors in my micro or something

I honestly have no idea what kind of code to show you

It happens completely randomly. Adding 1024 bytes to a derived class does not have the same issue

I gotta go now, more debugging later... if any ideas, PLEASE post... don't let this topic die

Post by **pjcast** » Sat Mar 29, 2008 4:21 pm

This is most assuredly a memory corruption issue.

a) Make sure all class variables are initialized to some default value. Try running program through memory debugger (ie valgrind)

b) Make sure you are not overflowing (writing past the end of arrays)

c) Allocating bytes correctly (if manually allocating any raw bytes) - i.e. correct size

d) Make sure you are not mixing Ogre builds (or other lib) - ie old versions with new headers, or debug/release libs (eg plugins), etc

e) make sure you are not double freeing, or reusing deleted pointers.

Try your same project under MSVC, maybe it will produce better debugging information.

btmorex · Post by **btmorex** » Sat Mar 29, 2008 4:38 pm

I agree with pjcast. Sounds like memory corruption. If you have a linux version of your program, I highly recommend valgrind to debug it.

nullsquared · Post by **nullsquared** » Sun Mar 30, 2008 2:12 am

I thought it was memory corruption, too - albeit it, some of the weirdest memory corruption out there

Yeah, I thought about trying out MSVC... I guess it can't hurt, considering running gdb here makes the program run fine

Thanks for the suggestions, time for some hardcore debugging; wish me luck

nullsquared · Post by **nullsquared** » Sun Mar 30, 2008 8:29 pm

crashing point:

call stack:

It seems that the newCenter and newHalfSize values are incorrect, and thus case the assertion (crashes hard on setExtents()) - explaining why release mode just "goes along with it".

Ideas as to why or what to do?

eugen · Post by **eugen** » Sun Mar 30, 2008 9:33 pm

i also get set extents debug assertions with the 1.6.7 version...i had those with 1.6.4 version also
here the error is related to the frustum attached to a shadow texture when rendering additive soft pcf shadows...i get it all the time in the same place and i have some weird nan values for bb corners for the root node (i cant reproduce it right now to give little more details but ill be doing in tomorow)

here i dont think there is any corrupting going on anyway, just an unusual case...

nullsquared · Post by **nullsquared** » Sun Mar 30, 2008 10:25 pm

eugen wrote:i also get set extents debug assertions with the 1.6.7 version...i had those with 1.6.4 version also
here the error is related to the frustum attached to a shadow texture when rendering additive soft pcf shadows...i get it all the time in the same place and i have some weird nan values for bb corners for the root node (i cant reproduce it right now to give little more details but ill be doing in tomorow)

here i dont think there is any corrupting going on anyway, just an unusual case...

Really, eh? Because my previous case of the same error was when I had PCF soft shadows, too - and another before that, in relation to my lighting, or so.

Also, I'm getting this error both in HEAD (Shoggoth) and 1.4.7 (Eihort). I think I'll just edit the assert in there to something that simply "fixes" the min/max corners.

novaumas · Post by **novaumas** » Sun Mar 30, 2008 10:51 pm

I think what pjcast said has lots of sense... I'd look along those lines.

I think I'll just edit the assert in there to something that simply "fixes" the min/max corners

Although a temporary fix, I don't know if it's really a good idea, you'll only be hiding the real underlying issue.

Asserts are there so that your program crashes hard and you know something weird is going on

nullsquared · Post by **nullsquared** » Sun Mar 30, 2008 10:58 pm

novaumas wrote:I think what pjcast said has lots of sense... I'd look along those lines.

I think I'll just edit the assert in there to something that simply "fixes" the min/max corners
Although a temporary fix, I don't know if it's really a good idea, you'll only be hiding the real underlying issue.

Asserts are there so that your program crashes hard and you know something weird is going on

I did look along those lines, and came up with ... not much.

transformAffine() comes up with 1.#IND000 for the corners, which triggers the assert. Before the transform, the corners look alright. Release (no asserts) runs fine, each and every time - and, unless someone can come up with a solution, this "temp fix" will save me a lot of trouble

.

nullsquared · Post by **nullsquared** » Mon Mar 31, 2008 12:12 am

Really odd: I stepped through everything in MSVC's debugger - and found out that the offending bounding box was my main camera's "merged" bounding box (updated on SceneManager::_updateSceneGraph()).

Immediately, I uncommented the line that attaches my camera to an Ogre node, excluding it from shoving it's bounding box down into the scene graph - guess what? No more assert!!!1!!!1!1111!one1!1!one1!!

It seems that the camera does not always give an AABB-happy bounding box, and causes the application to become FUBAR, when attached to a node - also, explaining why a camera is part of the problem for eugen, too!

Ideas?!

eugen · Post by **eugen** » Mon Mar 31, 2008 12:31 pm

The error here happens when updating the frustum of one of the camera attached to a viewport for a shadow texture
The error is raised in axisalignedbox::setExtents, the min and max values having qnan values for x and y, -10000 and 0 for z respectively. The parent method calling setextent is Frustum::_updatefrustumimpl, min and max values being calculated in calcProjectionParameters at the start of the method

syedhs · Post by **syedhs** » Mon Mar 31, 2008 1:50 pm

There is a possibility you are running mismatched Ogremain.dll ie link to 1.4.7 but run with 1.4.6. I have got quite similar problem - what used to an innocent function calling trigger an ecxeption. Commenting out the offending line seem to solve it, but what really solves it is putting back the correct DLL.

eugen · Post by **eugen** » Mon Mar 31, 2008 4:07 pm

im quite sure the dll mixing case is not the thing here, we have a complex project configuration here and we make sure this never happens.
I believe somewhere there is a subtle bug; i will test it in detail in a few days or so

nullsquared · Post by **nullsquared** » Mon Mar 31, 2008 8:21 pm

eugen wrote:im quite sure the dll mixing case is not the thing here, we have a complex project configuration here and we make sure this never happens.

Ditto. Also, I tried three configurations (which, BTW, were a pain in the *): Code::Blocks SDK 1.4.6, VC++ HEAD from source, VC++ 1.4.7 from source. Each gave the exact same error (except HEAD had an amount of other errors due to the changes).

I believe somewhere there is a subtle bug; i will test it in detail in a few days or so

Mine occurs, as posted, in SceneNode::_updateBounds() when the camera submits its world bounding box, calling transformAffine() on mWorldBoundingBox (thus, calling setExtents() which triggers the assert).