crash in OgreSubEntity when binding shader param

Discussion area about developing or extending OGRE, adding plugins for it or building applications on it. No newbie questions please, use the Help forum for that.
User avatar
lunkhound
Gremlin
Posts: 169
Joined: Sun Apr 29, 2012 1:03 am
Location: Santa Monica, California
x 19

crash in OgreSubEntity when binding shader param

Post by lunkhound »

I get a crash (dereferencing a NULL pointer) in SubEntity::_updateCustomGpuParameter() when binding the "animation_parametric" shader parameter on a mesh with shared vertices. I'm using a vertex shader to apply morph animation to a mesh.

Here is a patch against the v1-8 branch:

Code: Select all

# HG changeset patch
# User lunkhound
# Date 1338935710 25200
# Branch v1-8
# Node ID 60f669bc11e61dcf878addc409ad3fa41033ffe7
# Parent  23c150f91e2089ddc17a360733268a134a2cdc14
[OgreMain] fix for crash when binding animation_parametric to a mesh with shared vertices

diff -r 23c150f91e20 -r 60f669bc11e6 OgreMain/src/OgreSubEntity.cpp
--- a/OgreMain/src/OgreSubEntity.cpp	Thu Aug 23 16:30:05 2012 -0500
+++ b/OgreMain/src/OgreSubEntity.cpp	Tue Jun 05 15:35:10 2012 -0700
@@ -385,13 +385,16 @@
 			// If there are more than 4 entries, this will be called more than once
 			Vector4 val(0.0f,0.0f,0.0f,0.0f);
 
+            // if using shared vertices, dont dereference a NULL ptr
+            const VertexData* vd = mHardwareVertexAnimVertexData ? mHardwareVertexAnimVertexData : mParentEntity->mHardwareVertexAnimVertexData;
+
 			size_t animIndex = constantEntry.data * 4;
 			for (size_t i = 0; i < 4 && 
-				animIndex < mHardwareVertexAnimVertexData->hwAnimationDataList.size();
+				animIndex < vd->hwAnimationDataList.size();
 				++i, ++animIndex)
 			{
 				val[i] = 
-					mHardwareVertexAnimVertexData->hwAnimationDataList[animIndex].parametric;
+					vd->hwAnimationDataList[animIndex].parametric;
 			}
 			// set the parametric morph value
 			params->_writeRawConstant(constantEntry.physicalIndex, val);
This fixes my crash and allows my mesh to animate correctly. Does this look like a sane fix?