Infinite loop in scene query?

[Sharsnik]

I traced an infinite loop to raySceneQuery->execute().

I'm still having some trouble with this so I'll modify my original post. The sceneQuery eventually causes a stack overflow, it seems the cause is a very small (or zero) length on the direction of the ray. To solve this I weeded out any (0, 0, 0) directions, and normalized the vector before using it in the query. But, the problem is still occurring. There are a few ways that my rayIntersection function can be called, but I'm 99% sure the issues is in testClipping, as the rest of the calls involve straight down queries. I'll include relevant code.

Code: Select all

bool ogreEngine::rayIntersection(Vector3 start, Vector3 direction, Vector3* intersectionPoint, triangle* intersectionTri, float* distance, int flags)
{
	float closeHit = -1;
	triangle testTri;
	Vector3 testPoint;
	float testDist;
	bool hasHit = false;

	float rayLength = direction.length();

	direction.normalise();

	if ((float)direction.x == 0 && (float)direction.y == 0 && (float)direction.z == 0) 
//Ogre reals consider 0 and -0 to be different values? 
 //Prolly an issue with double -> float precision.
		return false;

	printe("(%f, %f, %f)", direction.x, direction.y, direction.z);

	rayTest = Ray(start, direction);
	raySceneQuery->setRay(rayTest);
	raySceneQuery->setQueryMask((uint32)flags);

	//printe("Execute query");
	result = raySceneQuery->execute();
	//printe("End query");

	if (!result.size())
		return false;
	
	for (u_int i = 0; i < result.size(); i++)
    {
		if (result[i].worldFragment && (!hasHit || result[i].distance < testDist) && flags != 3)//FIX ME
		{
			if (intersectionTri)
			{
				Vector3 temp = result[i].worldFragment->singleIntersection;

				testTri.p1 = D3DXVECTOR3(temp.x + 1, getHeight(Vector3(temp.x + 1, 1024, temp.z)), temp.z);
				testTri.p2 = D3DXVECTOR3(temp.x - 1, getHeight(Vector3(temp.x - 1, 1024, temp.z)), temp.z);
				testTri.p3 = D3DXVECTOR3(temp.x, getHeight(Vector3(temp.x, 1024, temp.z + 1)), temp.z + 1);
			}

			if (intersectionPoint)
			{
				testPoint = result[i].worldFragment->singleIntersection;
			}

			testDist = result[i].distance;

			hasHit = true;
		}
		else if (result[i].movable && (result[i].movable->getMovableType().compare("Entity") == 0))
		{
			Ogre::Entity* entity = static_cast<Ogre::Entity*>(result[i].movable);

			size_t vertex_count = 0;
			size_t index_count = 0;
			Ogre::Vector3 *vertices = NULL;
			unsigned long *indices = NULL;

			collision->GetMeshInformation(entity->getMesh(), vertex_count, vertices, index_count, indices,             
							  entity->getParentNode()->_getDerivedPosition(),
							  entity->getParentNode()->_getDerivedOrientation(),
							  entity->getParentNode()->_getDerivedScale());

			for (int i = 0; i < static_cast<int>(index_count); i += 3)
			{
				std::pair<bool, Ogre::Real> hit = Ogre::Math::intersects(rayTest, vertices[indices[i]], vertices[indices[i+1]], vertices[indices[i+2]], true, false);

				if (hit.first)
				{
					if (!hasHit || hit.second < testDist)
					{
						if (intersectionTri)
						{
							testTri.p1 = Vec2DX(vertices[indices[i]]);
							testTri.p2 = Vec2DX(vertices[indices[i+1]]);
							testTri.p3 = Vec2DX(vertices[indices[i+2]]);
						}

						if (intersectionPoint)
						{
							testPoint = rayTest.getPoint(hit.second);
						}

						testDist = hit.second;

						hasHit = true;
					}
				}
			}

			delete[] vertices;
			delete[] indices;
		}
	}

	if (hasHit)
	{
		if (intersectionTri)
		{
			*intersectionTri = testTri;
		}

		if (intersectionPoint)
		{
			*intersectionPoint = testPoint;
		}

		if (distance)
		{
			*distance = testDist;
		}
	}

	if (hasHit && flags == 3 && testDist >= rayLength)//FIX ME
		return false;

	return hasHit;
}

Code: Select all

bool instance::testClipping(D3DXVECTOR3 start, D3DXVECTOR3 ray, triangle* intersectTri)
{
	rayOrigin = start;
	rayOrigin.y += PATHING_CUTOFF;
	rayDir = ray ;

	if (rayDir.x == 0 && rayDir.z == 0)//Causes hang in ray execute... FIX ME
		return false;

       return ogre.rayIntersection((Vector3)rayOrigin, (Vector3)rayDir, NULL, intersectTri, NULL, 3);//FIX ME
}

Code: Select all

D3DXVECTOR3 instance::getClipping(triangle testTriangle, D3DXVECTOR3 pos, D3DXVECTOR3 heading)
{
	D3DXVECTOR3 normal;
	D3DXVECTOR3 edgeLine;
	triangle tempTriangle;

	D3DXVec3Cross(&normal, &(testTriangle.p2 - testTriangle.p1), &(testTriangle.p3 - testTriangle.p1));

	edgeLine = D3DXVECTOR3(normal.z * pos.y, 0, -normal.x * pos.y);
	D3DXVec3Normalize(&edgeLine, &edgeLine);
	D3DXVec3Scale(&edgeLine, &edgeLine, (length = D3DXVec3Dot(&heading, &edgeLine)));
	
	if (testClipping(pos + edgeLine, edgeLine, &tempTriangle))
	{
		return getClipping(tempTriangle, pos + edgeLine, edgeLine);
	}
	return edgeLine;
}

Could it be a problem converting from the floats in the D3DX structures?

[Wolfmanfx]

show some code how you make the scene query

[Sharsnik]

It's a bug that occurs when the direction of the ray in the rayCast is (0, 0, 0)

[Wolfmanfx]

i tried it with the octree scenemanger and its working

Code: Select all

		m_RaySceneQuery->setRay(Ogre::Ray(Ogre::Vector3::ZERO, Ogre::Vector3::ZERO));

So you have to give more information what scenemanger you are using or at least it this behaviour is reproduceable with one of the demos otherwise i am wasting my time when you keep your answer as short as possible with no infromation inside.