x86\x64 undefined behavior feedback

[Scriptslol]

Hi there, just wanted to leave some feedback on some odd behavior that I ran into while working with Bullet.

Visual Studio 2013
Msvc: v100
targets: x86, x64
Bullet: 2.81
BT_USE_DOUBLE_PRECISION

All of Bullet's features and performance has been acceptable between x86 and x64. However, a game breaking error worked itself into the x86 build which would trigger an assert when a character would move on the terrain:

Code: Select all

//btQuantizedBvh.h
SIMD_FORCE_INLINE void quantize(unsigned short* out, const btVector3& point,int isMax) const
{
	btAssert(m_useQuantization);

	btAssert(point.getX() <= m_bvhAabbMax.getX()); //<--- this gets triggered when btKinematicController moves.
	//...
}

Examination

The debugger says the value 'point.getX()' is "1.#QNAN00000000000"

Quiet Not A Number. It's not a division by zero. But, a floating point was unable to resolve trailing floating point values. I figured that since the game is mainly built for x64 this could be an issue with Bullet being fed bad values for the terrain. I checked the creation of btBvhTriangleMeshShape during terrain generation and that looked good.

I started working my way through Bullet with a variant of this logic in attempt to catch when values went bad:

Code: Select all

if (_isnan(point.getX()))
	int test = 0; //<-- Break Point here

Surprisingly, when I put this check right before Broad Phase detection the assert would fire right at game start - NOT when the player moved. Step in the wrong direction.

I finally worked my way down enough in the processes, before values went bad, to recognize that the Transform for btBvhTriangleMeshShape was becoming QNAN'd - this always happen against a capsule shape.

After a couple more hours of trying to understand the 'why' I decided to just work my way back up in game versions for the past 2 months until I found the version it broke and then deduce the call that was making Bullet Assert in x86 and not x64. Here's what I found:

Issue

The problem was in the ghost object for the character controller. Specifically, the collision flags. The commit that broke x86 changed:

Code: Select all

m_ghostObject->setCollisionFlags(btCollisionObject::CF_CHARACTER_OBJECT | btCollisionObject::CF_STATIC_OBJECT);

to

Code: Select all

m_ghostObject->setCollisionFlags(btCollisionObject::CF_CHARACTER_OBJECT);

The above line had no issues in x64 builds. But would always cause an Assert in x86. This is undefined behavior.

This has probably worked itself out with newer versions. But, it is a very indirect issue and I hope some of the feedback I've given has been helpful.