Serious bug in btOptimizedBvh.cpp

[reltham]

This is added to the google code issue tracker as well.

These lines:
807 rayDirection[0] = btScalar(1.0) / rayDirection[0];
808 rayDirection[1] = btScalar(1.0) / rayDirection[1];
809 rayDirection[2] = btScalar(1.0) / rayDirection[2];

Were changed to this:
807 rayDirection[0] = rayDirection[0] == btScalar(0.0) ? btScalar(1.0) : btScalar(1.0) / rayDirection[0];
808 rayDirection[1] = rayDirection[1] == btScalar(0.0) ? btScalar(1.0) : btScalar(1.0) / rayDirection[1];
809 rayDirection[2] = rayDirection[2] == btScalar(0.0) ? btScalar(1.0) : btScalar(1.0) / rayDirection[2];

This code is invalied for any ray direction that is along an axis. (for example (0, -1, 0), it ends up changing the ray direction badly.
This causes many raytest or convexsweeptest to fail (miss or report an incorrect hit).

In my code I changed the lines to this:
807 rayDirection[0] = rayDirection[0] == btScalar(0.0) ? btScalar(9999999.0) : btScalar(1.0) / rayDirection[0];
808 rayDirection[1] = rayDirection[1] == btScalar(0.0) ? btScalar(9999999.0) : btScalar(1.0) / rayDirection[1];
809 rayDirection[2] = rayDirection[2] == btScalar(0.0) ? btScalar(9999999.0) : btScalar(1.0) / rayDirection[2];

And this resolved the problem for me. Perhaps there is another way to accomplish what is desired that would be more correct than my solution?
Anyway, can this change be made in the meantime?

[Erwin Coumans]

Thanks for reporting, this problem was introduced just before 2.70 release to get rid of some FPU exceptions.

We will fix it asap,
Erwin