Raytest and Backface filtering Bug?

[AlexL]

Hi.

I tried the backface filtering during triangle raycasts using

Code: Select all

  btCollisionWorld::ClosestRayResultCallback resultCB (...);
  resultCB.m_flags = btTriangleRaycastCallback::kF_FilterBackfaces;

but this does not seem to work correctly, as it sometimes doesn't report a hit or only on backfaces.
So I changed the method btTriangleRaycastCallback::processTriangle in btRaycastCallback.cpp (latest svn) from

Code: Select all

   //@BP Mod - Backface filtering
   if (((m_flags & kF_FilterBackfaces) != 0) && (dist_a > btScalar(0.0)))
   {
      // Backface, skip check
      return;
   }

to

Code: Select all

   //@BP Mod - Backface filtering
   if (((m_flags & kF_FilterBackfaces) != 0) && (dist_b > dist_a))
   {
      // Backface, skip check
      return;
   }

which seems to work as expected.
I found this solution in this thread http://bulletphysics.org/Bullet/phpBB3/ ... 216#p11216
and also verified this on paper as (distB-distA) equals triangleNormal.dot(m_to-m_from).

Or am I completely misled?

[Erwin Coumans]

It might be an issue as we haven't tested this feature. We should add a unit test for it, and fix it if broken.

Thanks for bringing this up, the issue is reported in the tracker: http://code.google.com/p/bullet/issues/detail?id=378
Erwin

[AlexL]

Thanks for looking into this.
I guess the check for not flipping the normal (some lines below) would need to be adjusted as well, though I haven't checked this.

Code: Select all

                  //@BP Mod - Allow for unflipped normal when raycasting against backfaces
                  if (((m_flags & kF_KeepUnflippedNormal) != 0) || (dist_a <= btScalar(0.0)))
						{
							m_hitFraction = reportHit(-triangleNormal,distance,partId,triangleIndex);
						}

to

Code: Select all

                  //@BP Mod - Allow for unflipped normal when raycasting against backfaces
                  if (((m_flags & kF_KeepUnflippedNormal) != 0) || (dist_b <= dist_a))
						{
							m_hitFraction = reportHit(-triangleNormal,distance,partId,triangleIndex);
						}

Of course all these changes only if my original assumption is true (that dist_a > 0.0 is the wrong test for a backface and should rather be dist_b > dist_a)

Alex

[ink]

There is some issue with keeping backface normal unflipped (btRaycastCallback.cpp, line 105, bullet 2.77) :

Code: Select all

if (((m_flags & kF_KeepUnflippedNormal) != 0) || (dist_a <= btScalar(0.0)))
{
    m_hitFraction = reportHit(-triangleNormal,distance,partId,triangleIndex);
}

With this condition, normal is _always_ flipped if kF_KeepUnflippedNormal flag is set, regardless of triangle facing (since ((m_flags & kF_KeepUnflippedNormal) != 0) is true and condition parts are combined with OR).

I changed it with this (flip normal only if we DON'T need to keep it unflipped AND it's backface) :

Code: Select all

if (((m_flags & kF_KeepUnflippedNormal) == 0) && (dist_a <= btScalar(0.0)))
{
    m_hitFraction = reportHit(-triangleNormal,distance,partId,triangleIndex);
}

It seems to work fine, but I'm still not sure which test for backfacing is correct since on line 61 it is opposite to test on line 105 (dist_a > btScalar(0.0)) and some other options were suggested in the thread, like (dist_b <= dist_a), though I haven't tried them yet.

[VicariousEnt]

I posted a patch to fix this problem....

http://code.google.com/p/bullet/issues/detail?id=378

ink, I came to the same conclusion as your change it appears. <= is the way to go. My patch also fixes another issue where TriMeshes inside btCompoundShapes weren't being effected by the culling flags.