issue with ray/physics collision detection

[mgs_oleg]

All,

Used http://www.opengl-tutorial.org/miscella ... cs-library article as base for collision detection, I've got following issue.
Few inputs before problem itself
1. I am initializing my projection and view matrices once, within following piece of code

Code: Select all

projectionMatrix = glm::perspective(90.0f, 4.0f / 3.0f, 0.1f, 100.0f);
	

	float horizontalAngle = 3.14f;
	float verticalAngle =0.0f;/// problems here 

	glm::vec3 direction(
		cos(verticalAngle) * sin(horizontalAngle), 
		sin(verticalAngle),
		cos(verticalAngle) * cos(horizontalAngle)
	);

	glm::vec3 position = glm::vec3( 0.0, 10.0, 32.0 ); 

	glm::vec3 right = glm::vec3(
		sin(horizontalAngle - 3.14f/2.0 ), 
		0,
		cos(horizontalAngle - 3.14f/2.0 )
	);

	// Up vector
	glm::vec3 up = glm::cross( right, direction );

	viewMatrix =  glm::lookAt(
								position,          
								position+direction, 
								up                  
						   );

2. I do not change view and projection matrix based on cursor movements as it was in mentioned tutorial.
3. I am using btConvexHullShape adding a points to it.
4. Here is the code of ray calculation

Code: Select all

 ScreenPosToWorldRay(
	int mouseX, int mouseY,             // Mouse position, in pixels, from bottom-left corner of the window
	int screenWidth, int screenHeight,  // Window size, in pixels
	glm::mat4 ViewMatrix,               // Camera position and orientation
	glm::mat4 ProjectionMatrix,         // Camera parameters (ratio, field of view, near and far planes)
	glm::vec3& out_origin,              // Ouput : Origin of the ray. /!\ Starts at the near plane, so if you want the ray to start at the camera's position instead, ignore this.
	glm::vec3& out_direction            // Ouput : Direction, in world space, of the ray that goes "through" the mouse.
){

	// The ray Start and End positions, in Normalized Device Coordinates (Have you read Tutorial 4 ?)
	glm::vec4 lRayStart_NDC(
		((float)mouseX/(float)screenWidth  - 0.5f) * 2.0f, // [0,1024] -> [-1,1]
		((float)mouseY/(float)screenHeight - 0.5f) * 2.0f, // [0, 768] -> [-1,1]
		-1.0, // The near plane maps to Z=-1 in Normalized Device Coordinates
		1.0f
	);
	glm::vec4 lRayEnd_NDC(
		((float)mouseX/(float)screenWidth  - 0.5f) * 2.0f,
		((float)mouseY/(float)screenHeight - 0.5f) * 2.0f,
		0.0,
		1.0f
	);



	// The Projection matrix goes from Camera Space to NDC.
	// So inverse(ProjectionMatrix) goes from NDC to Camera Space.
	glm::mat4 InverseProjectionMatrix = glm::inverse(ProjectionMatrix);
	
	// The View Matrix goes from World Space to Camera Space.
	// So inverse(ViewMatrix) goes from Camera Space to World Space.
	glm::mat4 InverseViewMatrix = glm::inverse(ViewMatrix);
	
	glm::vec4 lRayStart_camera = InverseProjectionMatrix * lRayStart_NDC;    lRayStart_camera/=lRayStart_camera.w;
	glm::vec4 lRayStart_world  = InverseViewMatrix       * lRayStart_camera; lRayStart_world /=lRayStart_world .w;
	glm::vec4 lRayEnd_camera   = InverseProjectionMatrix * lRayEnd_NDC;      lRayEnd_camera  /=lRayEnd_camera  .w;
	glm::vec4 lRayEnd_world    = InverseViewMatrix       * lRayEnd_camera;   lRayEnd_world   /=lRayEnd_world   .w;


	glm::vec3 lRayDir_world(lRayEnd_world - lRayStart_world);
	lRayDir_world = glm::normalize(lRayDir_world);


	out_origin = glm::vec3(lRayStart_world);
	out_direction = glm::normalize(lRayDir_world);
}

5. Detection routines

Code: Select all

 
out_direction = out_direction*1000.0f;

        btCollisionWorld::ClosestRayResultCallback RayCallback(btVector3(out_origin.x, out_origin.y, out_origin.z), btVector3(out_direction.x, out_direction.y, out_direction.z));
        dynamicsWorld->rayTest(btVector3(out_origin.x, out_origin.y, out_origin.z), btVector3(out_direction.x, out_direction.y, out_direction.z), RayCallback);
        if(RayCallback.hasHit()) {

            int i =  (int)RayCallback.m_collisionObject->getUserPointer();
            printf("DETECTED index = %d \n",i);
        }
        else {
            printf("NOT DETECTED \n");
        }

So, on the screenshot below you can see red area which is actual object that should be picked on clicking it, but it does not. Only when I am clicking inside green area I am getting satisfied results.

Capture.PNG (7.82 KiB) Viewed 2698 times

Please help me on this.

PS
May assume there is an issues with my view matrix - but does not know what might be an issue.

[mgs_oleg]

Please anyone - it is very important for me now - that is major showstopper i am facing at the moment.