Intrinsic and Extrinsic Camera Properties

Post Reply
goktug
Posts: 4
Joined: Mon Oct 21, 2019 1:41 pm

Intrinsic and Extrinsic Camera Properties

Post by goktug »

I need to use calibration matrices of pybullet camera (intrinsic and extrinsic camera matrices) in my project. However, I could not find them. As far as I know, pybullet only provides view matrix and projection matrix of the camera. According to my research, pybullet benefits from OpenGL.

I have checked and examined OpenGL documentation to construct intrinsic and extrinsic matrices from projection and view matrix. However, I could not find a clear conversion. What I have found up to now is two of three components in intrinsic matrix in pybullet by making extensive amount of calculation and research:

Intrinsic Components:
------------------------------
* Focal length values (fx, fy)
* Two principal point values (cx, cy)
* Pixel Skew (s)

What I found:
------------------
* Focal Length: Since there is only one camera eye, there is only one focal length, which is equal to 1 / tan(q/2) in which q refers to the angle "field of view" in computeProjectionMatrixFOV function.

* Principal Values: The principal points are indices of middle pixel in the image, which is 250 for 500x500 image.

However, In spite of my 2 days research, I could not find pixel skew value. Is there anyone who can provide pixel skew and extrinsic matrix of pybullet camera image ? I think that we need these matrices for many computer vision purposes.
zachjiang
Posts: 1
Joined: Thu Jun 17, 2021 3:09 pm

Re: Intrinsic and Extrinsic Camera Properties

Post by zachjiang »

Check out this post in stackoverflow:
https://stackoverflow.com/questions/604 ... 0#60450420

The skew is 0
Emkey
Posts: 3
Joined: Thu Feb 03, 2022 12:45 pm

Re: Intrinsic and Extrinsic Camera Properties

Post by Emkey »

I stumbled across your post and hoped it would work. I will share my results with you now. Basically I got an RGBD image I managed to get finally, and from how I set the camera, the pinhole prime default settings makes the camera be in a different position (you will be able to see it in the blank void spaces of the images). The depth of the pointcloud and the following depth plot are correct in the sense that depth is shown. So it means that the RGBD image is also in a correct format and properly processed (??) The first image is the pointcloud of a table, a robotic arm and a rectangular object on top. The second image is the same pointcloud, but turning it with my mouse, as it is in 3D. The third image is the depth plot. As you can see, there are some blanks that are where the real camera should be pointing.

Code: Select all

    pcd = o3d.geometry.PointCloud.create_from_rgbd_image(
    rgbd_image,
    o3d.camera.PinholeCameraIntrinsic(
        o3d.camera.PinholeCameraIntrinsicParameters.PrimeSenseDefault))
    # Flip it, otherwise the pointcloud will be upside down.
    pcd.transform([[1, 0, 0, 0], [0, -1, 0, 0], [0, 0, -1, 0], [0, 0, 0, 1]])

    o3d.visualization.draw_geometries([pcd])
depth_shown.png
depth_shown.png (55.86 KiB) Viewed 2990 times
But when changing it to your calculations made + the post I had already checked and rechecked to try and get my head around something, I got this. Taking into account I know fov.

Code: Select all

    q = fov
    Fx = 1/(math.tan(q/2))
    Fy = Fx

    Cx = IMG_SIZE/2
    Cy = Cx

    cam = o3d.camera.PinholeCameraIntrinsic(IMG_SIZE, IMG_SIZE, Fx, Fy, Cx, Cy)
    
    pcd = o3d.geometry.PointCloud.create_from_rgbd_image(
        rgbd_image,cam)
As you can see in the following images, the camera now is in the perfect position, so the calculations seem right. But for some reason, the depth is not getting displayed. The Pointlcoud and the same pointcloud turning it a bit with the mouse, and the depth plot, are completely flat.
depth_not_shown.png
depth_not_shown.png (52.97 KiB) Viewed 2990 times
Post Reply