STATE_LOGGING_VIDEO_MP4 in DIRECT mode?

Official Python bindings with a focus on reinforcement learning and robotics.
Post Reply
deepcode
Posts: 8
Joined: Sat Aug 26, 2017 2:04 am

STATE_LOGGING_VIDEO_MP4 in DIRECT mode?

Post by deepcode »

it seems like you can only record mp4 video in GUI mode, is that correct?

is there any way to record a video session of what is happening when running in DIRECT mode?
Im running multiple simulations on a linux server and I review sessions of my simulations.
User avatar
Erwin Coumans
Site Admin
Posts: 4221
Joined: Sun Jun 26, 2005 6:43 pm
Location: California, USA
Contact:

Re: STATE_LOGGING_VIDEO_MP4 in DIRECT mode?

Post by Erwin Coumans »

Indeed, STATE_LOGGING_VIDEO_MP4 is only available when using OpenGL (GUI or SHARED_MEMORY).
Thinking of it, we could implement this also in DIRECT mode, using the TinyRenderer to generate images.

Some people log the state of objects, and create the video/playback afterwards.

Another option is to use PyBullet.getCameraImage (which also works in DIRECT mode, it uses a CPU renderer) and write those images to create a video using Python. You can write individual images to disk and use a tool to convert it into a video. OpenAI Gym has also some utility for this, various PyBullet Gym environments provide the 'render' method.
deepcode
Posts: 8
Joined: Sat Aug 26, 2017 2:04 am

Re: STATE_LOGGING_VIDEO_MP4 in DIRECT mode?

Post by deepcode »

I used the render code from the openAI gym examples, and it sort of works, I have 2 objects loaded, both via urdf/obj files.
Only one of the objects appears in the image even though they are next to each other. This would seem like a pybullet issue, is there a way to make sure all the objects get rendered.

here is the render code I use:

Code: Select all

    def _render(self, mode="rgb_array", close=False):
        print("opoWTF")
        print(mode)
        mode ="rgb_array"
        if mode != "rgb_array":
            return np.array([])
        base_pos = [0,0,0]
        _cam_dist = 5  #.3
        _cam_yaw = 50
        _cam_pitch = -35
        _render_width=480
        _render_height=480

        view_matrix = pb.computeViewMatrixFromYawPitchRoll(
            cameraTargetPosition=base_pos,
            distance=_cam_dist,
            yaw=_cam_yaw,
            pitch=_cam_pitch,
            roll=0,
            upAxisIndex=2)
        proj_matrix = pb.computeProjectionMatrixFOV(
            fov=90, aspect=float(_render_width)/_render_height,
            nearVal=0.01, farVal=100.0)
        #proj_matrix=[1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, -1.0000200271606445, -1.0, 0.0, 0.0, -0.02000020071864128, 0.0]
        (_, _, px, _, _) = pb.getCameraImage(
            width=_render_width, height=_render_height, viewMatrix=view_matrix,
            projectionMatrix=proj_matrix, renderer=pb.ER_TINY_RENDERER) #ER_BULLET_HARDWARE_OPENGL)
        rgb_array = np.array(px, dtype=np.uint8)
        rgb_array = np.reshape(rgb_array, (_render_height, _render_width, 4))
        rgb_array = rgb_array[:, :, :3]
        return rgb_array
deepcode
Posts: 8
Joined: Sat Aug 26, 2017 2:04 am

Re: STATE_LOGGING_VIDEO_MP4 in DIRECT mode?

Post by deepcode »

if I use ER_TINY_RENDERER, it is not rendering everything.

if I use ER_BULLET_HARDWARE_OPENGL, it works fine. I can't reliably use HARDWARE_OPENGL because I can't control If other computers have hardware OpenGL though. is this a bug with TINY_RENDERER? the object it doesn't display is an obj file that was created with the vhacd tools in bullet. should any bug be filed for this?
Kukanani
Posts: 25
Joined: Sat Feb 21, 2009 5:08 am

Re: STATE_LOGGING_VIDEO_MP4 in DIRECT mode?

Post by Kukanani »

It would probably be useful to have this information in the quickstart guide.
Post Reply