Page 1 of 1
STATE_LOGGING_VIDEO_MP4 in DIRECT mode?
Posted: Tue Feb 06, 2018 3:39 pm
by deepcode
it seems like you can only record mp4 video in GUI mode, is that correct?
is there any way to record a video session of what is happening when running in DIRECT mode?
Im running multiple simulations on a linux server and I review sessions of my simulations.
Re: STATE_LOGGING_VIDEO_MP4 in DIRECT mode?
Posted: Wed Feb 07, 2018 3:52 am
by Erwin Coumans
Indeed, STATE_LOGGING_VIDEO_MP4 is only available when using OpenGL (GUI or SHARED_MEMORY).
Thinking of it, we could implement this also in DIRECT mode, using the TinyRenderer to generate images.
Some people log the state of objects, and create the video/playback afterwards.
Another option is to use PyBullet.getCameraImage (which also works in DIRECT mode, it uses a CPU renderer) and write those images to create a video using Python. You can write individual images to disk and use a tool to convert it into a video. OpenAI Gym has also some utility for this, various PyBullet Gym environments provide the 'render' method.
Re: STATE_LOGGING_VIDEO_MP4 in DIRECT mode?
Posted: Mon Mar 19, 2018 10:03 pm
by deepcode
I used the render code from the openAI gym examples, and it sort of works, I have 2 objects loaded, both via urdf/obj files.
Only one of the objects appears in the image even though they are next to each other. This would seem like a pybullet issue, is there a way to make sure all the objects get rendered.
here is the render code I use:
Code: Select all
def _render(self, mode="rgb_array", close=False):
print("opoWTF")
print(mode)
mode ="rgb_array"
if mode != "rgb_array":
return np.array([])
base_pos = [0,0,0]
_cam_dist = 5 #.3
_cam_yaw = 50
_cam_pitch = -35
_render_width=480
_render_height=480
view_matrix = pb.computeViewMatrixFromYawPitchRoll(
cameraTargetPosition=base_pos,
distance=_cam_dist,
yaw=_cam_yaw,
pitch=_cam_pitch,
roll=0,
upAxisIndex=2)
proj_matrix = pb.computeProjectionMatrixFOV(
fov=90, aspect=float(_render_width)/_render_height,
nearVal=0.01, farVal=100.0)
#proj_matrix=[1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, -1.0000200271606445, -1.0, 0.0, 0.0, -0.02000020071864128, 0.0]
(_, _, px, _, _) = pb.getCameraImage(
width=_render_width, height=_render_height, viewMatrix=view_matrix,
projectionMatrix=proj_matrix, renderer=pb.ER_TINY_RENDERER) #ER_BULLET_HARDWARE_OPENGL)
rgb_array = np.array(px, dtype=np.uint8)
rgb_array = np.reshape(rgb_array, (_render_height, _render_width, 4))
rgb_array = rgb_array[:, :, :3]
return rgb_array
Re: STATE_LOGGING_VIDEO_MP4 in DIRECT mode?
Posted: Tue Mar 20, 2018 1:28 pm
by deepcode
if I use ER_TINY_RENDERER, it is not rendering everything.
if I use ER_BULLET_HARDWARE_OPENGL, it works fine. I can't reliably use HARDWARE_OPENGL because I can't control If other computers have hardware OpenGL though. is this a bug with TINY_RENDERER? the object it doesn't display is an obj file that was created with the vhacd tools in bullet. should any bug be filed for this?
Re: STATE_LOGGING_VIDEO_MP4 in DIRECT mode?
Posted: Wed Mar 21, 2018 3:19 pm
by Kukanani
It would probably be useful to have this information in the quickstart guide.