What parameters to tune to stabilize collisions

Post Reply
JJJK
Posts: 3
Joined: Mon Nov 04, 2019 2:07 pm

What parameters to tune to stabilize collisions

Post by JJJK » Mon Nov 04, 2019 4:10 pm

Hi all,

I'm working on opening a door with a humanoid robot using reinforcement learning. To this extent, I created a gym environment that contains a room with a door (URDF description in room_with_door.txt) in which a humanoid robot (URDF description in humanoid.txt, as taken from pybullet_data) is spawned. The environment is included in the runfile (runfile_pybullet_help.txt) as a class definition. The files are uploaded as .txt as the site does not allow me to upload .urdf or .py extensions. To run these files locally, change xml_path in the environment class definition to the local folder in which the URDF files are contained and change the file extensions back to the correct ones (.urdf and .py respectively).

The door joints are controlled by torque control to implement a hybrid feedback loop (handle should be pushed before door can be opened). The robot is controlled using pybullet.POSITION_CONTROL, in which an agent is able to choose the desired joint positions. When testing the functionality of the environment by running it for a random agent, the behaviour is usually as expected (the robot starts flaying its limbs around, as to be expected for a random agent). However, after a seemingly random amount of time has passed, the humanoid pulls a superman (state explodes, see https://vimeo.com/user104809910/review/ ... 6d3efba7cd ), after which the environment is reset due to violating the condition on the cartesian torso/base coordinates.

I hypothesized this has to do with collision forces that start to grow unbounded (providing an impulse to the body), so I tried playing around with the solver parameters, e.g. step size, solver iterations and the default contact ERP to try and get a more accurate calculation of the contact forces and thus a more stable simulation. While I subjectively experienced that it took longer for the state to explode (on average) for certain settings than for other settings, I did not manage to remove the problem completely.

So my questions are
  • What is causing this simulated robot behaviour?
  • Is tuning the physics parameters and adequate solution to the problem, and if so, are there general guidelines for tuning these? (In the user manual, it is only stated that changing the time step induces the need to change the ERP value, which is tuned by default to work for a broad range of problems). Or should I try solving the problem through some other way?
Lastly, note that the robot and the plane are loaded with URDF_USE_SELF_COLLISION and URDF_USE_SELF_COLLISION_EXCLUDE_ALL_PARENTS. I use pybullet.setDefaultContactERP(.9), which is the value found in scene_abstract.py, which I think is used for all locomotion environments? As these also use a humanoid, this value seemed appropriate as an initial guess.

Thanks!
Attachments
humanoid.txt
URDF specification of humanoid robot, taken from pybullet_data/humanoid
(21.39 KiB) Downloaded 15 times
room_with_door.txt
URDF specification of the room containing a door
(10.43 KiB) Downloaded 16 times
runfile_pybullet_help.txt
Python runfile contain environment class
(18.03 KiB) Downloaded 13 times

steven
Posts: 73
Joined: Mon Nov 05, 2018 8:16 am
Location: China

Re: What parameters to tune to stabilize collisions

Post by steven » Thu Nov 07, 2019 11:40 am

thanks for your sharing. i think it should be caused by your selected actions which maybe make the robot jump very high.

JJJK
Posts: 3
Joined: Mon Nov 04, 2019 2:07 pm

Re: What parameters to tune to stabilize collisions

Post by JJJK » Fri Nov 08, 2019 10:09 am

Thanks for the suggestion! Per the manual, I'm trying to do position control (instead of torque control), i.e. (line 141 of runfile):

Code: Select all

pybullet.setJointMotorControlArray(bodyIndex=self.body_name_2_joints['robot']['body_id'],
                                   targetPositions=action,
                                   jointIndices=self.robot_joints_index,
                                   controlMode=pybullet.POSITION_CONTROL,
                                   forces=10*np.ones_like(self.action_space.high))
This maximum force factor 10 has been chosen based on the torques outputted by enjoy_TF_HumanoidBulletEnv_v0_2017may.py, which are in this order of magnitude. Should I decrease the max force even further to prevent the jumping (scaling back to 1 still has the same problem)? Is there a way I can check that it is not caused by unbounded contact forces?

steven
Posts: 73
Joined: Mon Nov 05, 2018 8:16 am
Location: China

Re: What parameters to tune to stabilize collisions

Post by steven » Mon Nov 11, 2019 9:00 am

i just take a test that change the gravity acceleration to setGravity(0, 0, -0.981) and set robot initial position to robot_init_pos=(0, 0, 200) and then after a while we can find that the base velocity sometimes become very large. i also debug this case and find the acceleration of the base and joints sometimes is very large which maybe caused by our random actions. the POSITION_CONTROL mode will try to make our robot to get its target.
so i think it is not caused by the contact force.

steven
Posts: 73
Joined: Mon Nov 05, 2018 8:16 am
Location: China

Re: What parameters to tune to stabilize collisions

Post by steven » Mon Nov 11, 2019 9:37 am

looks not so reasonable. I will check it again later.

JJJK
Posts: 3
Joined: Mon Nov 04, 2019 2:07 pm

Re: What parameters to tune to stabilize collisions

Post by JJJK » Mon Nov 11, 2019 1:04 pm

Thanks for the help! I also tried some other things, including replacing the robot URDF by the visually similar MJCF robot description (included in pybullet_data), which is also used in the example environments. However, when inspecting the wireframes of the two robots in the GUI, the collision shapes for the links (which is what I guess is visualized as the wireframe) seem to differ: while the URDF has a collision shape fully covering the respective link, the MJCF only has smaller spheres close to the joint of the link, leaving the rest of the link uncovered. I can also drag parts the links (around which there does not seem to be a collision shape) into the wall, hinting that this is indeed the collision shape. Could this difference cause the high accelerations?

Using the MJCF description solves all the issues mentioned before, so I can continue my work.

steven
Posts: 73
Joined: Mon Nov 05, 2018 8:16 am
Location: China

Re: What parameters to tune to stabilize collisions

Post by steven » Tue Nov 12, 2019 12:55 am

did you only change the robot description to the pybullet_data/mjcf/humanoid.xml then it can work now? did you have any other changes? could you also attach them also? thanks.

steven
Posts: 73
Joined: Mon Nov 05, 2018 8:16 am
Location: China

Re: What parameters to tune to stabilize collisions

Post by steven » Tue Nov 12, 2019 7:53 am

hi all,
i have another question not specific to this case, where does the motor constraint force come from? this force will increase the system energy? thanks.

steven
Posts: 73
Joined: Mon Nov 05, 2018 8:16 am
Location: China

Re: What parameters to tune to stabilize collisions

Post by steven » Thu Nov 14, 2019 7:14 am

hi JJJK,
i still suspect that this is caused by the non-suitable actions. i find the every joint of the human description with URDF format doesn't have any limitation while in the MJCF format it does.
hope you can update your latest demo.
thanks.

Post Reply