control robot using jupyter notebook with threads

[kfa22]

Hi,

I am controlling a minitaur environment from jupyter notebook, and want to experiment with different gaits. For this, I have set up several slider widgets, one for each leg motor. Every time I move a slider, a new action array is sent to env.step(action_array), which causes the simulation to move one step.

Now I would like to make it realtime, but as soon as i enable pb.setRealtimeSimulation(1) the robot falls over. I am guessing it needs to receive an action array continuously, not just when my slider values change.

My attempt to do this was:
a) create an infinite loop that reads the latest action_array and calls env.step(action_array) - I assume this would stabilise the robot and keep it fixed in position
b) have a seperate thread for the sliders, which updates the action_array

I used python multiprocessing to create the infinite loop, passing variable env to a worker, which calls env.step, reading action_array from a Queue, but the simulation does not step. Elsewhere I put an action_array on the queue. I would expect the worker thread to read action_array from the queue, and run env.step. If there is nothing on the queue, it will just sent latest_action_array.

Could someone help, maybe I am making it too complicated.

Many thanks,
Kai

[kfa22]

Found a solution that kind of works, see attached jupyter notebook - it provides a way to control Minitaur motors with Sliders in realtime, and plots resulting angles in realtime.

It uses https://github.com/kirill888/jupyter-ui-poll to request information from sliders in realtime, and convert to an action array.

Live graphing is done via: https://github.com/matplotlib/ipympl. The frame rate is not great (about 1 fps), but blitting doesn't seem to work in jupyterlab yet.