The way I like to think of the whole collision masking situation is as follows. Think of the group as what ever group the object belongs to and the mask as a list of groups that object can collide with.
Internally Bullet just does a bitwise & with the mask and group of either object respectively and if the result isn't zero they will collide. You can have objects in multiple groups but in general I would suggest limiting the number of groups an object belongs to to just one to keep it simple since if just one of the groups collides with one in the mask the collision will occur even if the other groups the object belongs to are not supposed to collide.
If I have understood your question correctly you would want to do something like the following
Code: Select all
#define BASE_GROUP 1<<0
#define WHEEL_GROUP 1<<1
#define BALL_GROUP 1<<2
const short ball_group = BALL_GROUP
const short ball_mask = BASE_GROUP | WHEEL_GROUP | BALL_GROUP ///< assuming the ball collides with everything
const short wheel_group = WHEEL_GROUP
const short wheel_mask = BALL_GROUP ///< assuming we only want to wheel to collide with the ball
dynamics_world->addRigidBody(ball_body, ball_group, ball_mask );
dynamics_world->addRigidBody(wheel_body, wheel_group, wheel_mask);
If you want to change these groups while the objects are simulating my understanding is you must remove them from the world and re-add them with the new groups.
Does that answer your question?