physical audio

Dylan Menzies
Posts: 18
Joined: Sat Nov 03, 2007 10:40 pm

physical audio

Post by Dylan Menzies » Sat Nov 03, 2007 11:49 pm

Hi, I am new on this forum. I have developed a library for integrating physically modelled sound with rigid body engines, which goes back a few years, although its been gathering dust. The motivation for physically modelled audio is that its a lot more realistic and interesting than sample based sound, and is a natural match for macro-physical behaviour. Together these two dynamics cover the perceptual bandwidth.

Here are some demos with an old engine. They will probably run too fast..
http://www.zenprobe.com/dylan/project/phya

I'm considering turning this into an open source project, bullet looks like a good candidate for providing some new demos, and maybe later more integration. First thing that grabbed my eye was the comprehensive contact coverage, including proper cylinders, and then mention of persistent contacts, and seperate impacts.

Anyways my experience is that engines vary quite a bit in their usability with physical audio, so I would like some opinions. The main requirement is to find when engine contacts are made or broken, what their current dynamics are. If they are persistent great, if not this is not usually a disaster, provided the body pair lists for contacts can be scanned.

Any comments appreciated

Dylan Menzies

Dylan Menzies
Posts: 18
Joined: Sat Nov 03, 2007 10:40 pm

Re: physical audio

Post by Dylan Menzies » Mon Nov 05, 2007 7:05 pm

To follow up the previous message, I have some specific questions-

- The btBroadphasePair passed to a customNearCallback appears persistent. Would it be unreasonable to request a userdata option on this object to allow cross referencing to acoustic contact objects?

- Can persistent contact data also be found from a btPersistentManifold in the customNearCallback? There doesn't seem to be a userdata option here either.

The alternative would appear to build a linked list of (btBroadphasePair, acoustic contact).

thanks for any help.

User avatar
Erwin Coumans
Site Admin
Posts: 4157
Joined: Sun Jun 26, 2005 6:43 pm
Location: California, USA
Contact:

Re: physical audio

Post by Erwin Coumans » Mon Nov 05, 2007 8:54 pm

Hi,

Thanks for sharing this info, that sounds pretty cool. It would be great to have a demo combinding physics-based sound and Bullet collision/rigidbody dynamics.

The btBroadphasePair passed in a customNearCallback is persistent, and has a user info pointer. However, under certain conditions this user-info pointer is used by the rigidbody dynamics system internally (and destroyed during the gContactDestroyedCallback, see around line 110 of bullet\src\BulletDynamics\ConstraintSolver\btSequentialImpulseConstraintSolver.cpp.

It should be possible to either use this user-info pointer in btBroadphasePair , or extend the system to make it easier to add a user pointer (with added/destruction callback).

Are you using BulletCollision collision detection only, or do you combine it will BulletDynamics rigid bodies?
Thanks,
Erwin

Dylan Menzies
Posts: 18
Joined: Sat Nov 03, 2007 10:40 pm

Re: physical audio

Post by Dylan Menzies » Mon Nov 05, 2007 9:57 pm

I think btBroadphasePair user pointer would be the best from an integration viewpoint. It would be need to be maintained for the duration of the contact, so I guess that would not be possible as it stands. I'm not clear when a btBroadphasePair begins. It seems to be just before contact. This is ok, but its better to avoid using acoustic contacts that are not needed, so maybe something using the persistent contact manifold might be better.

At the moment I am just trying to figure out the best approach. I would intend to use the dynamics in bullet initially anyway. Visual inspection of the cylinders spinning down shows they behave well, and should give good acoustic results. They seem to switch off a bit too suddenly though, and also sometimes the spin down near the end is too slow, ie the rolling friction isn't totally working. In reality the spin down tends to accelerate in this last phase. Still, I don't think this would be a big deal. Physical audio is good with smooth surfaces - you can really 'hear out' the dynamics. If a surface is approximated with triangles you can get bumpy results unless you filter the dynamics. One way is to put filter switches on surfaces to describe the degree of smoothing.

stenyak
Posts: 29
Joined: Fri Aug 12, 2005 4:59 pm

Re: physical audio

Post by stenyak » Mon Dec 03, 2007 10:33 am

Are there any news? I'm pretty interested on realtime generated audio support for use with physic libraries.

Dylan Menzies
Posts: 18
Joined: Sat Nov 03, 2007 10:40 pm

Re: physical audio

Post by Dylan Menzies » Mon Dec 03, 2007 2:56 pm

I have committed to doing this, but I have a lot of other things to do.. Hopefully won't be long.

stenyak
Posts: 29
Joined: Fri Aug 12, 2005 4:59 pm

Re: physical audio

Post by stenyak » Mon Dec 03, 2007 4:37 pm

Ok, thanks for the update!

Dylan Menzies
Posts: 18
Joined: Sat Nov 03, 2007 10:40 pm

Re: physical audio

Post by Dylan Menzies » Fri May 16, 2008 2:31 pm

Ok, I 've been doing some integration and its going ok, I have physical audio running in bullet. The problem I have right now is how to reset a reference to an audio contact from a dynamic pair, when the pair is no longer appearing in the collision call back, because objects have moved apart. For instance is there a callback for pairs that are being released?

Dylan

Dylan Menzies
Posts: 18
Joined: Sat Nov 03, 2007 10:40 pm

Re: physical audio

Post by Dylan Menzies » Fri May 16, 2008 3:40 pm

It looks like I have a choice of two appraoches for handling contacts- generating audio contact interactions on a per body pair basis or per manifold point pasis. Both appear to be persistent and have user data.

The manifold point approach seems better supported with callbacks, but potentially could lead to uneccessary costly audio detail, or maybe better detail.

The pair approach seems to have less support for callbacks, such as pair removed, but provides a way of condensing physical audio interactions.

mmm. Doese this make sense? Are there things I am overlooking here before I blindly start hacking around? With other engines I have taken the pair approach, but here the manifold point may be better.

User avatar
Erwin Coumans
Site Admin
Posts: 4157
Joined: Sun Jun 26, 2005 6:43 pm
Location: California, USA
Contact:

Re: physical audio

Post by Erwin Coumans » Sat May 17, 2008 11:10 pm

Each overlapping pair (btBroadphasePair) can have one or more persistent contact manifolds (btPersistentManifold) created by its btCollisionAlgorithm. Each contact manifold has up to 4 contact points.

But right now it is hard to iterate/traverse those manifolds, given a btBroadphasePair, so it is easier to use contact point callbacks.

If necessary we can make it easier to get access to contact manifolds/contact points from a btBroadphasePair.
Obviously, you can modify the code to suit your audio needs, and let us know what improvements you recommend,

Hope this helps,
Erwin

Dylan Menzies
Posts: 18
Joined: Sat Nov 03, 2007 10:40 pm

Re: physical audio

Post by Dylan Menzies » Sun May 18, 2008 10:26 am

thanks for your reply. :)

I will try tracking manifold points -mps first.

I think I have found a bug. I have been listing mps using iteration each step and comparing with callbacks set with gContactAddedCallback gContactDestroyedCallback. I find calls to AddedCallback are completely inconsistent with the iteration monitor, although DestroyedCallback works fine once user data is set in the monitor. DestroyedCallback is the most important, so should be able to make this work without too much fuss.

I take it the user data on mps is never used by dynamics?

With complex objects what happens with multiple areas of contact, does this generate multiple manifolds?

I guess m_appliedImpulse acts along m_normalWorldOnB. Is there a friction force perpendicular to this or not?

Dylan

Dylan Menzies
Posts: 18
Joined: Sat Nov 03, 2007 10:40 pm

Re: physical audio

Post by Dylan Menzies » Sun May 18, 2008 6:42 pm

I have it working now with audio contact on each manifold point, however this could be costly. To make it work with an audio contact per manifold, I would need a callback for manifold deletion, and user data in the manifold. Maybe you can point me where I should add the callback in bullet.

Dylan

User avatar
Erwin Coumans
Site Admin
Posts: 4157
Joined: Sun Jun 26, 2005 6:43 pm
Location: California, USA
Contact:

Re: physical audio

Post by Erwin Coumans » Mon May 19, 2008 2:24 am

I would need a callback for manifold deletion, and user data in the manifold. Maybe you can point me where I should add the callback in bullet.
You can derive your 'audio' version of btCollisionDispatcher, and override the following virtual methods (and call the base class):

Code: Select all

	virtual btPersistentManifold*	getNewManifold(void* b0,void* b1);
	
	virtual void releaseManifold(btPersistentManifold* manifold);
I think I have found a bug. I have been listing mps using iteration each step and comparing with callbacks set with gContactAddedCallback gContactDestroyedCallback. I find calls to AddedCallback are completely inconsistent with the iteration monitor, although DestroyedCallback works fine once user data is set in the monitor. DestroyedCallback is the most important, so should be able to make this work without too much fuss.
Can you explain what you exactly mean by 'iteration monitor'? btPersistentManifold is a contact point cache, and it can contain 0,1,2,3 or 4 points. Those contact points are added, removed and updated every simulation step.
I take it the user data on mps is never used by dynamics?
Indeed. You can use m_userPersistentData in btManifoldPoint for your audio purposes.
With complex objects what happens with multiple areas of contact, does this generate multiple manifolds?
Indeed, it is possible that complex or composite objects generate multiple contact manifolds. If you need it, we can add a manifold iterator given a btBroadphasePair.
I guess m_appliedImpulse acts along m_normalWorldOnB. Is there a friction force perpendicular to this or not?
Yes, it is the applied impulse along the contact normal. In the past we also cached the applied impulse along the two friction directions. We can easily add this if you need it.

Hope this helps,
Erwin

Dylan Menzies
Posts: 18
Joined: Sat Nov 03, 2007 10:40 pm

Re: physical audio

Post by Dylan Menzies » Thu May 22, 2008 10:52 am

Erwin Coumans wrote:
I would need a callback for manifold deletion, and user data in the manifold. Maybe you can point me where I should add the callback in bullet.
You can derive your 'audio' version of btCollisionDispatcher, and override the following virtual methods (and call the base class):

Code: Select all

	virtual btPersistentManifold*	getNewManifold(void* b0,void* b1);
	
	virtual void releaseManifold(btPersistentManifold* manifold);
Ok, thanks.
I think I have found a bug. I have been listing mps using iteration each step and comparing with callbacks set with gContactAddedCallback gContactDestroyedCallback. I find calls to AddedCallback are completely inconsistent with the iteration monitor, although DestroyedCallback works fine once user data is set in the monitor. DestroyedCallback is the most important, so should be able to make this work without too much fuss.
Can you explain what you exactly mean by 'iteration monitor'? btPersistentManifold is a contact point cache, and it can contain 0,1,2,3 or 4 points. Those contact points are added, removed and updated every simulation step.
I just mean taht I iterate to find out the current mps each step, and compare with calls to AddedCallback and DeletedCallback. Calls to AddedCallback are occuring even when the mps are remaining unchanged, and the mps passed by the callback are different.

I guess m_appliedImpulse acts along m_normalWorldOnB. Is there a friction force perpendicular to this or not?
Yes, it is the applied impulse along the contact normal. In the past we also cached the applied impulse along the two friction directions. We can easily add this if you need it.
Its not needed right now, I just wanted to understand the layout.
Thanks
Dylan

Dylan Menzies
Posts: 18
Joined: Sat Nov 03, 2007 10:40 pm

Re: physical audio

Post by Dylan Menzies » Fri May 23, 2008 3:36 pm

A question about m_appliedImpulse : If the same simulation is run at twice the time step size, the corresponding values of m_appliedImpulse should be twice as big, so the average force over a step is found by dividing by the step size?

Post Reply