Erwin,
If you were to meet the devs responsible for the patent application in a bar in Dublin over a glass of Guinness, they'd probably tell you, off the record, that they get paid a bonus for every patent app and more if/when the patent is granted. It's part of corporate 'legal marketing'; 'we have n physics related patents', used to promote the company for investors as well as defend against competitor legal action. It's true that many patents can be invalidated by prior art, but as EK noted, (any/all) litigation is expensive, and lawyer time to research each patent and try to invalidate the claims costs money. Having many, effectively worthless patents has value as time and money must be spent for invalidation: kind of like disposable armor.
The physics related patent claims that I have read are so complicated, and have so many detailed claims; it is relatively simple to work around the claims. I would not worry about getting a suit filed against the Bullet project: with so many developers aware of the source code and prior art, the public/media effects would be costly for the litigator. Additionally, if the prior art is clear and obvious, help is likely available at no cost via the FSF and similar organizations.
My experience with patent litigation and the legal system in general is that the system itself is quite simple compared to writing real-time physics software. Patent claims are simple Boolean algebra- easy to understand as a form of basic math (even Markush claims), and legal process is also relatively simple: file the proper papers with the proper format and go to the next level. For a software developer with no experience in this area, it can appear daunting. Honest lawyers (

) will tell you that their job is really pretty easy, it just takes time to do the research and file the papers, correctly formatted and on time. Their experience in knowing what is important, and where to focus time is what can justify their high cost.
Since my physics tech is closed source, and I have published the major concepts which I believe someone might try to patent, I'm not worried about a software patent suit. Additionally, the aggressive party won't just hit a software developer with a suit; they'll first send a letter claiming infringement, to which you can respond. If you respond with clear, obvious, incontrovertible proof of prior art, the issue could go away, and never become public. If they press the issue, you can make it public, and ask for community support. I would certainly help in such a case. Another option would be to change your code slightly so that the aggressor no longer believes you are infringing (even if you have proof of prior art): take the path of least resistance. The most likely patent actions will be between Havok, Ageia, nVidia, and AMD. Small devs and open source projects won't likely even be a blip on their radar, unless it somehow costs them money, or makes their competitor(s) money.
The patent system is a form of business warfare. Arm yourself with knowledge and don?t be intimidated. The absolute worst case scenario is you?d have to change or remove a piece of code from your library. There?s always another way to solve the problem, especially since fundamental (broad) patent claims aren?t being issued for physics technology (a ?one-click? type patent claim in the physics space would be hard to get pushed through the PTO; if someone did manage to come up with something broad and simple, with no prior art, all the power to them for getting a patent and protecting it).