Real-Time Physics Simulation Forum

Re: Great Physics Engine Comparison (PEEL)

2013-08-13T09:41:55+00:00

Not taking into account the emotional stuff in this thread,
I can mention that I struggle with the sweep test performance in Bullet still, and don't know how to optimize it. I did test in Blender(bullet) vs Unity(PhysX), testing a sphere against a highpoly mesh(unity supports max 65k meshes). Unity performed better, much better, although definitely not 300x faster, maybe about 5x-10x. I also noticed sweeping rotated primitives(e.g. cylinder)in blender is much slower than when the body is unrotated and swept just on z axis.

I know bullet has great performance, I've seen the results many times. Also, Physx(in Unity) doesn't support so many basic shapes, which I need. So I am still thinking what might be used to optimize the results for sweep test?

Statistics: Posted by pildanovak — Tue Aug 13, 2013 9:41 am

Re: Great Physics Engine Comparison (PEEL)

2013-06-11T06:35:46+00:00

On the positive side, the benchmark looks well engineered at first glance, and I hope it will be open sourced (or duplicated).

I can't agree 100% with that one, since you can see some very large deflections in the convex stack with physx that aren't in bullet, as well as joint results that vary drastically between tests. Unless they are willing to rerun simulations to a certain accuracy (i.e. meshes don't sink into each other when stacking, joint angles don't vary drastically between tests, etc) or open source it so others can do it for them, there isn't too much useful information (as you mentioned).

For those of us concerned that Erwin and the other contributors might have run out of things to work on, a good benchmark can give them some more targets to work on :)

All for that! Hopefully they both throw the academic side a bone with featherstone or another more robust solving method, that's practically the only thing keeping bullet from overtaking ODE within ROS, and what's keeping physX inside the microsoft robot simulator box. I know a dozen labs that would love to have it, and probably a hundred times that would likely use it rather than making their own.

Statistics: Posted by Basroil — Tue Jun 11, 2013 6:35 am

Re: Great Physics Engine Comparison (PEEL)

2013-06-10T02:04:00+00:00

I would have been more impressed with the benchmark had it been posted objectively and dispassionately. Instead the grandstanding lingo, "spectacular", "embarrassing", etc. really clouds the whole thing for me. Additionally, the author posts in this thread with more off-putting comments, which only adds to the negative impression.

On the positive side, the benchmark looks well engineered at first glance, and I hope it will be open sourced (or duplicated). For those of us concerned that Erwin and the other contributors might have run out of things to work on, a good benchmark can give them some more targets to work on

I feel compelled to add that I don't think the benchmark means that much. It reminds me of the various Javascript browser benchmarks which mean little because most people are not running javascript to calculate fibonacci numbers in the browser. For my physics library use, my own case does not involve running 1k simultaneous sweeps or raycasts or whatever. In fact, the point that Bullet is mostly in the ballpark in terms of performance, and fully functional, and open source with a liberal license, makes it an easy choice for me (and a lot of others apparently).

Statistics: Posted by quantum — Mon Jun 10, 2013 2:04 am

Re: Great Physics Engine Comparison (PEEL)

2013-05-25T13:25:28+00:00

there was a time not so long ago where posting on the Bullet forum lead to enlightening discussions instead of flame wars.

Contributing knowledge to advance the field and help the community is one thing; creating a blog post to put down an open source engine in order to prop up your own closed source one because you are "pissed off" at the "clueless morons on the internet" is another. The later is bound to attract some flack.

I agree all should be concerned about advancing the field instead of flaming, yet when one starts, as Pierre did, because of their passion, it's normal to expect others to weight in; I hope you realize you also did just that now...

Statistics: Posted by RBD — Sat May 25, 2013 1:25 pm

Re: Great Physics Engine Comparison (PEEL)

2013-05-24T09:16:40+00:00

I for one is really happy that Pierre published his results, regardless of supposed biases, there was a time not so long ago where posting on the Bullet forum lead to enlightening discussions instead of flame wars.

This should be a technical discussion, because the reality is, most real time physics users are happy with ten crates and two rag-dolls, in which case their concern is API / framework , stability, tool support, etc..., not performances.

In any event, take my word for it, PhysX is competitive and it is getting better to the point where more and more of my internal benchmarks includes a PhysX data point in order to judge performances, this was not the case few years ago (before Pierre's code kicked-in I would guess:)).

Nat.

Statistics: Posted by Nathanael — Fri May 24, 2013 9:16 am

Re: Great Physics Engine Comparison (PEEL)

2013-05-17T18:45:30+00:00

I am not sure. I remember there was some global constant you can set to false. I would check in the sleeping logic in btRigidBody and see if you can find it there. Also check the benchmark tests in Bullet. I think Erwin disables sleeping there as well and this might be the best way. The way I am describing here might be deprecated though.

Statistics: Posted by Dirk Gregorius — Fri May 17, 2013 6:45 pm

Re: Great Physics Engine Comparison (PEEL)

2013-05-16T15:33:55+00:00

Ok, I found the issue. This line is guilty:

Code:

body->setActivationState(DISABLE_DEACTIVATION);

As written in the posts, I disable sleeping (in all engines) to make sure it does not interfere with benchmarks. Well, if I do that, the setForceUpdateAllAabbs(false) call has no effect anymore. Is there a better way to disable sleeping for benchmark purpose?

If I do not call setActivationState, the scene does run faster. However it still takes about 5ms here at work, on a PC which is usually about 2X faster than what I have at home (where the 34ms were recorded). I will try at home later and report the new number (*). It will likely be better than 34ms. We probably agree though that 5ms is still "too much" for simulating an empty scene.

In any case I am not sure why the sleeping stuff has an impact on static objects. That sounds counter intuitive.

(*) EDIT: it takes 16ms now at home (instead of 34), with sleeping enabled and AABB updates disabled. This is certainly better, but the problem is basically still here.

Statistics: Posted by Pierre — Thu May 16, 2013 3:33 pm

Re: Great Physics Engine Comparison (PEEL)

2013-05-16T10:21:58+00:00

I have showed you this for one test (the 255*255 fixed boxes), but you have been quiet about that.

At best you have showed me that you cannot use the Bullet SDK properly

I reported in this thread that your suggested fix did not work. People just can't even read - like the guy telling me to try Havok.

You have the code. Please tell me how to fix the sweep test scene "without changing the SDK". As I said, I would happily update the blog posts with new results, if it turns out I misused the SDK.

Statistics: Posted by Pierre — Thu May 16, 2013 10:21 am

Re: Great Physics Engine Comparison (PEEL)

2013-05-16T08:54:12+00:00

All I am saying is that several of your most dramatic results can be fixed by the user, without changing the SDK.

I have showed you this for one test (the 255*255 fixed boxes), but you have been quiet about that. I am pretty sure that the wost-case sweep benchmark can also be fixed, without changing the SDK.

At best you have showed me that you cannot use the Bullet SDK properly You could complain that the documentation is bad or lacking, and I agree with that.
Thanks,
Erwin

Statistics: Posted by Erwin Coumans — Thu May 16, 2013 8:54 am

Re: Great Physics Engine Comparison (PEEL)

2013-05-16T08:27:13+00:00

Ok.... I'm not following.

The test you mention is not "cherry picked". Cherry picking would be publishing results for just the sweep tests where PhysX is faster, conveniently ignoring all the other ones. But this is not at all what happened. PhysX was faster in all the raycast or sweep tests that I tried. Why should I ignore the most dramatic results?

I really don't understand your point. It does not matter that X or Y can be "trivially fixed". Everything can. What's the point of benchmarking at all if you ignore all the problems that "can" be fixed in a future version? Of course it can be fixed. And maybe it will. It does not change the fact that contrary to what I read online several times, the current version of Bullet is not "faster on the CPU". This is exactly the reality check I wanted to provide, and spectacular results are a good way to drive the point home.

As for the "proper analysis", I'm sorry but... you can't be serious. I don't have time to do a "proper analysis" for all the issues in all the engines supported by PEEL. Are you really saying that the results are somehow questionable because I did not do a proper analysis of why it was slow in Bullet? This is a bit of an unbelievable statement. There are results. You are free to replicate the test and investigate the issue yourself, if you feel there is something fishy that should be fixed.

one engine performs a sweep test that handles both translation and rotation

Fair enough. Is Bullet supporting rotations when the given transforms are just for a regular linear cast? That could be one explanation, certainly. But even so, I still don't see why it invalidates the results. Most people use linear casts only, so it is fair to ask how much of a performance hit you get for supporting rotations. If it does make everything orders of magnitude slower, well, people should be aware of this. It would make the decision of supporting rotations very questionable to me, for example.

In any case I agree that this discussion is not going in the right direction. Going back to coding now.

Statistics: Posted by Pierre — Thu May 16, 2013 8:27 am