Ok, I think I know what's wrong. It's probably documented someplace obscure..
I'm pretty sure the issue is when the ball makes contact with two parts of the ramp at the same moment. When this happens, it appears to be solving the situation by applying corrective forces for each collision point and then adding them together or something like that to come up with the final force vector that is applied to the ball.
The issue is that if the front of the ball is deep inside a part of the ramp, the engine sees that as a correction in need of a large counter force. While the bottom of the ball may be penetrated into a ramp only a small amount, so it gets a relatively small correction. When the forces are added together, the big forward penetration's correction force "wins" so to speak, and so makes up the majority of the vector of the ball afterward. This translates to the rather unnatural case of the ball seemingly hitting a wall or flying more vertically than it seems like it should (sometimes going back the way it came, adding a weird spin, whatever).
Nature tries to minimize energy usage, and a change in vector requires using some energy. What should really happen in reality is more along the lines of: when the collision of two points occurs, the collision which takes the least amount of energy is rectified first. In this case the ball's bottom collision point would be resolved and thus the ball is pushed up slightly. This changes the collision at the front of the ball which would need to be recalculated before any corrective forces are applied (if the collision even happens at all after the adjustment). My guess (without understanding much of any of the bullet internals) is that it doesn't work this way in the engine. Therefore multi-point contacts are, essentially, undefined and results in unrealistic behavior.
Is that even close to right? Not being very good at this kind of low level stuff (and having more of an intuitive sense of physics than an actual scientific one), I could be laughably wrong.
Assuming any of this is true, is there a real fix for this? I got slightly better results trying to replace my ramp with a btCylinderShapeX after I decided it might be a multi-contact issue. It kind of works, but of course it doesn't match the real shape or behavior of the ramp that was intended so it doesn't feel entirely right. But it's closer and at least seems to have less problems with the ball acting strange at varying speeds. (Although for some reason it seems like my ball has a huge spin or something sometimes because it'll hit the cylinder and then kind of shoot off to the left or right - but not all the time, just at certain speeds. Very weird.)
</ramble>