I sometimes think about passing distances as I filter up, and note that I'm happy to pass by with 2 inches to spare but that I'm most upset when someone passes me with 2 feet to spare.
But there's 2 key differences as I see it.
In filtering, you are the party that is moving faster and is in control of your route and progress. You make a decision to progress based on the condition of road, space, speeds, if there is space ahead etc. When it comes to passing, someone else makes the decision for you, there's very little you can do to get out the way if they make the wrong decision. I'm happy to go up the side of a lorry (wait for it) if the thing is stationary, if I can see a clear cycle lane route up the side and a clear exit infront that will take me beyond the vehicle and if I know the junction well enough that I know I have time to make my maneouver and get clear. I know the lorry can't suddenly lurch a foot to the left when its static. I'm also more than happy not to make that decision and to stay put if I can't satisfy the above criteria. However if the lorry is moving and making a close pass, I've no way of knowing it isn't going to make a sudden lurch towards me and I've no way of getting away from it if it does.
Secondly, as mentioned above, it's simple physics. When you filter, you're probably doing no more than 10mph past a queue of stationary and slowly moving vehicles. The biggest danger to watch out for is the driver that suddenly decides to switch lanes without indicating or checking mirrors. When you're passed by traffic, you're probably still doing at least 10mph (plus) but the other vehciles quite probably twice if not thrice your speed. Even the most careful driver has nothing like the control over their vehicle as a cyclist does and situational awareness inside the glass and steel box, with all its distractions, restricted visibility and restricted hearing conditions means that the situational awareness is never going to be as acute as a cyclists. Contrast and compare what 30mph feels like on a bike (massive endorphin rush akin to almost breaking the sound barrier) to 30mph in a car (dull, boring, monotonous, rumbly feeling).
Steveo's solution is very neat - a faster vehicle needs to give more room so that firstly it reduces the impact it has as it passes you (close passes by buses and lorries can feel like getting blown off the bike by the wind) but also it allows for the fact that at higher speeds, reaction distances are greater and the results of any contact are exponentially worse.