Given that I'm generally doing short distances (although with vehicular cycling sprints and speed when without the kids) and work attire is combats (I like the pockets) and t-shirts, I have avoided cycling clothing on the whole.
Exceptions have crept in - waterproof jackets are now vented shells. Last one was cycle specific, current one is more generically "active". Waterproof trousers are cycling specific, but then used for everything - the calf and ankle are much slimmer fit to avoid catching in the cranks/chain.
Finally, I did switch to cycling shoes (and changing) as I was finding that the commute was wrecking my brogues (sole breaking on stabilising foot).
I do find that trousers are wearing out sooner, due to cycling specific holes, rather than wearing out at the knees. That might suggest cycling specific trousers eventually, were the combats not quite so cheap. I've just ended up with a large, disproportionate pile for "rough" work. It was much easier on the farm where good clothes could end their days as farm clothes, but when there isn't much rough/muddy/oily work happening in proportion to the cascade it seems wasteful!
Come to think of it, these issues are indeed a function of distance cycled, but not acute distance, rather chronic distance, and even a small commute, carried out daily, will take its toll on "ordinary" clothes.
Maybe the solution is found in Brooks instead of cycling specific clothes?
Robert