The 'problem' is (perhaps) the word Lycra.
Apart from the pejoratively associated word lout, there is somehow an assumption of yellow or roadie or(?)
There's probably Lycra in your underwear, jeans etc. already!
There are 'cycle specific' clothes for good, and other, reasons.
Key points are comfort, comfort (as in being happy 'to be seen' wearing them), waterproof, and de-sweating.
The latter is only an issue (for 'normal' people riding bikes) if you get overheated - either by wearing too many layers (not stopping to take one off) or cycling 'too fast' for your clothes or standing in the world.
Of course the last bit relates to whether you leave one place in enough time to get to the next 'comfortably'. Are there two classes of cyclist - organised and late?
People new/returning to cycling have the added problem of not knowing their fitness, (even 'fit' people can find cycling difficult/different because of using unfamiliar muscles), capabilities - speed/distance etc.
Some people decide it's 'too difficult for me' and give up.
Some do that after buying 'all the right gear'.
So it's a bit like the advice on the current new bike thread/DIY - do it in steps (@wingpig above) and do what suits - jeans and Ts, business suit, full-on TdeF clone - with or without 'change when you get there' options.