"safer than not cycling"
Well there's a nice blanket statement!
@Puzzle - cycling is certainly a lot safer than some of your associates seem to imagine!
Don't know whether they think that all drivers are out to get cyclists - or just you in particular(?)
Perhaps they think you are unsafe or maybe incompetent (or just 'inexperienced').
The latter is clearly true - but that doesn't mean you are unsafe or that it's unsafe for you to be on roads.
I trust your bike is safe and adjusted to fit you so that it is comforfable to ride and safe to operate (particularly brakes).
I assume you understand the 'rules of the road' (mostly as laid out in the Highway Code).
It's always wise to expect the unexpected - car doors opening, poor road surfaces, wayward dogs and pedestrians - especially on 'safe' cycle paths.
The 'view' of this forum (though not all its members!) is that wearing of helmets and hi-viz stuff is "personal choice" - 'we' have had 'discussions' and have an unwritten rule 'not to go back there'. It seems to be one area of 'safe' that there are entrenched views - a bit like those of some people you know about cycling generally!
There are roads/roundabouts etc. where even experienced cyclists don't feel comfortable. As people cycle more they get more confident (over confident is dangerous). There are usually routes avoiding the worst places (not just off road paths - though you may find ones you didn't know about). Play with http://Edinburgh.CycleStreets.net and try some of the suggestions - part of the fun of cycling is exploring and not having to go the same way every day.
You are free at any time to get off and push your bike if this seems your best option.
It is (statistically) the case that people who cycle live longer (perhaps what wfb was partially alluding to). Some people who cycle get injured (and killed) - but so do people who walk, drive, mow the lawn, climb stairs, go hang-gliding etc.
Caution, equipment, training, practice, experience etc. generally increase 'safety' - so assess your risks and keep cycling! (Though you sound as though you don't need most of this advice - you've muddled through and joined the other ex-novices.)