I have been following the core idea behind this very famous quote for most of my life (even before I heard of it)- ‘When the going gets tough, the tough get going’ and it has usually worked very well for me.
If you are restraining yourself from doing the thing(s) you wish to because at some point you were told by someone that the said thing wasn’t for you, you are listening to the wrong voice.