Why do you say there is a fee for trading when outside the free market of the EU? For a German, trading with Britain as opposed to France involves the fee of exchange rate conversion and risk. But this won’t change. Perhaps you means tariffs levied on non EU members? That such tariffs exist means that the EU Common Market is not a “free market” (and it’s not). You should use different terminology; mostly people talk about the Single Market or the Common Market. In that case, you also need to realise that once Britain joined the Common Market, it moved behind those tariff walls (which were once very high, in the days of the butter mountains and wine lakes). Britain’s trade inside the common market grew of course, since it was now “in” the castle. But it’s trade outside the tariff walls fell … buying Australian apples, for example, became much more expensive because EU tariffs and quotas now applied. So Britain bought apples from European orchards (presumably inferior or more expensive, since they weren’t bought until the new EU tariffs made the Australian apples artificially more expensive).
If Britain now moves outside the tariffs walls, such as they are, trade to Europe will fall: those tariffs which once punished EU consumers for buying products made outside the EU will now punish them for buying British. Of course, tariff walls are subject to WTO rules and they are much lower now. But there are plenty of prosperous trading nations not in the EU.
Britain was a great trading nation before joining the single market, and it has long been a proponent of free trade, having been an early pioneer in tariff barriers and their problems (see Corn Laws). Just look at Australia: the apples just got sold elsewhere. And there are a lot of Japanese cars on European roads.