Well, yes, memories can be stored outside the brain. For example, when you write a note to yourself on your phone, or when you write a diary. But you have to access those memories through your senses. Maybe one day soon we will have brain-computer interfaces that will allow us to store and access memories in computer chips. But there are no magical, non-material reservoirs of information outside the brain.
As for the second part of your responses, it is established that memory is stored in the brain as synaptic plasticity, that is, increases and decreases in the efficiency by which some synapses transmit information. What happens is that that information is not localized, but stored in several synapses in different neurons. It is the pattern of synaptic activity what encodes the information. Still, you can bring up specific memories by electrically stimulating specific neurons in the brain.
