If my only evidence was that I saw it raining on the window yesterday, and the day before (and so on) then I have no evidence. I would be deducing that it was raining outside because I deduced yesterday it was raining outside! Perhaps it was just a piece of gardening equipment which watered the grass, or a leaky pipe?
Further, your brain seems to use conditional (or Bayesian) probability with this approach. It has a ‘distribution’ for the meanings of a word and how likely it is to be used in a certain way. For instance, if I say the word ‘red’ you get a sense of what its possible meanings are. Given the overall shape of the sentence and the other words, it then adjusts what it is most likely to mean. If say ‘I hit him, and he saw red’ I am referring to ‘seeing red’, i.e. that he was angry. It is possible he just saw something red, but given I hit him, this is unlikely. Instead, if I say ‘I thought the paint was pink, but he saw red’ this is (probably) referring to him seeing the colour red, not getting angry.