I’m standing under the porch of our old redbrick house on the outskirts of London, a rickety fence to the left propping up unwieldy roses, and in front of me, my nana crouched down with her hands resting on her knees, smiling encouragingly at me to walk toward her. She’s wearing a red cardigan and tan-rimmed glasses, her light-colored hair curly and neat. The lines on her freckled face are vivid, and they crinkle around her eyes as she beams up at me.

My memory lied to me.

This is one of my earliest and fondest childhood memories, from when I was four, but it’s also tinged with sadness. Years later, after my grandmother died, I was leafing through photos stashed in a box above the fridge. Suddenly, there she was on the glossy paper, with the same joyful expression—the neat curly hair, bright beaming face, those wrinkles, her beautiful smile.

My memory lied to me. My memory told me I saw her face in that moment on our front porch, but actually I had remembered it from this photograph. I felt confused and disappointed, mingled with something that felt like grief — like the moment I realized a boyfriend had been cheating.

This concept of misremembering a moment from youth is a common, calamitous feature in novels, but it turns out that many of us are unreliable narrators of our own life story. Around 40 percent of us have a fictional first memory, according to a new study by the Center for Memory and Law at City, University of London. Scientists asked 6,641 people to describe their first memory, along with their age at the time, and discovered that 2,487 first memories were unlikely to be true because they were captured before the age of two. An astonishing 14 percent said they remembered an event before their first birthday — some even saying they remembered their birth.

It’s scientifically accepted that autobiographical memories are possible only after the age of three. Before this, babies’ brains are physiologically incapable of forming and storing episodic memories, because the parts of the brain involved in these tasks are underdeveloped. In fact, some scientists believe we can recall autobiographical events only from around five or six years of age, and anything we remember before this is what’s called a “fragment.”

Despite this, a large number of those surveyed were convinced they could recall peering out of their pram, having their diaper changed, or even taking their first steps. So why do so many of us remember what is thought to be impossible?

It seems most likely that we create fictional early memories by building on stories we have heard and photos we have seen. Martin Conway, director of the center and leader of the study, gave an example of someone who recalled being in their pram. “This type of memory could have resulted from someone saying, ‘Mother had a large green pram,’ and the person then imagines what it would have looked like,” he said. “Over time, these fragments become a memory, and often the person will start to add things in, such as a string of toys along the top.”

Fictional memories seem just as real as those we have evidence of and therefore know to be true. Brain scans have shown that the neural activity for false memories in adults looks incredibly similar to the activity for a real memory and involves the same regions of the brain, including the hippocampus. This means it could be questionable whether we have any “real memories” that can be relied upon at all, because to some degree all our memories are reconstructions.

Fictional early memories may also serve an evolutionary purpose.

“Rather than reexperiencing an event, we reconstruct it based on representations stored in our brains,” explained Brock Kirwan, an associate professor at Brigham Young University’s Psychology Department and Neuroscience Center. “What seems to happen in the brain is that when you reactivate these representations, they become malleable and can be modified. This allows you to update your memory representations and link them to new, similar events. It also means that your memories change over time.”

Whether the memory of my own grandma is accurate or not, recalling it might be doing me some good. “It may be beneficial in that if you have positive memories about early childhood,” Kirwan said. “It makes your overall outlook more positive, and you’re better able to cope with life’s challenges.”

Fictional early memories may also serve an evolutionary purpose, allowing us to make better decisions in the future. If we can draw on similar events from the past, we can make a good prediction of what will happen next. But since no two experiences are the same, having a literal playback is of limited help. “Memory is about the past, but it allows us to bond with one another and can act like social glue,” Conway said. “Instead, it’s more efficient to remember the gist of what happened, and if something different happens this time, it makes sense to update that gist-like representation with the new information.”

In this way, our memories change over time and are updated as we incorporate new knowledge of how the world works. “I suspect that these fictional early memories are a byproduct of that updating process,” Kirwan added. “We learn something about our own infancy, or infancy in general, and later mistake that information for an actual episodic memory because of its episodic-like qualities.”

Could this understanding prompt you to question your own past? And, if so, how can you tell if a memory is fictional? Conway believes the clues lie in the complexity of memories. “In many cases, they [first memories] are too conceptually complex [to be possible]. If you asked a two-year-old to remember what happened months ago, they couldn’t say that they used to play with a ball in their cot and their mother would come in and laugh. That suggests embellishment.”

Jon Simons, a reader in cognitive neuroscience at the University of Cambridge, believes autobiographical memories are dependent from language, which enables us to tell and understand a story. He said the anterior prefrontal cortex — an area of the brain just behind the forehead — is key to detecting whether a memory is true or false, and a judgment is made based on the quality of a memory. Those rich in sensory details are more likely to be real. “It’s quite reliable most of the time, but then there are ‘memories’ based on what someone has told you that are less clear,” he said.

Of course, you could search for a photo or ask your parents if they have ever told you about the event you remember, but that could still mean your memory is based on a fragment that you subsequently embellished. Then there’s the question of whether you would even want to know whether a cherished moment is false; some participants in this study were “furious” when it was suggested that a treasured memory was fictitious, Conway said, and some simply refused to believe it.

While I’m pretty sure I have fictionalized the vivid memory of my grandmother from an old photo, I would like to believe I really do remember a fragment of that happy day. As Oscar Wilde said, “The truth is rarely pure and never simple.”