I would argue that all human beings are born as atheists, yes.
Religious beliefs are interchangeable and are acquired as you grow up, very much like language. An infant speaks no language, but has the capacity to acquire any language from their environment. In the absence of any human elements whatsoever, it’s possible that the child develops a unique, custom language of their own, but that would be a process of development, not an innate or intrinsic feature of their birth.
Same with religion and, in the case of atheism, the lack thereof. A child can learn any religion, slowly over time, and optionally make it part of their identity. It’s generally a combination of what they’re taught by parents, what they encounter in their environment, and what choices they make with that information. But at the very start, prior to any external influence, an infant does not yet have a concept of deities or specific beliefs. They may form such a concept, of course, much like how the human child growing up in complete isolation away from any sign of mankind may form a language of their own, but if they do that, again, it would be a process of development over time, not something they were born with.
(This, yes, is taking atheism at its purest and simplest meaning: the absence of any specific theistic belief. “Atheism” as a concept is a lot more loaded for us as adults, but that’s what it is at its core.)