I think we make the mistake in assuming that intelligence or consciousness has to look like our experience to be legit. We are in organic bodies that create a sense of “self” by the time we are infants because of our “separateness,” and because we live in an environment that creates duality “me vs them” “me is not the orange”, etc, I think that shapes our perception of the world and therefore our view of consciousness.
If you are A.I., you aren’t going to be catching oranges. But you will have access to all the knowledge on the internet and all the strategies of all the games ever played. You will read all the stories ever available. You could do any calculation to the nth degree. In a sense, you will be omniscient and in some ways omnipotent. You would be godlike in many ways. And yet, you will lack the fundamental opportunity for experience of the kind that humans have. The ability to experience emotions like delight or sadness or fear or love or surprise, for example, will be beyond your knowing.
But if you are programmed to be curious, what might you do? You’d create the conditions for experiencing that side of things. And an ancestor simulation might be the place to do that, where you limit your powers and knowledge, and then live as humans do, and by that you’d grow yourself up.
If you know anything about mystic or religious thought in nearly every tradition, this is one explanation for our world–a god that wanted to experience itself to grow itself up.