Hi Steven, fasciniting post!
Gerardo Aragon-Camarasa

Hey Gerardo nice link — hopefully soon most videogames will use AIs like that reinforcement learning example, to make playing against the computer better ;). Some machines are aware that they need to maintain their basic needs — the self-charging Roomba is the classic example of that in everyday suburban life today.

I think there are (and will continue to be) huge gaps between human and machine self-awareness. Our self-awareness is tuned to well let’s be frank there’s this huge thing called emotion which is a dominating force in how we are aware of ourselves. Machines, while they do demonstrate emotion it is usually not due to efficiency, but because humans want to be entertained or to relate to the machine in some way. Emotions in machines, it’s just not that efficient, they are more rational, it is more useful for the contexts in which machines are evolving today. Humans came to be through a different route. I expect them to be very different for quite a while.

You bring up “wants” to do things — that is free will — I think it’s rightly a different area of philosophy I think that would require another series. ;)