Avatar follows mouse

Before I say anything, if you’ve tried to rotate the position of an Avatars eyes to follow your mouse X & Y position, you’re on the wrong track, the key to making this work is a Skelton

If your avatar/object has a Skelton you can make it follow your mouse position, like the example below.

Firstly I just want to do a shout-out to Ready Player Me(RPM), using the RPM platform the avatars you create come pre-installed with a lot of time-saving features such as shape keys and of course a Skelton.

So if you’ve not created an RPM avatar, go ahead and do that now as I’ll be referring to mine in this write-up.

Once you’ve created an avatar, create a new react application, in order to get your avatar loaded into JSX check out https://github.com/pmndrs/gltfjsx

Run the command (in the terminal) npx gltfjsx {name of GLB}

The output you get will look something like this

Once you have this file this is the starting point to begin to target the Skelton elements.

You need to create a hook that returns the mouse position in degrees, *very important*

Once you’ve created the hook we need to create a function that is responsible for moving the joints, like so.

We need the moveJoint function to fire as the mouse moves so we need to utilise the useFrame hook

const { size } = useThree()

useFrame((state, delta) => {const mouse = { x: size.width / 2 + (state.mouse.x * size.width) / 2, y: size.height / 2 + (-state.mouse.y * size.height) / 2 }

moveJoint(mouse, nodes.Head )})

The nodes are generated by the npx gltfjsx so all one needs to do is link the outputted JSX into the MoveJoint function, and voila.

I’ve kept all this pretty top-level, as once you think about it from a skeleton moving point of view, opposed to rotating the eyes e.g it’ll all make sense.

Check out the Codebox here https://codesandbox.io/s/wonderful-ellis-7soklh?file=/src/App.js