Lol, you invited him now you want to kick him out!!!
I really don’t know how you can win this one because you can’t get him to reason with you now. However, you can talk with him during the day about it. If he talks, get him to say somethings like this after you, “Lee loves to sleep in his bed’. ‘Lee loves to sleep in his room’. ‘Lee is happy to lie in his own bed’. ‘This is Lee’s room and he loves to sleep in it’.
This method worked for me years ago on another matter. If you decide to try it, I hope it works for you.
Whatever you decide to do, it’s going to be tough so you have to make up your mind to follow through.
Good luck with ejecting your toddler.