Wow. Wow. Really, wow! This is a real work of art. Your code and ideas for implementing this system are brilliant, sparse and elegant. It takes a lot of cross discipline knowledge for one person to pull this off. It’s not a hack or one-off. I can definitely see you turning this into a product or platform for creating and trading assets and generating performances.
I’m guessing you will be getting some offers soon. I also think this would make a great Kickstarter project if you aren't doing it full time now or need more resources.
Keep it up, stay focused, keep it simple like you have done so far. I think the rendering and syncing the music will be challenging to automate or pull off in a scalable, platform like way, but I could be wrong. There are lots of cloud services for rendering. I can also see a future where digital content and characters can be provably owned and traded on a blockchain. https://medium.com/@simondlr/towards-a-generic-rare-pepe-design-trading-digitally-scarce-content-f221d9bb4755