My Attempt at Building the Metaverse in Public (pt2)

I’m not going to try to introduce all about myself here, but a quick TL;DR for why I’m attempting this project would be:

1. I’m in the middle of starting my own adventure with my own startup, and need to be able to prototype ideas.
2. It’s been a crappy year and I could use something I’m proud of.

Start here to read from the beginning.

Last post was outlining what needed doing (more or less). This post will be about the first part of doing. I should be honest though, this one is about stuff I’ve already done, so it might be a little quick. Luckily, though, it’s also the one that’s the least interesting to folks — setting up and the bare minimum of installing stuff.

It also occurs to me that I should mention a little bit about my plans for this series. I’m not writing this as a tutorial, so please don’t expect step by step instructions or a ton of screenshots. I’m mostly doing this for my own knowledge, but also in case anyone is curious in fifty years about what dipping your toe into dev work really looked like.

You know, in case our robot overlords want the massive AI brains that control every aspect of society to write a historical sitcom about their great great grandparents. Or something.

Anyways, onward to building!

Oh wait — you might want to know what it is that I’m building…

Nothing special, really — just…

Kinda. A limited and custom designed one — one that’s specifically intended to coach the desired style of interaction. The there’ll be more about this later, I think. Anyways, back to…

Yeah — I used to work at what was arguably the most successful Metaverse startup, Gather.town — until there were rounds of layoffs and all of us early hires with nice (and unvested) equity packages got the boot.

Since then I was recruited to join a new Metaverse startup, and I’ve been working to get to launch publicly. I am joining to be responsible for product strategy, but I’m also going to have to handle some amount of technical needs too. As a first step, we’ve been testing using existing platforms, but I want to be able to make more fundamental changes. To that end, I have found SkyOffice, a self-described 'Open Source Gather.town clone.’

I plan to use a heavily modified version of this to refine my vision for what we wanted to build. So yeah, that’s the plan. Use some OSS to develop a strong direction for the product, and make it useful enough to get worthwhile data from customers and users. I won’t actually use this software for the product, I don’t think it’s nearly robust enough. But actually, and more importantly, I don’t know enough to make that decision. I just need to have something to show to people and say 'build a good version of that!’

So, now to building.

In case you’ve forgotten, Steps 1 and 2 are pretty fundamental. In a nutshell, they are “do everything you need to download and run the original code.”

Step 1: Get the right software/providers set up. I know that I’ll need to install node.js, and probably VSCode or some other editor. I’ll also need to remember how to log into my github account, and find somewhere that will give me some hobby server space so I don’t waste too much time yak shaving to run node.js on my NAS here.

Step 1a: Refresh/learn the fundamentals of the tools that I’ve installed above in Step 1. These are things that I have either never done or haven’t done in a LONG time.

* Clone a Repo
* Fork a Repo (those are different, right?)
* Connect VSCode to Github
* Compile code
* Connect whatever hobby server I select to Github
Whatever else I don’t know that I don’t know how to do

Step 2: Compile and run the code locally. This will be a good first step, and the software that I’m working with has some very lightweight instructions on how to make that happen.

Step 1:

So, to make that happen, I’m pulling on what I can google and what I can remember from a couple of lessons in Typescript that friends have given to me over the years.

First off, I should read the instructions, which say

You’ll need Node.js, npm installed.

Ok, let’s go get those. I also remember my dear friend Evelyn telling me “VSCode is fine” — so I go grab that too. Next I hop on my favorite source of education, Youtube, and google for a couple minutes until I find something. I’m not really sure what I’m looking for, so I just look for “run node js app from github” — and land on this video.

This looks pretty doable, so I go grab Github desktop and look for a terminal… editor? Terminal window? I’m not sure the terminology here, but anyways, I end up with CMDR. I look at the repo I want and I see

This looks like it’s time for…

Step 2:

Ok — great. Run through those instructions with the server, I see a progress bar move across the screen, and suddenly I have a bunch of extra files on my machine. I poke into it and see “Yup, there are some images, that looks like a client and a server folder, and this looks like what I understand a well formatted Node.js app to look like. Every module is laid out and somewhat commented, and I can see that most of the variables and functions appear to have somewhat descriptive names. Kevin Shen, you do nice work.

Oh yeah — and I can also go to my localhost and see a non-descript error message. Neat! It’s telling me that there’s something running at least, and it’s not just a standard “nothing here” window.

Build/run the client, and then — yup, there it is!

I can load up my browser and hop into the space and explore the world of SkyOffice. Let’s talk about the features of this that I will need — again remembering that it’s a clone of Gather.town. I’ll do a much more in depth analysis of these at a later date, maybe even a feature on each of the important ones, but for now, I’m just going to quickly define what I’m looking at/for. I might get crazy and find a way of illustrating what I’m trying to describe — maybe I’ll put together some wireframes.

Features:

Avatar Control: In almost every metaverse environment I’ve experienced, you’ve had some way of controlling an avatar that represents your “YOU” in the environment. One of the key features of “The Metaverse” is the idea that the content you are exposed to or able to access is limited based on a concept of ‘location.’ This is a feature that is a dependency of the next one, but it’s also an easily overlooked consideration. Right now, this appears to be handled primarily by a customish game engine that feeds data into the rendering engine, Phaser. I’m not entirely sure what everything will look like when I am able to really dig into the code.

Proximity Video Chat: I’m tempted to say this is the most important feature of the concept, but honestly, I’m not sure anymore. For many use cases, proximity video chat (i.e. when your avatar is close to someone else’s, you’re both connected bidirectionally to each other and every other avatar that’s close enough). I think there are a lot of considerations of this from a user perspective here as well. This appears to be handled by PeerJS, which I think is a wrapper for WebRTC peer-to-peer video communications. I know that Gather.town used SFUs for scalability, but I also know that adds a substantial layer of complexity, so I think unless I’m feeling INCREDIBLY ambitious — I’ll stick to peer-to-peer. NOTE: I think this probably also handles chat, since I think WebRTC can carry text messages as well as media streams.

Multiplayability: That’s totally a word. Don’t look it up. Basically, for avatars and proximity video chat to really have any value, you also have to be able to have multiple users using the same software at the same time. This means that avatars represent real people, at that exact time, in the same space as you. That’s a lot to ask, but I think it’s another of my key features of the metaverse. This appears to be coordinated with an open source gameserver management service/framework called “Colyseus.” I have already bumped into some complications with this that I think are actively blocking me (but more on that later).

Explorable Space: This is a bit of an odd one to explain. The core idea is that your perspective and access to data is represented by an Avatar, and so that avatar needs an environment to move around in. This needs to be recognizably a space intended for navigation, and it can be handy if the space design conveys enough information to tell a user how to act in the space and how to interact with the space. I’ve got a lot of thoughts about this, and I’m pretty sure that I’m one of the people in the world that has thought about this the most. From a quick look, it appears that SkyOffice defaults (update: it’s hard coded) to a pixel art office — which is from a tileset I recognize. Coincidentally, it’s the one that I used when I made my first Gather map. I’ll tell y’all that story another time. The good thing is that this appears to be a file format I’m relatively familiat with, a Tiled Map. I think that this might be one of the first things I’ll approach.

Interactable Objects: This is kind of a broad category, but in general it’s the idea that inside of the environment, along with avatars, there should be something to DO. Most platforms I’m familiar with handle this by embedding content in an iframe, or (much less common) writing custom code for each object. I actually don’t know how SkyOffice handles this, but I’d be willing to bet it’s with a variation on the iframe version. I do know that SkyOffice hardcodes all of their object types, while Gather.town made ours variable.

Related to both of the above features: I’ll dig into it a bit more and see if there’s a way I can import them from a Tiled Map as an object property. My sidegoal is to be able to use Tiled as the main way of editing maps, and a distant future goal is to write some scripts for Tiled to generate those maps programmatically. There’s a long way to go for that.

Other Aspects: Inventory, gamification, extensions, etc Basically, there are a ton of other things that I’ve thought about in the metaverse space. Some I’ve got some ideas for how to handle them, some I don’t. Some I think are going to be important for the future, and some I’m pretty sure are more of a bridge technology, or just aren’t important at all! I’ll break this down more in a future post.

So that’s it — I’m (probably) done with Steps 1 & 2. I’ve got the basic tools I’ll need to access and manipulate the code, I’ve got the code itself, and I’ve got confirmation that it appears to work at least somewhat. I have a few things I’d like to confirm for next time:

• That I can make trivial changes to the code and see them reflected when it’s running (like changing the name of things or menu items).
• That I can run it on multiple machines and they can connect to each other.
• That I can make changes to the images that it uses (like the avatar spritesheets).
• That I can make changes to the map environment it uses (which includes both the space backdrop plus the collision tiles and maybe the default embedded objects).

Ok — on to Part Three!