We’re publishing this article to share the complete framework UnfoldVR developed to bridge a native Unreal Engine 5 app to the Internet Computer(IC) blockchain using WebRTC protocol.
All code can be downloaded from the links bellow. We also cover all the steps on how to create a basic Windows UE5 app that reads/writes to a very basic IC app written in Motoko and TypeScript.
Here’s the sample project we’re sharing and documenting in this article:
The same core architecture is powering our VR app, UnfoldVR:
https://vimeo.com/727695530
Background
Web 3 has taken the development community by storm in the last couple of years. A core limitation of both Bitcoin and Ethereum blockchains is the high transaction cost which essentially limits apps built on top of these chains to employ basic ledger-based features.
A number of new blockchains have appeared which are trying to solve this issue. Usually named general-purpose blockchains, these frameworks offer orders of magnitude faster computation speed and lower fees than traditional blockchains.
Arguably, the fastest and most scalable general-purpose blockchain is the Internet Computer. A primer for it can be followed here: https://youtu.be/YWHTNr8RZHg.
The problem
An inherent issue with all Web3 frameworks is that, well, the web part. Specifically, they’re first and foremost targeting web platforms, like your browser. This doesn’t mean they don’t support native apps, the browser itself is a native app.
The real problem is that there is little focus to document native app development, very few projects target native development and fewer still share their process.
We strongly believe native apps have a solid place in the Web3 ecosystem. The main reason they’re not as popular is mainly the young age of really powerful blockchains such as the IC. There was just no need for native apps, taking full advantage of hardware resources, to handle frontends that worked just fine on the web.
Template file and code repository
We created an UE5 sample project, starting from the third person template. You can download all windows files from here: https://bit.ly/3QeZ4Ws
Or specifically, UE5 sample project: https://www.dropbox.com/s/qs6usk2x36khdcw/UE5_sampleProject.zip?dl=0
Web app sample, communicating with the UE5 project above: https://github.com/RP3a/ue_template
JS signaling server: https://www.dropbox.com/s/pa2ntom6dsp0gbc/signaling-server.zip?dl=0 — Windows only — replace this signaling server with an alternative one to support other platforms.
These files and documentation should be taken as means to get you started on creating Unreal Engine apps that communicate with IC blockchain using Internet Identity as a user login protocol. There is much room to expand and our approach isn’t the only way to achieve this, but we feel it is the simplest.4
This solution works when both UE5 and web app are running on the same network. An external signaling server is required for other use cases.
We’re currently developing a modified framework to allow MobileVR headsets such as the Quest2 to connect to the IC. One limitation we’re facing is poor WebRTC documentation for mobileVR, hence our approach on this relies on WebSockets.
Step by step guide
All the code is well documented but you still need to set up your Windows environment to run the blockchain locally. Publishing the web app on-chain will also require some additional steps.
How the code works and the set-up steps are all described in this article.
A mid-level developer should expect between 2–3 hours to set everything up (assuming UE5 is locally installed)
Before getting started, you should briefly familiarize yourself with the IC framework. We strongly encourage you to watch the primer above and the following video playlist from Dfinity: Coding with Kyle
1. Getting Started. UE Environment
First make sure you downloaded all the required files and code base and then you can go ahead and open the sample UE5 project.
The project already has the pixel streaming plugin enabled. The pixel streaming plugin includes all the WebRTC libraries you need. The entire logic behind the IC bridge communication is held in a BP actor named BP_IC which is placed in the main level.
The two key events that manage reading and writing are the following:
More information about how pixel streaming works can be found here: https://docs.unrealengine.com/5.0/en-US/getting-started-with-pixel-streaming-in-unreal-engine/
2. Signaling server
In the pixel streaming documentation you’ll learn how to set up basic communication functions and how to start a signaling server to establish the WebRTC connection.
For our solution, we’re using a simplified signaling server, derived from the one provided by Epic. You can download it from here(same link as above; credits for it are on the link):
Download link for signaling server(same as above): https://www.dropbox.com/s/pa2ntom6dsp0gbc/signaling-server.zip?dl=0
After downloading the signaling server, make sure you have nodeJS installed (https://nodejs.org/) and open a terminal window inside the signaling server folder and run:
node signal.js3. Deploying the blockchain webapp
Install the SDK for the Internet Computer on your machine following the steps described here:
We used DFX 0.9.3 for our demo https://github.com/RP3a/ue_template though you could use a newer version if you wish. To install a particular version use:
DFX_VERSION=0.9.3 sh -ci “$(curl -fsSL https://smartcontracts.org/install.sh)"To use the web app locally you first need to download the SDK code so you can setup a separate canister for Internet-Identity login. Download the repository from here: https://github.com/dfinity/internet-identity
After checking out the files above, run the following code in the repository folder:
cd internet-identity
dfx start — background — clean
npm ci
dfx deploy — no-wallet — argument ‘(null)’
npm startNow when you access http://localhost:8080/ you should see something like this:
To deploy the web app ue_template project locally, download it from: https://github.com/RP3a/ue_template and run the following commands:
cd ue_template
npm i
dfx deployAfter successfully deploying it, you’ll get two links returned in the console, one with the frontend and another with the backend. Accessing the frontend will give you actual web app which will look like this:
4. Digging into the web app code
The web app manages send/receive communications through the App.tsx file. The communication is handled by two functions. To read we have:
export async function ReceivedFromUnreal(detail : string)To write we have:
SendToUE(string)Both of these functions are bound to functions in peer-stream.js which is the interface to the pixel streaming plugin.
You can use the two functions to call any Motoko functions and queries. In our demo we simply pass strings to and from Unreal Engine. When the character steps on the platform we use SendMessage BP event to send a string, then on the web app, the function ReceivedFromUnreal is called by peer-stream.js which captures the incoming data(our string) and triggers a Motoko function
await ue_template.greet(string)This in turn returns the greeting text(string) and then sends it to the UE app through the SendToUE function. This is confirmed in the UE app with the OnInputEvent which prints the same message.
Following the same logic the ChangeColor function in the web app communicates the color change to the UE app, using the same OnInputEvent which has a simple string parsing logic.
5. Going from local to the blockchain
All the above will simulate the IC blockchain on your local machine. To publish the web app on the actual, live blockchain, you’ll need to create an NNS account and set up two canisters for this project (backend and frontend). For this you’ll need a wallet funded with a very small amount of ICP for the cycles required to run the canisters(around 1ICP ~ $8).
You can claim some free cycles by following this link: DFINITY Cycles Faucet (ic0.app)
To deploy the same app on the IC you need to deploy the same project on the newly created canisters. The whole process should take a novice about an hour tops but you can approach it in several ways. The following links explain this process in detail:
https://internetcomputer.org/docs/current/tokenomics/identity-auth/auth-how-to/
https://internetcomputer.org/docs/current/developer-docs/quickstart/network-quickstart
Parting thoughts
We’re very excited to hear what you’re building on the IC and we actually hope to learn a lot from other teams using our framework or other approaches.
Reach out to us on twitter at https://twitter.com/unfold_vr
For support and any kind feedback please join our discord and write us there: https://discord.gg/dUHW97fQ4B
You can join UnfoldVR at www.unfoldvr.com and download our PC VR build at https://bit.ly/3b2EZms
Additional developer resources pertaining to the IC that we found very useful are listed here:
https://github.com/dfinity/awesome-dfinity
