Sandbox IRL Twitch Extension Introduction and Instructions

The “Sandbox” Extension allows viewers to make decisions and interact with live streams in real-time using two unique tools: Heatmap and Wayfinder.

The “Heatmap” tool allows viewers to click on the screen to visually highlight objects, places, or points of interest. The Heatmap allows for different colors to be displayed when viewers click on the screen. You will be able to see the Heatmap’s output through the browser source provided in the Sandbox Extension Config page.

The Heatmap from Muxy’s Sandbox IRL extension was used by viewers to decide which number to drop the coin from to try to win the game.

The “Wayfinder” tool allows the viewer to give directional instructions on how to navigate a physical environment. There is also a way to ask your viewers questions inside the Wayfinder, much like a poll. An example for an IRL streamer they could ask, “Go to the Park” and “Go to the Pier”, and viewers would be able to make a choice based on the options you have set.

The Wayfinder is basically a Poll that appears on the screen for viewers to interact with. The broadcaster sets the length of time for the poll and the results show up in the Viewer Vote URL that the broadcaster has access to in their desktop and mobile web browser.

This is the webpage the broadcaster can start and view the results of the Wayfinder poll. The arrow direction and history display the results from viewers using the Wayfinder to suggest where the broadcaster should go. Broadcasters can also use this webpage to start Wayfinder polls and set the duration for their access to viewers.

Why was the Sandbox Twitch Extension created?

Muxy developed the Sandbox extension with IRL streamers in mind. We originally ran an IRL stream with Twitch called “Twitch Goes to E3”. You can see an example of the video here. The idea was Twitch viewers could direct a person around the floor of E3 and have them interact with the expo. Sort of like how “Twitch Plays Pokemon” drove the character around through the chat commands, but this time we used the interactivity of Extensions to make the experience have a deeper interaction.

Twitch Goes to E3 was the first experiment Muxy participated in creating that allowed viewers to suggest the actions of a person on the expo floor at E3 while scheduled content was being prepared at the studio.

What are some ways the Sandbox Extension can be used?

The tools could easily be used to supplement gaming and IRL streams in a variety of ways. For IRL streams the design is to allow viewers to pick where the IRL stream is going, such as the directions on your game controller.

The same principle could be applied to game streams allowing viewers to pick which path the streamer is taking, or to pick between different rewards offered by the game. Streamers could also use the tools to help them pick games or any variety of in-game decisions.

E League / TBS / Twitch SuperPunch show used the Sandbox Heatmap to create a minigame for viewers to engage with during their first broadcast.

Meow Wolf used the Sandbox Extention to create an interactive story and PBS Boston used the tools to create an interactive escape room.

How do I get started using the Sandbox Extension?

The Muxy Sandbox Extension was designed for use on mobile and desktop streaming machines. First, you need to determine how you would like to use the Sandbox Extension experience. Twitch Goes to E3 and Twitch Goes to Meow Wolf both used a dedicated streaming PC to composite and encode the live video stream. We will start with the mobile instructions followed by the PC instructions.

Mobile Sandbox Extension Streaming Instructions

We tested the Sandbox Extension to work with the Streamlabs mobile apps on iOs and Android.

Android: We were able to use all of the features while live streaming on a Pixel 2 XL, though we suggest using two phones.

iOs: We have determined that two phones are necessary because iOs does not allow the stream to continue recording if you leave the streaming app. One will be streaming your video, and the other will allow you to interact with the extension.

1. Install Streamlabs app
2. Open app
3. Configure for streaming if you haven’t already

On Twitch dashboard:

1. Install Sandbox extension
2. Activate the Sandbox Extension as “Overlay 1”

Activate the Sandbox Extension and set as Overlay 1

3. Click ‘configure’

4. Generate a new Visualization URL. The Visualization URL will display the prompts and viewer inputs on your stream. This URL will be added to your streaming software/app.

In Streamlabs app

Add a new custom URL widget with visualization URL from the extension config page

1. Tap Menu icon in the upper left

2. Tap the Editor icon

3. Tap the Menu Layer button in the upper right

4. Now you will have access to the layers on the video screen, the order they are in, and the ability to add new layers. Click the + button to bring up another menu.

5. Now you will see the option to add a Streamlabs widget and a Custom URL. Tap the Custom URL and enter the Visualization URL that can be found on your Twitch Extension configuration for the Sandbox Extension.

The Visualization URL is the URL you add as a layer into your broadcasting app/software.

This is the custom URL area in the Streamlabs app for adding a new visual layer in your broadcast.

6. Now make the Visualization URL layer full screen and tap on the check button to save.

7. In the Layer Menu in steps 3 and 4, make sure that the Visualization URL layer in the first layer. If not it may be hidden by your other layers.

8. Testing to make sure it works! Muxy added a function that simulates viewers clicking on the screen. This feature is accessed in the Sandbox Extension config under the “Heatmap” menu. Click the button to start simulating clicks and remember to press the button to stop the simulated clicks.

The “Start Click Demo” button allows you to see if you have the click map Visualization URL set up correctly in your streaming software/app.

It is important to remember the differences between the two URLs given to you on the Sandbox Extension config page.

Visualization URL — displays viewer clicks from the Heatmap and directions from the Wayfinder on your stream

Viewer Vote Direction URL — is just for you. Open in browser and use the Wayfinder menu in the config to verify the connection so you can use the webpage to start Wayfinder polls. A PIN will be provided when you open the URL on the webpage. Enter that 4 digit PIN into the URL page, Viewer Vote Direction section and validate the pin. Then a new “start vote” button and duration selection will appear on the website.

The Viewer Vote Direction page will have an Authorization PIN that should be entered in the URLs / Viewer Vote Direction area to validate the connection.

Type the PIN into the box and validate the connection. Now you can start votes from the website while you are streaming.

This shows the Wayfinder vote outcomes and the duration and start vote buttons that display the Wayfinder for viewers to vote on the live broadcast.

Desktop Sandbox Extension Setup Instructions

1. We recommend using OBS Studio. Feel free to follow this guide for how to setup OBS Studio and connect your Twitch account to it.
2. Now you will install and activate the Sandbox Extension on Twitch to access your Visualization URL to add to OBS Studio.

On Twitch dashboard:

1. Install Sandbox extension
2. Activate the Sandbox Extension as “Overlay 1”

Activate the Sandbox Extension and set as Overlay 1

3. Click ‘configure’

4. Generate a new Visualization URL. The Visualization URL will display the prompts and viewer inputs on your stream. This URL will be added to your streaming software/app. Copy the URL and go back to OBS Studio.

In OBS Studio

1. Create a new layer as a BrowserSource. Insert the Visualization URL as the source. Now make that layer full screen in the visual editor. This shows the viewers and broadcaster where viewers are clicking in the Heatmap. Follow this guide on how to set up a BrowserSource in OBS Studio.

2. Now please start at step 8 above in the mobile instructions to learn about the other features. All of the URL’s can be opened as normal webpages so you can see the results of the Wayfinder and Heatmap on a computer.

How the Control Panel works

The Sandbox Control Panel is how you can administer the different tools of the Sandbox to their fullest potential. The way to access the Sandbox Control Panel is through the Extension/My Extensions area. The Sandbox Extension needs to be activated before you can configure it.

Activate and set as Overlay 1

Once the Sandbox is activated you can click on this Gear icon to see the config and Control Panel.

The Control Panel is accessed through the menu button all the way to the Upper Right.

Auto-Timer: This is the expanded version of the Poll feature in the Viewer Vote Direction URL. The Viewer Vote Direction URL poll only displays the Wayfinder for your viewers to engage with. The Auto-Timer on the Control Panel allows you to set duration and choose which of the tools you want available for your viewers to interact with.

Visibility: This area has 3 buttons for the different tools of the Sandbox. Here you can manually toggle on and off the different tools.

Call To Action: Here you can put text on the broadcast to ask your viewers a choice/poll. They can answer in chat or you can use the heat map to have them click on the screen to show their choice.

Let us know what creative ways you are using Sandbox by tweeting us your clips and ideas to Twitter.com/muxyatx