The rectangle behind you

The making of Four Laps

  1. The creative process and the setup
  2. Specific tech questions
  3. What I learned about controlling and scripting OBS (incl. all of my code)

1. The creative process and the setup

  • Can one do compositing in OBS? (can I overlap myself?)
  • Can one feed the recording into itself as playback?
  • How capable is OBS scripting? Can I do everything programmatically?
  • Do I know how to make a consistently white background?
  • Can I practice enough for this to feel good?
  • Do I actually have something to say here?

The story

  • I didn’t originally plan to fast forward, but that would make the first lap much too long. I hated this for the longest time, but now I think it works nicely.
  • Originally, I wanted to put on the hat at the end at the exact moment I take it off originally — creating an impression this was the same hat — but the structure actually made it impossible without introducing a visible fifth lap!
  • programmatically changing colours of the shirt or making me lighter with each loop
  • cough at the same time (too hard to sync up)
  • duck/throw something (dangerous)
  • yawn (weird pause)
  • subvert the glasses thing at the end by wearing weird glasses (didn’t fit with the story)
  • instead of spelling F·O·U·R, do something more sophisticated (didn’t have a better idea)
  • some more advanced choreo (ran out of time)
  • turn on the light above me or pull up a banner or change the environment (ran out of time)

The setup

  • Mac computer: This computer was grabbing the video from my camera, then in OBS the recording from that camera was fed back into OBS as video input after a 70s delay via a set of Lua scripts I wrote. The video camera source was always on top, so it required color key so that the white was “transparent” and the delayed videos in the background could be seen.
  • Windows computer: The video output from the Mac computer was fed to a Windows computer’s OBS and composited with a pre-recorded 5-minute After Effects video that contained titles, “slides,” and all the music via another set of Lua scripts I wrote. The Mac computer output was always on top, and required a similar color key as above to allow the slides to be “behind me.” This computer also grabbed the sound from the mic, recorded everything, and sent things to Zoom.
  • iPad: The iPad was responsible for showing the “timer” that alerted me to various events that were supposed to happen — it was like live quick time events. Most of the events were announced in real time, but for crucial things like moving to a new position (where I absolutely needed to do it at the right moment) there were counted down.
  • There was also a fourth computer, but this was mostly used to connect to the Zoom independently, and just wait for my cue during the lightning talk session.

I’m surprised this actually worked

  • when the first attempts to script OBS resulted mostly in crashes, and when I realized how poorly documented everything is
  • when I realized I would have to have two computers
  • when my other Mac was too slow to handle OBS
  • when the Windows computer that replaced it worked well for a while, and then stopped performing well

2. Specific tech questions

How was the video looped?

How did you make the slides/titles/etc.?

How did you do the little rotoscoping bit?

How did you know where to stand?

What kind of white screen did you have?

Why white screen and not green screen?

How did you make the timer?

  • most of the UI is at the top of the screen. It was originally centered vertically, but I noticed that this resulted in my eyes moving on the screen much more visibly
  • there are special annotations when the music plays loudly. I actually didn’t hear any music as I was speaking, because I was worried it would create feedback or be recorded twice. So I needed a visual indication that music is playing so that I wouldn’t speak at that time.

How did you mount the iPad below the camera?

How did you mount the Windows computer close to the camera, so it could serve as a monitor?

How much did you practice?

What kind of mic did you use?

How to connect the lav mic to a Windows laptop?

How did you prepare for failure?

  • a few days before, I made a “release candidate” video recording to have in the backpocket for the audience
  • in the next few days, I kept replacing that video as I kept improving
  • I had a pretty extensive checklist for the day off (this is only about 25% of it) and also a last-minute checklist on the wall:
  • for the Mac, the checklist started from actually rebooting (sometimes connecting HDMI would make it go into an endless loop)
  • I made both OBS have a deterministic start state (by default, OBS starts the way you left it last)
  • on my Mac, I had this wallpaper set up just in case OBS decided to randomly crash in the middle (which it did once in a while):

What kind of camera did you use and how did you connect it?

How did you maximize room?

  • wide angle lens
  • tilted the camera a bit up
  • I didn’t know how to raise the camera so I didn’t cover so much of the screen… so instead I moved the recording down by 100 pixels. (Just needed to be careful not to move my hand up too much so they’d appear clipped.)

How did you pipe the video from Mac into the Windows computer?

Why did you use a Windows computer as a second computer?

How did you make Windows computer perform better?

How did you share the video to Zoom?

  • your webcam (prioritizes framerate over quality)
  • your screen share (prioritizes quality over framerate, although there is an option to make it more closer to the webcam if you’re sharing video). Also, no pixels are cropped
  • as a webcam (making OBS create a fake webcam in your system that basically has your OBS screen)
  • as a “projector” or just screen sharing an OBS window

How did you share the audio to Zoom?

How did you solve the issue of Voice Meeter audio getting garbled?

How did you solve the sync issue between video and audio?

ffmpeg -ss 00:00:28.00 -i input.mkv -itsoffset 0.3 -ss 00:00:28.00 -i input.mkv -map 0:v -map 1:a -c copy -t 05:00:00 output.mkv

How did you make Zoom go 1080p?

How did you make sure the three devices started at the same time?

What font is that?

How did you stop your cat from interfering with your practice or messing up your sound by meowing?

3. What I learned about controlling and scripting OBS

local obs = obslua

Why OBS and not Streamlabs OBS?

How to log to OBS’s console (called Script Log)?

local function log(name, msg)
if msg ~= nil then
msg = " > " .. tostring(msg)
else
msg = ""
end
obs.script_log(obs.LOG_DEBUG, name .. msg)
end

How to effectively iterate in OBS?

How to start OBS in a certain state (Mac)?

rm -rf ~/Library/Application\ Support/obs-studiocp -R ~/Library/Application\ Support/obs-studio-kylie 
~/Library/Application\ Support/obs-studio
open /Applications/OBS.app

How to start OBS in a certain state (Windows)?

del /q c:\users\mwich\AppData\Roaming\obs-studio\basic\scenes\*.*copy c:\users\mwich\desktop\kylie\bat\z.json 
c:\users\mwich\AppData\Roaming\obs-studio\basic\scenes
cd "C:\Program Files\obs-studio\bin\64bit"
obs64.exe

How to make OBS go full screen?

  • the last time I used them, they still had bits and pieces of UI
  • I only had one display, and I felt this would’ve made things complicated
  • I remembered it taking extra CPU
  • I actually, believe it or not, couldn’t find it in the UI
  • I toggled all of these options off, and used Lock UI so that they wouldn’t reset
  • In the Edit menu (why there???), I opened Preview Scaling, and changed it to be 1:1 (otherwise even in fullscreen, there was a bit of a bezel)
  • I made OBS go fullscreen via Mac’s regular “green button”

How to move an item in the scene?

local item = obs.obs_scene_add(current_source, camera)
local pos = obs.vec2()
pos.x = 0
pos.y = 100
obs.obs_sceneitem_set_pos(item, pos)

How to resize an item?

local item = obs.obs_scene_add(current_source, camera)
local scale = obs.vec2()
scale.x = 0.9375
scale.y = 0.9375
obs.obs_sceneitem_set_scale(item, scale)

How to move an item in the z-index?

local item = obs.obs_scene_add(current_source, camera)
obs.obs_sceneitem_set_order(item, obs.OBS_ORDER_MOVE_BOTTOM)

How to remove all items in a scene?

local sceneitems = obs.obs_scene_enum_items(current_source)for i, sceneitem in ipairs(sceneitems) do
local sourceSrc = obs.obs_sceneitem_get_source(sceneitem)
local type = obs.obs_source_get_id(sourceSrc)
local settings = obs.obs_source_get_settings(sourceSrc)
obs.obs_sceneitem_remove(sceneitem)
end

How to prevent a red border from appearing around things?

local color_item = obs.obs_scene_add(current_source, color_source)
obs.obs_sceneitem_set_locked(color_item, true)

How to set up a color key filter programmatically?

local recording_filter_settings = obs.obs_data_create()obs.obs_data_set_int(recording_filter_settings, "key_color", 0xffffffff)
obs.obs_data_set_string(recording_filter_settings, "key_color_type", "custom")
obs.obs_data_set_int(recording_filter_settings, "similarity", 57)
obs.obs_data_set_int(recording_filter_settings, "smoothness", 94)
local recording_filter = obs.obs_source_create_private("color_key_filter", "Recording Filter", recording_filter_settings)obs.obs_source_filter_add(camera, recording_filter)
obs.obs_data_release(recording_filter_settings)
obs.obs_source_release(recording_filter)

How to fade out?

fadeout_scene = obs.obs_scene_create('Fadeout')
local new_something = obs.obs_scene_get_source(fadeout_scene)
local new_source = obs.obs_scene_from_source(new_something)
local color_settings = obs.obs_data_create()obs.obs_data_set_int(color_settings, "width", 1920)
obs.obs_data_set_int(color_settings, "height", 1080)
obs.obs_data_set_int(color_settings, "color", 0xffffffff) -- White
local color_source = obs.obs_source_create("color_source",
"Color", color_settings, nil)
local color_item = obs.obs_scene_add(new_source, color_source)
obs.obs_sceneitem_set_locked(color_item, true)
obs.obs_data_release(color_settings)
obs.timer_add(start_fadeout, (5 * 60 + 25) * 1000) -- 5m25s
function start_fadeout()
obs.timer_remove(start_fadeout)
local new_something = obs.obs_scene_get_source(fadeout_scene)local transitions = obs.obs_frontend_get_transitions()
local fade = find_source_by_name_in_list(transitions, "Fade")
obs.obs_frontend_set_current_transition(fade)obs.obs_transition_start(fade, obs.OBS_TRANSITION_MODE_AUTO,
2000, new_something)
end

How to only fire a timer once?

obs.timer_add(something, 2000)
function something() {
obs.timer_remove(something)
...
}

How to add CamLink capture programmatically? (Mac)

local camera_settings = obs.obs_data_create()obs.obs_data_set_string(camera_settings, "device", "0x1100000fd90066") -- Cam Link obs.obs_data_set_string(camera_settings, "preset", "AVCaptureSessionPresetHigh") -- This means 1920x1080local camera = obs.obs_source_create("av_capture_input", "Camera video", camera_settings, nil)
local item = obs.obs_scene_add(current_source, camera)
obs.obs_data_release(camera_settings)

How to add CamLink capture programmatically? (Windows)

local camera_settings = obs.obs_data_create()obs.obs_data_set_string(camera_settings, "video_device_id", "Cam Link 4K:\\\\?\\usb#22vid_0fd9&pid_0066&mi_00#227&71832f1&0&0000#22{65e8773d-8f56-11d0-a3b9-00a0c9223196}\\global")local camera = obs.obs_source_create("dshow_input", "Camera video", camera_settings, nil)
local item = obs.obs_scene_add(current_source, camera)
obs.obs_data_release(camera_settings)

How to get the necessary CamLink id? (Mac)

obs.obs_data_set_string(camera_settings, "device", "0x12000000fd90066")

How to get the necessary CamLink id? (Windows)

obs.obs_data_set_string(camera_settings, "video_device_id", "Cam Link 4K:\\\\?\\usb#22vid_0fd9&pid_0066&mi_00#227&71832f1&0&0000#22{65e8773d-8f56-11d0-a3b9-00a0c9223196}\\global")

How to know the various names of settings for various items you can put in the scene?

local recording_filter_settings = obs.obs_data_create()obs.obs_data_set_int(recording_filter_settings, "key_color", 0xffffffff)
obs.obs_data_set_string(recording_filter_settings, "key_color_type", "custom")
obs.obs_data_set_int(recording_filter_settings, "similarity", 57)
obs.obs_data_set_int(recording_filter_settings, "smoothness", 94)
local recording_filter = obs.obs_source_create_private("color_key_filter", "Recording Filter", recording_filter_settings)obs.obs_source_filter_add(camera, recording_filter)
obs.obs_data_release(recording_filter_settings)
obs.obs_source_release(recording_filter)

How to figure out what are the right setting values if those are not obvious?

local sceneitems = obs.obs_scene_enum_items(current_source)
for i, sceneitem in ipairs(sceneitems) do
local sourceSrc = obs.obs_sceneitem_get_source(sceneitem)
local type = obs.obs_source_get_id(sourceSrc)
local settings = obs.obs_source_get_settings(sourceSrc)
log('local_file', obs.obs_data_get_string(settings, "local_file"))
log('looping', obs.obs_data_get_bool(settings, "looping"))
end
obs.obs_data_set_string(camera_settings, "device", "0x1100000fd90066")
obs.obs_data_set_string(camera_settings, "preset", "AVCaptureSessionPresetHigh")

How to start and stop recording?

obs.obs_frontend_recording_start()
obs.obs_frontend_recording_stop()

How to start a virtual camera? (Windows)

local ffi = require 'ffi'
local obsffi
if ffi.os == "OSX" then
obsffi = ffi.load("obs.0.dylib")
else
obsffi = ffi.load("obs")
end
ffi.cdef[[
typedef struct obs_hotkey obs_hotkey_t;
typedef size_t obs_hotkey_id;
const char *obs_hotkey_get_name(const obs_hotkey_t *key);typedef bool (*obs_hotkey_enum_func)(void *data, obs_hotkey_id id, obs_hotkey_t *key);
void obs_enum_hotkeys(obs_hotkey_enum_func func, void *data);
]]
function start_virtual_cam()
local target = 'OBSBasic.StartVirtualCam'
local htk_id
function callback_htk(data, id, key)
local name = obsffi.obs_hotkey_get_name(key)
if ffi.string(name) == target then
htk_id = tonumber(id)
return false
else
return true
end
end
local cb = ffi.cast("obs_hotkey_enum_func", callback_htk)
obsffi.obs_enum_hotkeys(cb, data)
if htk_id then
obs.obs_hotkey_trigger_routed_callback(htk_id, false)
obs.obs_hotkey_trigger_routed_callback(htk_id, true)
obs.obs_hotkey_trigger_routed_callback(htk_id, false)
end
end

How to check whether a file exists?

function file_exists(name)
local f = io.open(name, "r")
if f ~= nil then
io.close(f)
return true
end
return false
end

How to read from a local file?

local handle = io.open("c:\\Users\\xxx\\Desktop\\yyy.txt", "r")
local result = handle:read("*a")
handle:close()

How to access a certain URL (Mac)?

local url = "https://aaa.bbb/ccc"
local status = os.execute("curl --connect-timeout 1 --max-time 1 '" .. url .. "' 2>&1 &" )

How to access a certain URL without blocking the UI (Mac)?

How to mute a video source?

local recording_filter_settings = obs.obs_data_create()
obs.obs_data_set_double(recording_filter_settings, "db", -30.0)
local recording_filter =
obs.obs_source_create_private("gain_filter", "Mute",
recording_filter_settings)
obs.obs_source_filter_add(recording, recording_filter)
obs.obs_data_release(recording_filter_settings)
obs.obs_source_release(recording_filter)

What’s audio output and monitoring and how does it work?

  • By default, all the audio input sources in OBS are in Monitor Off mode, which means they get sent to streaming if you enable streaming (I didn’t use that functionality), and recorded properly. However, they are not actually output through the speakers (“monitored”) since you might not want to e.g. hear yourself again.
  • You can switch any audio mode to Monitor And Output, which means they will both get recorded and output to speakers.
  • You can switch any audio mode to Monitor Only (Mute Output), which means you can only hear it, but it doesn’t get streamed/recorded.
  • You can also just disable any audio input, although I am not sure how.

How do you set it programmatically?

function script_load(settings)
local sh = obs.obs_get_signal_handler()
obs.signal_handler_connect(sh, "source_activate", source_activated)
obs.signal_handler_connect(sh, "source_deactivate", source_deactivated)
end
function source_activated(cd)
set_monitoring(cd, true)
end
function source_deactivated(cd)
set_monitoring(cd, false)
end
function set_monitoring(cd, enable)
local source = obs.calldata_source(cd, "source")
if source ~= nil then
local source_id = obs.obs_source_get_unversioned_id(source)
if (source_id == 'ffmpeg_source') then if enable then obs.obs_source_set_monitoring_type(source, 2)
-- OBS_MONITORING_TYPE_MONITOR_AND_OUTPUT
else obs.obs_source_set_monitoring_type(source, 0)
-- OBS_MONITORING_TYPE_NONE
end end if (source_id == 'scene') then
obs.obs_source_set_monitoring_type(source, 0)
-- OBS_MONITORING_TYPE_NONE
end
if (source_id == 'wasapi_input_capture') then
obs.obs_source_set_monitoring_type(source, 0)
-- OBS_MONITORING_TYPE_NONE
end
end
end

How to change the volume or set the audio source to be active or not?

obs.obs_source_set_volume(source, 0)
obs.obs_source_set_audio_active(source, false)

What is the difference between “scene” and “source” in OBS?

new_scene = obs.obs_scene_create('NAME')
new_something = obs.obs_scene_get_source(new_scene)
new_source = obs.obs_scene_from_source(new_something)

What’s the importance of “local” in front of a variable?

--

--

A series of articles about interactive presentations

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store