How to download the ‘ He Will Not Divide Us ‘ stream ( hewillnotdivide.us ) ( HWNDU ) (meme)

TLDR: use youtube-dl with some wrapper driver script to restart if there’s an interruption (included below)

Unnecessary Prologue

I think Shia Laboeuf’s latest art project is very fun indeed. I’ve always loved the idea of bridging mediums, and the art of mediums themselves, and this audiovisual bridge connecting Manhattan to the web is certainly very cool in my book.

I’ve already thought of a number of fun things to do with the footage he is collecting from the webcam exhibit. So I went about trying to download the stream to my computer — I tried a number of methods with eventual success.

At first I tried the most naive approach — find a normal video link in the webpage— usually if you want to download a video from the web, its pretty straight forward. If its a youtube video, you can just copy the URL into a website like keepvid.com — but i couldn’t find any generic static link within the https://hewillnotdivide.us website.

If you open up the network tab in your browser developer tools on the website when the video is playing, you’ll see that there is a constant stream of POST requests being sent to livestream, and you might even stumble across a fragment of the stream in a link like this https://vod.livestream.com/events/0000000000694072/3f4489c2-b32d-4965-8a99-46ee7960166e_2320-2687.ts

It became clear to me that the streaming platform livestream wasn’t going to allow me to download the feed without at least hours of tinkering, and making my project more complicated.

Even if you try to grab a link the playlist of old videos in the dropdown on the right hand side, you will receive one of these streams, not a direct video download link. Nice try.

So I turned to Google (google.com) and found some code projects for downloading livestream feeds.

Initially I went with livestreamer but hit a wall with some errors ( let me know if you have better luck!)

Working Solution

I eventually tried youtube-dl which got me working very quickly.

Installation

Go to this github page and follow the steps for your OS — it is only a curl away. ( Requires ffmpeg which is also a great tool )

Use It

Once you’re installed, to immediately start downloading the live HWNDU stream, run

youtube-dl https://livestream.com/accounts/23093026/events/6897778/player?&enableInfoAndActivity=true&autoPlay=true&mute=false&startswith=true

Automate it

This is all good and well, but we don’t want to babysit the stream download. Preferably we can make it automatically restart if it crashes, and get it to save itself in a maximum of 6-hour video file segments.

Here’s my go at a bash script to do just that ( if you can think of any improvements, please share in the comments ) — to run it, in the command line run ‘bash ./hwndu.sh’

#!/usr/bin/bash
get_six_hours() {
    six_hours=:uuh:
    ((six_hours = 21600))
}
kill_in_six_hours() {
    trap 'printf >&2 "kill_in_six_hours exits\n"' EXIT
    for ((; ;)); do
        get_six_hours
        printf >&2 "\nkill_in_six_hours will sleep for %d seconds\n" \
                   "$six_hours"
        sleep "$six_hours"
        killall youtube-dl
    done
}
kill_in_six_hours &
kill_in_six_hours_pid=$!
trap 'kill "$kill_in_six_hours_pid"' EXIT
printf >&2 "Whenever you Ctrl-C this script, kill_in_six_hours process will be \
killed too.\n"
for ((; ;)); do
    # Line continuation syntax for arguments in double quotes: \<newline>
    # Make sure there are no spaces after "\" on line below.
    youtube-dl --hls-use-mpegts "https://livestream.com/accounts/23093026/events/6897778/\
player?enableInfoAndActivity=true&autoPlay=true&mute=false&startswith=true"
done

Long-term use-case

you should probably run through this whole set up again on a laptop that you can leave on 24/7 hooked up to an ethernet cable and an external harddrive — its going to use ~25GB a day / 1tb in 40 days..

Let me know if you want / have any improvements.