Converting FB Messages into a Youtube Playlist using NodeJS

I’ve been spamming my friend links to youtube videos, and I’ve asked him to convert those links to a playlist, but given that I’ve sent him over 1000 links, it’d be a little annoying if he went through and manually added every single one of them. So instead, I converted it myself using NodeJS, Facebook’s archiving service, and an online playlist converter.

Step-by-Step Process

Download messages from Facebook.

Select “Download a copy of your Facebook data.” This will tell Facebook to aggregate all the data they want to give you, and it’ll email you a link with that data in ~30 minutes as a zip folder. Within the zip folder, take out messages.htm. This file contains all the messages straight from messenger unformatted, so we’ll format that and turn it into a list of youtube links.

Parsing with NodeJS

If you’ve never programmed before, you’ll want to install brew and when brew is installed this from the command line:

brew install node

Then, inside the folder where messages.htm is, create a javascript file for parsing the messages. Additionally, run

npm init

And hit enter until it finishes, then run:

npm install --save striptags

This is another utility that’ll be used to parse the message file.

Within the parsing file(which I called index.js) add:

#!/usr/bin/env node
//index.js
const fs = require('fs');
const readline = require('readline');
const striptags = require('striptags');
const youtubeRegex = /^(?:https?:\/\/)?(?:www\.)?(?:youtu\.be\/|youtube\.com\/(?:embed\/|v\/|watch\?v=|watch\?.+&v=))((\w|-){11})(?:\S+)?$/;
//For detecting if something is a youtube url
const replaceList = (str, regexTuples) => {
let current = str;
for (let tuples of regexTuples) {
current = current.replace(tuples[0], tuples[1]);
}
return current;
}
//Helper function that replaces strings so we can isolate the urls.
//This function won't entirely resolve all issues.
const initialPass = (inputFilePath="./messages.htm", outputFilePath="./initial.txt") => {
  const stream = fs.createReadStream(inputFilePath);
//File we're reading from
  const outfile = fs.createWriteStream(outputFilePath);
//File to write to with the resulting text
  let streamStripper = striptags.init_streaming_mode();
const rl = readline.createInterface({input: stream});
rl.on('line', line => {
let htmlStripped = replaceList(
streamStripper(line), [[/;/g, ' '], [/http/g, ' http'], [/https/g, ' https']]);
    const urls = htmlStripped.split(' ').filter(str => {
if (!str.startsWith('http')) {
return false;
}
return str.match(youtubeRegex);
});
    process.stdout.write('|');
outfile.write(urls.join('\n'));
  });
 rl.on('close', () => {
console.log('done')
})
}

This function essentially rips out all excess html, splits all words by whitespace, and then keeps any that starts with http and also matches the regular expression. These are then appended to the file. After this, you should still pass through the file that was written to, and check that there’s no lines that contain multiple urls. If there are, just enter a newline before “http://”.

After manually parsing the file (doing that is actually important), we want to ensure that every url conforms to a standard. If you’re like me and use a lot of plugins on youtube, then you’ll have longer urls or you might have random names after the url.

In order to parse that, add another function to the javascript file:

const stripToLength = (input='./initial.txt', 
output='./final.txt', length=43) => {
const stream = fs.createReadStream(input);
const outfile = fs.createWriteStream(output);
const rl = readline.createInterface({input: stream});
rl.on('line', line => {
let str = line;
if (str.length > length) {
str = str.slice(0, length);
}
process.stdout.write('|');
outfile.write(str + '\n');
});
rl.on('close', () => {
console.log('done');
});
}

I’ve found that the usual length for a youtube url with secure protocol (https) has length of 43. Since we changed it into a giant list of urls before, we can go over each line which will now be a single url and trim it down to 43 characters. Then, we can write it to a new file, and this will be our final list of urls.

Using an Online Converter

With http://www.playlist-converter.net/, paste in the list of urls. Hit convert and give it authorization to add these urls to a playlist. And wait. This process took me a good two hours, and if it ever lags while finding urls, click login with youtube, and it seems to resume the process. Once that’s done, it’ll allow you to start adding links to a playlist. Whatever you do, don’t change permissions on the playlist, because it won’t be able to access the same playlist. After that, I suggest parsing what was added to the playlist, because it doesn’t filter what videos are added at all.

Update: Make sure to do the urls in batches with multiple tabs because it randomly fails sometimes.

Some things to note about the online converter:

  1. It’s all on the client end, so if it ever crashes, you’ll have to restart from the beginning, and there’s no way to save in the middle. In addition, the speed at which it works depends heavily on the network connection. It could be made more efficient by sending it to a server and processing it there, ensuring that the user’s network isn’t so important.
  2. Because it’s all on the client end, it might be better to do batches of videos (such as doing 1/10th of the total videos) so that if it does crash, you don’t have to redo the entire process. This requires an extra step of combining all the playlists in the end
  3. Not saying that the application exposes it’s client id and client secret, or that you can get your own access token easily, but using the NodeJS api with a client id, client secret, and authenticating yourself would let you implement a converter yourself. It might be quicker than the online one because there’s already access to all the ids.
Like what you read? Give Julian Knodt a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.