Creating A Fully Automated Youtube Channel

Getting Started:

Hello there. I am embarking on a project to create a fully automated youtube channel. I’m going to do my best to document my learnings along the way so that others may benefit from what I learn. Okay here we go…

Background:

To provide a little context, I am a software engineer who works in a corporate incubator so starting up businesses is something I think about a lot. I also have a background in music; in fact audio recording was the reason I ever got into computers in the first place. On top of that I am a huge personal finance nerd who loves thinking about the idea of generating passive income and achieving financial independence someday. With that in mind I am typically having thoughts that go something like this…

What can I do to create passive income?
Can I automate it?
Youtube is cool!

And then you wonder; could I use my technology skills to automate the management and maintenance of a youtube music channel? What about other types of channels?

Into the rabbit hole we go…

Technology:

There are logs of tools for creating videos. But most of them involve you actually editing videos. That wasn’t going to work for me seeing as I didn’t want to spend all day creating, rendering, uploading, tagging, and promoting videos. I’d rather have the all of that work take care of itself. At this time I was using AWS pretty heavily at work and figured, why can’t I put AWS infrastructure to work to help me make this idea a reality at scale. So that’s what I did.

Personal Finance:

I geek out on personal finance and I love explaining the concept of financial independence to people. It’s not about making money. It’s about giving yourself the freedom to not have to. With this idea in my mind I am always trying to think up ways I can create extra income where I don’t need to spend any time or effort to keep going. Cause if your getting paid to work because you have too, well that isn’t really freedom is it?

Youtube:

Lastly I like youtube. I like it a bit too much actually. Lots of awesome content with endless things to learn about.

It’s all of these that lead me to the idea of automating a channel in the style of channels like Majestic Casual or Mr. Revillz whom I had listened to before and thought their format would be a good place to start.

Goals:

Technical Learning:

When I started off this project, I had only used a small number of AWS services. I wanted to learn more about them and what they can do. I also have done a ton of project that I didn’t really have much to point to at the end; and hopefully (knock on wood) this one will be a pretty public way to show what I can do.

Free money!

Turns out you can get paid via AdWords on youtube. Sounds too good to be true, but it is something lots of people do. The traditional content creation and management doesn’t fit my passive income requirement as it’s not truly passive. You need to work at it. In some ways this is more work, but once it’s set up, it will take care of itself and I can walk away.

Summary

Long story short, what started off as a simple idea grew into a large project. But seeing as it hits many of my interest areas, I am excited to get into it.

If you are interested in following along I’ll be updating this medium series with what I find as I go along. I’ll be touching on things like:

Software Architecture
Infrastructure Cost Analysis
Software Design & Development
Design Decisions
Growing a Youtube Channel
Passive Income

Cheers!

Alpha 1 — Proof of Concept

So I’m ready to sit down and start creating my automated youtube channel. But for as simple as it may seem, there is a lot that goes into a fully automated channel. There is finding content, gathering content, rendering the video, uploading it, providing the appropriate metadata… the list goes on and on. So in order to test the concept I knew I needed a minimum set of features that would allow me to feel like this is possible.

For me; that meant I wouldn’t bite off that entire problem but rather start with the part I thought would be hardest; rendering the video, then publishing it to youtube. Now that the problems was somewhat simplified.

There are many services out there that offer integrations between services via an API. Just to name a few: (IFTTT, Zapier) I knew I ultimately wanted to build that functionality myself so I didn’t need to pay someone else for it. (The point of passive income is to make; not lose money) But, this is a MVP. If I can pull anything off the shelf, it will get it up and running faster. I ended up finding the publishing to youtube step in a zapier integration and just within minutes I was half of the way there. I just needed a way to render a video from an image and audio file.

In my mind videos were made in editing programs. Programs such as iMovie, Adobe Premiere, or something similar. For my purposes the UI’s that these programs offer were useless. It was the underlying rendering software I needed. While I did find an interesting library of a headless version of premiere; I didn’t want to try to get that running on AWS Lambda. So I choose to use a little command line utility called ffmpeg. With this tool I can create a video from a audio file with this command:

ffmpeg -loop 1 -framerate 2 -i test.jpg -i test.mp3 -s 1920x1080 -c:v libx264 -preset medium -tune stillimage -crf 18 -c:a copy -shortest -pix_fmt yuv420p /tmp/out.mp4

It’s a doozy of a command but hey! It works!

So now I have a plan…. Sort of.

A few questions arose in my mind at this point.

How can I run a shell command from AWS Lambda?
How should I trigger the youtube upload in Zapier?
How can I start this whole process once some content is discovered; and saved somewhere.

This is where I put on my software architect hat.

We want to design something that is scalable, but cost efficient. So a few things to consider for this particular problem:

Video rendering and ffmpeg are processor intensive operations. They can sometimes take more than 10 seconds to run depending on the file sizes.
What language is best suited for the problem?

After researching these questions for a bit, I decided to go the route of using AWS SAM. SAM is a Serverless architecture model that allows for developing api’s via lambda functions locally. Some of the things that are really nice about SAM is that it handles creating your infrastructure though Cloud Formation, allows you to develop locally, and is only billed for the computing resources you use. Cool! So I’ve got a good cost efficient option for running ffmpeg. What about the language? As much as I love writing in node, python typically handles files, subprocesses, and more intense computation better so I decided to write it in python.

This stack costs virtually nothing to run, especially if you have a AWS free-tier account.

Alright; so I’ve thought through my MVP, found a few tools to make the process easier, and figured out some logistical architecture questions. Time to implement!

Alpha 1 — Post Mortem

I implemented my design and eventually got a working flow from storing audio, images, and videos in S3 and using Zapier to upload them to youtube. That flow was complete. So now what?

I decided to call this a alpha release and step back for a second to see if I still liked the direction I was going in. I think it’s always a good idea to take a step back and think critically about how things have been going and how they could be improved. To me that is the best way to make things better.

Here is a super rough list of what was I came up with:

Pain Points:

Trouble with AWS lambda python virtualenvs (workaround was kinda ugly)
Lambda timed out for large files even with a max timeout and memory capacity.

What went well:

AWS infrastructure was free
Zapier is free but I couldn’t also notify myself of events via slack without a paid plan
It did what it was supposed to consistently

Ideas I had while developing:

Would be cool to stitch two videos together or append a video to the front or back, but this would most definitely cause the lambda to time out.
Current solution while this worked really well for one case, it wasn’t very flexible if I ever wanted to expand on the content.

Overall I felt pretty good with how this first sprint went. The concept was proven. While it worked, there were still a lot of manual work to create videos. That wasn’t going to make the management of the channel truly passive. Also it because clear that rendering videos simply required more computing power than AWS lambda offers. So a few changes needed to be made to the existing flow, as well as adding the featured needed to reduce the manual work.

So it was time to plan out the next set of work and get to work.

Joe Cuffney

6 min

4 cards