Building a Node App to Add Twitter Cards to Links

I consider myself a Twitter power-user, and from time to time I post found links. Usually when you paste a link in your tweet, Twitter parses it and displays a summary section with a title, description, and an image. Twitter calls these cards and they look like this:

Doesn’t this look engaging?

These cards aren’t automatic. The creator of the website needs to add some special markup to tell Twitter on what to display on the card. This is a shame, because photos and links boost engagement by 35%. Here’s what the same tweet looks like without a card:

Who’s going to click on this?

Because I want my followers to engage with my tweets, I decided to build a tool that would add cards to links that don’t have them.

I do a lot of front-end Javascript work with React and Redux, but I’d never written a server-side application in Javascript before, so I chose Node for this project. I used Babel to add support for ES2015 features.

Express seems to be the most popular web framework. I started by building a simple endpoint that prints the URL parameter.

That works, but now we need to figure out what to put on the card. I wanted to visit the link, see what’s on it, and generate a summary. I quickly found the X-Ray web scraping package for Node.

Cards have three parts: a title, description and image. Title should be simple to figure out, since almost every site has a title element. I started by grabbing that by suppling the title selector to X-Ray:

Now when you visited the app, it grabbed the title of the site. Time to get some more content for the card. X-Ray lets you scrape multiple selectors at a time by passing an object. I wanted to grab the first image and all the text on the site, too.

When the value of an item in the selector object is a string, X-Ray just grabs the first matching element. If you pass an array, it returns all matching elements. I wanted the first 300 characters of text on the page, since card descriptions have a 3-line limit.

Now I had all the content for the Twitter card, and just needed to display it when you visited the page. There’s lots of template libraries that are compatible with Express, but for this demo, I just decided to do it inline.

The markup for Twitter cards is documented here. There’s also a standard protocol called Open Graph, and Twitter will try that too if there’s no card markup. This is what it looked like when I plugged it into my response:

I uploaded it to Heroku, then tried it out using Twitter’s card markup validator, and success! It grabbed the title, image and description from the original site, and the card was valid:

There was just one problem. When someone clicks on my link, they’ll see a blank page, not the original site. I wanted to just show the card markup to Twitter, but redirect real users to the correct URL.

After a couple minutes of searching, I read that Twitter sets the User-Agent to Twitterbot for their web requests. That made things easy! We can show the card markup to Twitter, and redirect everyone else:

Now when a real user clicks on the link, they’ll see the original site, but Twitter will see the card. Now I know people will finally engage with my tweets.

Enjoyed this article? Tweet it ↘. Don’t worry, Medium has cards.

Read the code for this post on Github.