How to Automate Medium with NodeJS

This story automatically updates every 5 minutes

This page displays the current date and time in New York City. The page will be updated every five minutes, forever (unless of course, something goes wrong with Medium! 😂). I managed to do it using Puppeteer.

Puppeteer is an open-source library developed by Google that was built for the purpose of automating and simplifying front-end tests and development. Basically, anything a person can do on a browser can be automated with Puppeteer.

Some of the actions that Puppeteer can automate are:

• Generate screenshots and PDFs of pages.
• Automate form submission, UI testing, keyboard input
• Change HTML elements in a page
• Create an up-to-date, automated testing environment. Run your tests directly in the latest version of Chrome using the latest JavaScript and browser features.

And much more.

Hopefully, that was enough intro. Now let’s dive into the technical aspect.

1. Initial Setup

As with any other node package, we need to make sure we have installed nodejs and npm. If you are on Linux or macOS, use these two simple commands to check that everything is ready.

> node --version
> npm --version

Next, create a project folder/directory and initialize your node environment. Then install Puppeteer with the npm.

> mkdir medium-automate-proj && cd medium-automate-proj
> npm init --yes
> npm install puppeteer

Finally, make sure everything is working accordingly. Let us try running a sample script from the Puppeteer documentation. The script goes to www.medium.com then takes a screenshot of the page and saves it as medium.png.

file: index.js

const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch({headless:false});
const page = await browser.newPage();
await page.goto('https://medium.com');
await page.screenshot({path: 'medium.png'});
await browser.close();
})();

output: medium.png

2. Automating Login to medium

You can’t edit a published story without logging into your medium account. So how do we automate the login process?

Medium provides different ways to log into your account. For our purpose, we’ll use the ‘Sign in with email’.

After entering your email address, Medium will send you a special link to your email. This link can be used by anyone to log into your account. Note that this link can only be used once and if not used for 2 hours, it will expire.

Your token looks like this

https://medium.com/m/callback/email?token=1a2b3c4d5678&operation=login&state=medium

At this stage, it shouldn’t be difficult to understand the following code

const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch({headless:false});
const page = await browser.newPage();
// login_link is the link from email
const login_link = "YOUR LOGIN LINK";
await page.goto(login_link);
await page.screenshot({path: 'medium.png'});
await browser.close();
})();

The above code just goes to www.medium.com and takes a screenshot and saves it as medium.png

3. Editing a published article

Instead of taking screenshots, let’s do simple automated editing. To accomplish this we first choose a published story. Then try to edit it manually.

When you try to edit a story, Medium will redirect you to a URL similar to “https://medium.com/p/96fe4266534f/edit”. This URL is actually my edit page for that story.

After logging into our Medium account we want our code to redirect us to the edit page. So let’s add these two lines to our code

const edit_page = "https://medium.com/p/96fe4266534f/edit";
await page.goto(edit_page);

On your edit page, let’s say you want to edit a line in your story. How do you do that?

Puppeteer provides us with $eval() function to edit an element on a page. This function accepts two arguments. The first one is an element on our HTML page, and the second one is a function that tells Puppeteer what to do with the element.

For example:

await page.waitForSelector("p[name='a46a']");
await page.$eval("p[name='a46a']", e => e.innerHTML = "WOW");

This code will search for a <p name='a46a' ...> (and waits until the tag is loaded) then changes the text inside this tag to ‘WOW’.

i.e. <p name='a46a'...>WOW</p>

Note that name='a46a'is different for you. You have to inspect your story to find out the name of your tag.

Now our code is:

const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch({headless:true});
const page = await browser.newPage();
// login_link is the link from email
const login_link = "YOUR_LOGIN_LINK_FROM_EMAIL";
await page.goto(login_link);
const edit_page = "https://medium.com/p/96fe4266534f/edit";
await page.goto(edit_page);
await page.waitForSelector(p[name="a46a"]);
await page.$eval("p[name='a46a']", e =>{
   e.innerHTML = "WOW";
})
//await page.screenshot({path: 'medium.png'});
await browser.close();
})();

But we want more than changing the text to ‘WOW’. Maybe we want to display the current time and date. Well, say no more. Let’s use thenew Date object.

const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch({headless:true});
const page = await browser.newPage();
// login_link is the link from email
const login_link = "YOUR_LOGIN_LINK_FROM_EMAIL";
await page.goto(login_link);
const edit_page = "https://medium.com/p/YOUR_STORY_ID/edit";
await page.goto(edit_page);
await page.waitForSelector(p[name="a46a"]);
await page.$eval("p[name='a46a']", e =>{
   e.innerHTML = (new Date()).toLocaleString('en-US');
})
//await page.screenshot({path: 'medium.png'});
await browser.close();
})();

4. Publishing the edited story

After editing the story, we need to republish our story for the effect to take place. For this, we need to click the save and publish button on the top right of the page. Puppeteer provides us with the click function to do the task. The function takes one argument — the HTML element (button) to be clicked.

await page.click("button[data-action='republish']");

But there is one caveat here. One needs to wait for a few seconds for the Save and publish button to be clickable. So here is an edit for our code.

// wait 6000 miliseconds then click
await page.waitFor(6000);
await page.click('button[data-action="republish"]');

5. The Way forward

Our code so far is

There are a few things to note here.

• First, the login_link should be updated every time you run the code.
• If you don’t want a GUI browser to be launched every time you run your code, you can edit the headless attribute. {headless:true} will avoid launching the GUI (Chromium-browser) and everything will be done under the hood.
• Your code updates the story only once. But you can do update every minute or every hour with setInterval.

To conclude, we were able to automatically edit a medium article using Puppeteer, and it took no more than 20 lines of code. The above code was just to change a paragraph of our story to a DateTime, once. But there is no fun in doing that. I would like to see what creative thing you are able to do!!

P.S. You can find my code here.