How to Automate Medium with NodeJS
This story automatically updates every 5 minutes
This page displays the current date and time in New York City. The page will be updated every five minutes, forever (unless of course, something goes wrong with Medium! đ). I managed to do it using Puppeteer.
Puppeteer is an open-source library developed by Google that was built for the purpose of automating and simplifying front-end tests and development. Basically, anything a person can do on a browser can be automated with Puppeteer.
Some of the actions that Puppeteer can automate are:
- Generate screenshots and PDFs of pages.
- Automate form submission, UI testing, keyboard input
- Change HTML elements in a page
- Create an up-to-date, automated testing environment. Run your tests directly in the latest version of Chrome using the latest JavaScript and browser features.
And much more.
Hopefully, that was enough intro. Now letâs dive into the technical aspect.
1. Initial Setup
As with any other node package, we need to make sure we have installed nodejs and npm. If you are on Linux or macOS, use these two simple commands to check that everything is ready.
> node --version
> npm --version
Next, create a project folder/directory and initialize your node environment. Then install Puppeteer with the npm.
> mkdir medium-automate-proj && cd medium-automate-proj
> npm init --yes
> npm install puppeteer
Finally, make sure everything is working accordingly. Let us try running a sample script from the Puppeteer documentation. The script goes to www.medium.com
then takes a screenshot of the page and saves it as medium.png
.
file: index.js
const puppeteer = require('puppeteer');(async () => {
const browser = await puppeteer.launch({headless:false});
const page = await browser.newPage();
await page.goto('https://medium.com');
await page.screenshot({path: 'medium.png'});
await browser.close();
})();
output: medium.png
2. Automating Login to medium
You canât edit a published story without logging into your medium account. So how do we automate the login process?
Medium provides different ways to log into your account. For our purpose, weâll use the âSign in with emailâ.
After entering your email address, Medium will send you a special link to your email. This link can be used by anyone to log into your account. Note that this link can only be used once and if not used for 2 hours, it will expire.
Your token looks like this
https://medium.com/m/callback/email?token=1a2b3c4d5678&operation=login&state=medium
At this stage, it shouldnât be difficult to understand the following code
const puppeteer = require('puppeteer');(async () => {
const browser = await puppeteer.launch({headless:false});
const page = await browser.newPage();// login_link is the link from email
const login_link = "YOUR LOGIN LINK";
await page.goto(login_link);
await page.screenshot({path: 'medium.png'});
await browser.close();
})();
The above code just goes to www.medium.com
and takes a screenshot and saves it as medium.png
3. Editing a published article
Instead of taking screenshots, letâs do simple automated editing. To accomplish this we first choose a published story. Then try to edit it manually.
When you try to edit a story, Medium will redirect you to a URL similar to âhttps://medium.com/p/96fe4266534f/editâ. This URL is actually my edit page for that story.
After logging into our Medium account we want our code to redirect us to the edit page. So letâs add these two lines to our code
const edit_page = "https://medium.com/p/96fe4266534f/edit";
await page.goto(edit_page);
On your edit page, letâs say you want to edit a line in your story. How do you do that?
Puppeteer provides us with $eval()
function to edit an element on a page. This function accepts two arguments. The first one is an element on our HTML page, and the second one is a function that tells Puppeteer what to do with the element.
For example:
await page.waitForSelector("p[name='a46a']");
await page.$eval("p[name='a46a']", e => e.innerHTML = "WOW");
This code will search for a <p name='a46a' ...>
(and waits until the tag is loaded) then changes the text inside this tag to âWOWâ.
i.e. <p name='a46a'...>WOW</p>
Note that name='a46a'
is different for you. You have to inspect your story to find out the name of your tag.
Now our code is:
const puppeteer = require('puppeteer');(async () => {
const browser = await puppeteer.launch({headless:true});
const page = await browser.newPage();// login_link is the link from email
const login_link = "YOUR_LOGIN_LINK_FROM_EMAIL";
await page.goto(login_link);
const edit_page = "https://medium.com/p/96fe4266534f/edit";
await page.goto(edit_page);await page.waitForSelector(p[name="a46a"]);
await page.$eval("p[name='a46a']", e =>{
e.innerHTML = "WOW";
})//await page.screenshot({path: 'medium.png'});
await browser.close();
})();
But we want more than changing the text to âWOWâ. Maybe we want to display the current time and date. Well, say no more. Letâs use thenew Date
object.
const puppeteer = require('puppeteer');(async () => {
const browser = await puppeteer.launch({headless:true});
const page = await browser.newPage();// login_link is the link from email
const login_link = "YOUR_LOGIN_LINK_FROM_EMAIL";
await page.goto(login_link);
const edit_page = "https://medium.com/p/YOUR_STORY_ID/edit";
await page.goto(edit_page);await page.waitForSelector(p[name="a46a"]);
await page.$eval("p[name='a46a']", e =>{
e.innerHTML = (new Date()).toLocaleString('en-US');
})//await page.screenshot({path: 'medium.png'});
await browser.close();
})();
4. Publishing the edited story
After editing the story, we need to republish our story for the effect to take place. For this, we need to click the save and publish
button on the top right of the page. Puppeteer provides us with the click
function to do the task. The function takes one argument â the HTML element (button) to be clicked.
await page.click("button[data-action='republish']");
But there is one caveat here. One needs to wait for a few seconds for the Save and publish
button to be clickable. So here is an edit for our code.
// wait 6000 miliseconds then click
await page.waitFor(6000);
await page.click('button[data-action="republish"]');
5. The Way forward
Our code so far is
There are a few things to note here.
- First, the
login_link
should be updated every time you run the code. - If you donât want a GUI browser to be launched every time you run your code, you can edit the
headless
attribute.{headless:true}
will avoid launching the GUI (Chromium-browser) and everything will be done under the hood. - Your code updates the story only once. But you can do update every minute or every hour with
setInterval.
To conclude, we were able to automatically edit a medium article using Puppeteer, and it took no more than 20 lines of code. The above code was just to change a paragraph of our story to a DateTime, once. But there is no fun in doing that. I would like to see what creative thing you are able to do!!
P.S. You can find my code here.