This article is part of a series that has been written by our apprentice Celestine. She is currently taking an apprenticeship at Peerigon to become a professional web developer. As part of her weekly routine, she publishes a blog post each week about what she has been doing and what she has learned.
Problem to solve
With the GDPR taking effect, we got a lot of emails from our users that wanted their account at roomieplanet.com to be deleted. Unfortunately, the design of our legacy PHP application did not allow us to just execute it from the command line. We needed to do it manually which involved a lot of tedious steps: We would need to generate a new password, update the database, log in on behalf of the users to finally delete them by clicking the “Delete me” button for them.
My task was to write a Node.js script that automates these steps. My mentor and I decided to divide the problem in two separate tasks. The first one was to login as the user and delete the account. The second one was getting a connection to the database, look for the user, and change the password.
Solution
My biggest problem was that I had no idea how to access websites by code. Turns out that you can send regular HTTP request from Node.js — just like your browser does.
We didn’t succeed with our initial attempt to just send a POST
request to the authentication route, though. The server rejected the request without giving an error message.
Not knowing what data and attributes the server expects, we thought we had to extract PHP’s session_id
from the response of the initial GET
request to insert it into the following POST
request. Using the request-promise-native package, there is the possibility to “collect” the cookies in a jar like this: let jar = request.jar()
. So I only had to figure out how to actually send requests using that jar, which appears to be rather easy.
That is the GET
request to visit the website with the cookie jar
:
request.get("https://roomieplanet.com", {jar});
Now we have a proper session_id
. And
request.post("https://roomieplanet.com", {
jar,
form: {
login_email: "example@mail.com",
login_password: "password",
login_submit: "SomeAction",
},
followAllRedirects: true,
});
is the POST
request to perform actions on the website, using the assigned session_id
(in this case logging in). In the form
part you insert only what you want to send. Therefore, the contents of the object depend on what you want to do and what form data your website expects. After login, we just had to send another POST
request to finally delete the user.
The database part was easier for me because I had a lecture about databases, so I only had to read up on how to use the Knex package and what queries to use.
What I learned
- Google in case there are packages which make your life easier.
- Be precise when googling, otherwise you don’t get the matching results.
- Asynchronous programming is important in JavaScript (async, await).
- If you don’t know what servers are expecting, it’s trial and error.