Here's what you can do with node.js crawler and Google Auto Email.

So back in few days I was applying for Masters in China and there a requirement in there universities admission to get acceptance letter from universities professor for admission, china is big country and they got a lot of universities so I went to some of their top university website and searched for professors, and the problem was that there university website got only list of professor not specified any field so I have to go through every professor profile and find my interested field professor, Well there was too much repetition then I thought why do we do programming to overcome repetition, then I came up with the idea of using crawler and sending auto emails to all professors.

Getting data from websites

So I got data of professor list from Shanghai Jiao Tong University website, here the code for getting data

for (let a = 0; a < 500; a++) {
c.queue("http://en.sjtu.edu.cn/academics/faculty/teachers/" + a);
}

just got a list of professors url to get data.

Then I used javascript crawler library to crawl through these URLs.

code to get data from crawler

Now I got all my desired data in `arrObj`

Saving data into google spreadsheet

Now I saved data to google spreadsheet so I can send auto emails to professors, here the code to save data to google spreadsheet:

sheets.spreadsheets.values.append({
spreadsheetId: "1RtPpJRRK6y5Eeux_xrs3kXglg2IaE04RWV_crM",
range: "Sheet2!A2",
valueInputOption: "RAW",
insertDataOption: "INSERT_ROWS",
resource: {
// values: [[data.name, data.email, data.field]]
values: data
},
auth: oAuth2Client
},
(err, response) => {
if (err) return console.error(err);
else console.log("response", response.data);
}

here’s google spreadsheet link to follow further there documentations https://developers.google.com/sheets/api/guides/values

Sending Auto Emails to Professor

Now I got data into spreadsheet and have to send email to all professor for that google got an option called google spreadsheet script so you have to add a script for that.

Here’s code of script :

So this how I did a lot work through just programming in node.js and javascript.

Here’s github link of project : https://github.com/SalahudinMalik/nodejs_crawler

Thanks for reading, hope it helps you solve your repetition problem.