Here's what you can do with node.js crawler and Google Auto Email.
So back in few days I was applying for Masters in China and there a requirement in there universities admission to get acceptance letter from universities professor for admission, china is big country and they got a lot of universities so I went to some of their top university website and searched for professors, and the problem was that there university website got only list of professor not specified any field so I have to go through every professor profile and find my interested field professor, Well there was too much repetition then I thought why do we do programming to overcome repetition, then I came up with the idea of using crawler and sending auto emails to all professors.
Getting data from websites
So I got data of professor list from Shanghai Jiao Tong University website, here the code for getting data
for (let a = 0; a < 500; a++) {c.queue("http://en.sjtu.edu.cn/academics/faculty/teachers/" + a);}
just got a list of professors url to get data.
Then I used javascript crawler library to crawl through these URLs.
Now I got all my desired data in `arrObj`
Saving data into google spreadsheet
Now I saved data to google spreadsheet so I can send auto emails to professors, here the code to save data to google spreadsheet:
sheets.spreadsheets.values.append({
spreadsheetId: "1RtPpJRRK6y5Eeux_xrs3kXglg2IaE04RWV_crM",
range: "Sheet2!A2",
valueInputOption: "RAW",
insertDataOption: "INSERT_ROWS",
resource: {
// values: [[data.name, data.email, data.field]]
values: data
},
auth: oAuth2Client
},
(err, response) => {
if (err) return console.error(err);
else console.log("response", response.data);
}
here’s google spreadsheet link to follow further there documentations https://developers.google.com/sheets/api/guides/values
Sending Auto Emails to Professor
Now I got data into spreadsheet and have to send email to all professor for that google got an option called google spreadsheet script so you have to add a script for that.
Here’s code of script :
So this how I did a lot work through just programming in node.js and javascript.
Here’s github link of project : https://github.com/SalahudinMalik/nodejs_crawler
Thanks for reading, hope it helps you solve your repetition problem.