GroceryJS: JavaScript Your Groceries

Breaking Down Grocery Shopping

Shopping for groceries is a procedural activity:

  • You look through your fridge and pantry for things you need and make a list
  • You walk, bike, or drive to the store
  • You walk the aisles adding things to your cart as you see them
  • Finally you pay and bring your groceries home
  • Read items from a list
  • Search and add items to your cart
  • Prepare the cart for checkout & checkout

Breaking Down the Script

GroceryJS is broken up into several pieces of code after plenty of experimentation:

Source

The source is the data backend, it’s where the grocery list is held. It also is the holding place for the results of a grocery store run. For, GroceryJS, I started with a text (YAML) file, then transitioned to a Google Sheet. I found the Google Sheet to be something that is accessible from everywhere, desktop & mobile, without needing a bunch of UI. Google provides a pretty robust set of Nodejs libraries you can use to interact with the Google Drive and Sheet APIs.

async getGroceryList() {
let spreadsheetId = this._spreadsheetId;
let sheetsService = this._sheetsService;
return await new Promise((resolve, reject) => {
sheetsService.spreadsheets.values.get({
spreadsheetId: spreadsheetId,
range: 'A1:C50'
}, (err, result) => {
if (err) {
reject(err);
} else if (result && result.data && result.data.values) {
let items = [];
for (let i = 1; i < result.data.values.length; i++) {
let value = result.data.values[i];
items.push({ name: value[0], quantity: value[1] });
}
resolve(items);
} else {
resolve([]);
}
});
});
}
async addShoppingResults(title, sheetId, results) {
let sheetsService = this._sheetsService;
let spreadsheetId = this._spreadsheetId;
return new Promise((resolve, reject) => {
let requests = [];
let idx = 1;
// convert results to an array we can write
let data = [];
let headers = [
{ userEnteredValue: { stringValue: 'Requested' } },
{ userEnteredValue: { stringValue: 'Item' } },
{ userEnteredValue: { stringValue: 'Price' } },
];
data.push({ values: headers });
for (let i = 0; i < results.length; i++) {
let result = results[i];
let row = [];
row.push({ userEnteredValue: { stringValue: result.requested } });
if (result.result) {
row.push({ userEnteredValue: { stringValue: result.result.title } });
row.push({ userEnteredValue: { numberValue: result.result.price } });
}
data.push({ values: row });
}
// add the sheet
requests.push({
addSheet: {
/* removed for brevity's sake */
}
});
// updateCells request
requests.push({
/* removed for brevity's sake */
});

// auto size things
requests.push({
/* removed for brevity's sake */
});
// execute the batch update
sheetsService.spreadsheets.batchUpdate({
spreadsheetId: spreadsheetId,
resource: { requests: requests }
}, (err, result) => {

if (err) {
reject(err);
} else {
resolve(`https://docs.google.com/spreadsheets/d/${spreadsheetId}/edit#gid=${sheetId}`);
}
});
});
}

Shopper

The Shopper contains all the code and actions that make up a successful trip to the grocery store. It’s built on top of utility library I wrote called puppet-helper.js.

Puppet Helper

The Puppet Helper contains all of things needed to interact with a modern web app, like clicking a button, given a CSS selector:

async clickButton(selector, clickCount = 1) {
this.assertPageOpen();
let button = await this._page.$(selector);
if (button) {
await button.click({ clickCount: clickCount });
} else {
throw new Error(`Unable to click ${selector}`);
}
}
async getTextFromElement(element) {
this.assertPageOpen();
return await this._page.evaluate(el => el.innerText, element);
}

The Lowes Shopper

More and more grocery stores offer online shopping services on the internet, allowing customers the convenience of shopping from their computer, tablet, or mobile phone. We shop at Lowes Foods, a North Carolina-based grocery store chain. Lowes Foods offers an online shopping service, Lowes Foods To Go. For $49-$99 annually (or $4 to $5 per order), you can order your groceries using their web app. Once you place your order, a Lowes Foods employee will shop your order and call you when they are finished (or if they had any questions). When the order is complete, you can pick it up or have it delivered.

async searchImpl(query) {
this._logger.info(`Searching for ${query}`);
let productDivs = null;
await this._puppetHelper.clearText('#search-nav-input');
await this._puppetHelper.enterText('#search-nav-input', query);
await this._puppetHelper.wait(SHORT);
await this._puppetHelper.clickButton('#search-nav-search');
await this._puppetHelper.wait(MID);
// body > div:nth-child(5) > div > div > div.content-wrapper > div > lazy-load > ol
let resultsDiv = await this._puppetHelper.getElement('ol.cell-container');

if (resultsDiv) {
productDivs = await this._puppetHelper.getElementsFromParent(resultsDiv, '.cell.product-cell');
}
return productDivs;
}
async login(email, password) {
this._logger.info(`Logging into account ${email}...`);
await this._puppetHelper.goToUrl(SHOPPING_URL);
await this._puppetHelper.clickButton('#loyalty-onboarding-dismiss');
await this._puppetHelper.wait(SHORT);
await this._puppetHelper.clickButton('#shopping-selector-parent-process-modal-close-click');
await this._puppetHelper.wait(SHORT);
await this._puppetHelper.clickButton('#nav-register');
await this._puppetHelper.wait(SHORT)
await this._puppetHelper.enterText('#login-email', email);
await this._puppetHelper.wait(SHORT)
await this._puppetHelper.enterText('#login-password', password);
await this._puppetHelper.wait(SHORT)
await this._puppetHelper.clickButton('#login-submit');
await this._puppetHelper.wait(XLONG);
}
async showCart() {
this._logger.info(`Opening the shopping cart...`);
await this._puppetHelper.clickButton('#nav-cart-main-checkout-cart');
await this._puppetHelper.wait(MID);
}

async emptyCart() {
this._logger.info(`Emptying cart...`);
await this.showCart();
await this._puppetHelper.clickButton('#checkout-cart-empty');
await this._puppetHelper.wait(NANO);
await this._puppetHelper.clickButton('#error-modal-ok-button');
await this._puppetHelper.wait(MINI);
}

Putting it Together

With all the pieces mentioned, I can have GroceryJS, prepare a shopping cart full of groceries. When it’s done, it sends me an email with a link to the cart (so that I can quickly checkout) and a link to the Google Sheet for tracking purposes.

(async () => {
let shopper = null;
try {
let sheetSource = new SheetGrocerySource(logger, credential.client_email, credential.private_key, config.source.sheetId);
await sheetSource.init();
let list = await sheetSource.getGroceryList();
// login and create a blank slate to shop
shopper = new LowesShopper(logger);
await shopper.init(config.shopper.headless);
await shopper.login(config.shopper.email, config.shopper.password);
await shopper.emptyCart();
// do the shoppping
let shoppingResults = [];
for (let i = 0; i < list.length; i++) {
let requestedItem = list[i];
let shoppedItem = await shopper.addItemToCart(requestedItem.name, requestedItem.quantity);
shoppingResults.push({ requested: requestedItem.name, result: shoppedItem });
}
// notify
let dateStr = moment().format('MMMM Do YYYY @ h:mm a');
let title = `Shopping Trip on ${dateStr}`;
let urlToCart = 'https://shop.lowesfoods.com/checkout/cart';
let urlToSheet = await sheetSource.addShoppingResults(title, moment().unix(), shoppingResults);
let emailBody = `
<span><b>Shopping Cart:</b> ${urlToCart}</span><br />
<span><b>Shopping Results:</b> ${urlToSheet}</span>`;
let mailOptions = {
service: config.email.sender.service,
user: config.email.sender.email,
password: config.email.sender.appPassword
};
mailUtil.sendEmail(config.email.recipeint.sender,
config.email.recipeint.email,
title, emailBody, mailOptions);
} catch (e) {
logger.error('Error while shopping', e);
} finally {
if (shopper) {
await shopper.shutdown();
}
}
})();

Conclusion

So that’s it. GroceryJS isn’t done yet. The real work is actually in the details, like the algorithm for adding groceries from the search results to your cart. Lowes Foods To Go has it’s own search algorithm for determining the relevance of a result to a search. In many cases their algorithm will not match expectations, but it can be augmented:

  • Should GroceryJS prefer groceries that are on sale?
  • Should GroceryJS prefer groceries for a specific brand?
  • Should GroceryJS prefer groceries I’ve purchased before?

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Evan

Evan

I really love solving fun and interesting problems with software. https://evanhalley.dev