better shoeshopping through technology
three things about me:
- i like to find bargains
- i often overthink nerdy solutions to problems
- i like to wear shoes.
my favourite place to buy shoes is a skate shop down the road from me. they don’t have a lot of shelf space in the front of their store, so their sale shoes are often stocked in the back of the store, and can only be found on their website. their website doesn’t have builtin search, so it’s hard work to find shoes that are available in my size, and because the website is fairly slow, it can be frustrating to find something i like.
last weekend i decided i wanted to find a bargain on their website, but i didn’t want to get stressed out navigating it by hand. i initially thought i’d write a little scraper to click every link, create an array of all the shoes available in size 13 with their names, URLs and prices, but after writing maybe ten lines of ruby i decided this was far too much effort. i thought, “if only i had a fast, searchable mirror of this website”… and then it clicked.
to the command line!
first i ran
wget -mbc -np -p -rk --no-clobber <url>/show
which traversed the website and created a static mirror of it. i had never used wget for mirroring a website before, and there’s probably a more optimal approach, but this worked. i stepped away from the computer for a little while and when i returned i had a folder structured just like the website.
i then ran
ack 'option value="13"' -l \
| xargs grep Product_FormattedPrice \
| grep -v jQuery
on this folder, to get a list of prices and shoes available in size 13 (without the wacky inline JS on each page cluttering my results).
i combed this narrow result for the shoes i liked the look of most. a few days later I went to the store, and…
…i bought them!
lessons learned / notes to self
- don’t overthink solutions to technical problems. there are lots of standard-issue utilities that can get you most of what you’re looking for. for example, you don’t always have to write a scraper from scratch. shoe-searching does not require a clean, beautiful API.
- the vans old skool 92 pro is a very good looking and comfortable shoe.