Marketplacer and its websites served user-provided web-optimised images from the same webserver as its actual application, and stored both the original image and the web-optimised versions on disk. Each image’s filename was stored in a column on the related model, and uploads were handled with the Ruby gem carrierwave.
Marketplacer’s websites would often have many duplicate images stored against database records and on disk:
This post describes the structured logging setup used at Marketplacer to publish information about each request to our Rails application to our ELK stack.
Our primary goal for our ELK platform was forensics & long term usage analysis of back-office functionality. For example, we wanted to know:
As a developer, designer or other direct contributor to the product creation process, there’s a single principle that drives morale, productivity and motivation:
I feel that I am effectively working on something I find interesting and meaningful in a safe environment.
If these conditions are not at least mostly satisfied, other perks like team lunches, hack days or remote working options become less relevant. Annoyances like noisy open plan offices, frequent interruptions or a long commute have a greater effect.
It is not possible to completely satisfy all of these conditions for all contributors. Some of the items mentioned in…
Our full-system integration tests take too long to run: between five and eight minutes in a highly optimised CI environment, and over an hour if run on a developer’s laptop.
One of the ways we’re trying to manage this is by logging which tests take the longest and then printing them out at the end of the run, eg:
In this particular run
refunds_invoices_spec.rb is taking three minutes to complete. If we divide that by the eight workers on that CI instance, that’s adding a good 22 seconds to each run, assuming a perfect distribution of tests between workers.