EIJAL scoring — English is just another language

Evaluating the quality of your globalisation and finding the big improvement deltas

Taking the most holistic view of globalisation, what I like to call ‘EIJAL scoring’ is perhaps one of the easiest and simplest ways to see the depth/quality of your products current globalisation state. It can also highlight where you might be lacking cultural awareness or where your application of globalisation (internationalisation and/or localisation processes and quality) may be insufficient, and therefore where you could be leaving money on the table.

For more context about globalisation overall and how EIJAL scoring fits into the bigger picture of globalisation, read this post: Demystifying globalisation: Scaling products for a global market.

The rationale for EIJAL scoring

For English-based companies — engineers, product managers, designers and everyone are building products and features for the English language and most likely in an English-speaking geographical market. They’re thinking, designing, programming, building solutions and disproportionately considering English and their home market, relative to the other sets of expectations your global product is exposed to around the world.

This is perfectly normal and not something to be at all too critical of — they are just viewing the world through the window they’ve been doing so for their entire lives, and to try and consider every expectation around the world would cause analysis paralysis and nothing being built anyway. That said, too heavily or only thinking like this does not lend well to achieving global scale either.

In software product development, this actually makes English a good rough proxy for “easy”, “our best effort”, and “ideal state” in the vast majority of cases. This is because globalisation considerations and practices when programming and designing are not required if the company only existed and exposed itself to an English-only home market bubble.

In order for other non-home market locales to be accounted for, the product/codebase must have at least some consideration of how it appropriates itself to account for other locales, and thus have some level of internationalisation and localisation readiness. If not, the product will present poorly and this will be reflected in poor customer behaviour. So for an already global product, this makes non-English locale performance a rough proxy for globalisation quality, when compared against an English locales baseline for conversion-based (CVR) metrics.

Why ‘English locales’ rather than ‘English and home market locale’? While, for example, a UK-based company’s people will mainly be building with en-GB in mind, the en-US locale (United States set of customer expectations) is quite similar to en-GB (United Kingdom’s set of customer expectations). This means that errors with en-US localisation could be overlooked by customers because en-US vs en-GB differences may not cause enough friction for a customer to behave in a negative enough way that will be reflected in an impact metric. This will vary from product to product though.

So in order to not pollute the top level metric, the formula for EIJAL scoring is:

Reading the scoring

As this is a type of parity (equality) scoring, a good score aim is 1.00 — this means that the product impacts customers in an equal way between English and Non-English locales.

Note: There undoubtedly are cultural differences and other complexities which may cause conversion variations. However, near-parity (e.g. say 0.95–1.05) should be achievable for most business areas that have achieved a reasonable degree of globalised product market fit.

A score of below 1.00 means that the product is lacking parity and may be an indicator of poor globalisation or some aspect of globalisation therein.

A score of above 1.00 requires further attention or qualifiers. It may still indicate that the product is lacking globalisation parity between non-English and English locales, just in the opposite way — an over-favouring non-English locales’ needs and expectations against English ones in the product.

A score of exactly 0.00 means that the product is not affecting customers in Non-English locales and therefore has no data-points to know if they are globalised sufficiently, though as soon as they release to a non-English locale, this sufficiency should be reflected via the scoring. This is common for proof-of-concept products or features that might be limited to one or more English locales only for some time.

For reference, a score of 0.50 means that the product performs half as well when it is used in a non-English context!

This just presents you with a surface level score representing how much your product stands by the statement that “English is just another language”. It gives you a starting place to drill down on via the same parity-based methodology to find more specific blindspot deltas that can be captured to stop leaving money on the table.

An example

Let’s say you own the shopping piece of an eCommerce product that has a dedicated aim to inform a purchase decision and convert customers from your shopping experience into a checkout, from which the checkout can then convert customers into purchasers.

You apply EIJAL scoring and find that your shopping experience product has a score of 0.75 — this means your product performs 25% worse when not in an English context. Something seems awry. You dig down and find that English vs Chinese locales has 0.50 score for the same metric (Chinese performing 50% worse than its English counterpart). The Chinese traffic is not insignificant, so it is likely one of the reasons for the 0.75 overall EIJAL score. It appears there may be a big delta to be gained here by improving something in the shopping experience for Chinese customers.

While the metric itself doesn’t tell you the specific problem to be solved, it helps you find where to look for important globalisation problems. You can then put yourself into the shoes of a Chinese customer by speaking to real local customers or localisation experts and gathering insights to understand the reason(s) for the behaviour. It may be that your product has poor internationalisation so your translations are not coming through properly, it may be that your experience isn’t meeting some essential Chinese cultural expectations of a shopping experience resulting in decreased customer trust to the extent that it causes drop-off, or some other of many reasons.

The main point is that from here you have a normal product needs and expectations investigation on your hands from which you can prioritise based on impact. If you improved the score to 0.80 for English vs Chinese, that’s a 60% parity improvement, and means your product now only performs 20% worse than its English-locale counterpart, compared to 50%. For simplicity, assuming your checkout has great globalisation and converts checkout to purchase at a rate of 50% irrespective of locale, an improvement to 0.80 for shopping will amount to a 15% shopping through to purchase conversion increase for Chinese customers. That could be a lot of money you just picked up from being left on the table, and a massive return on investment.

Just one tool in the globalisation toolbox

While EIJAL scoring is a good way to identify problems in your current state of globalisation (find ‘cures’), it isn’t a substitute for ongoing globalisation efforts at the roots of your global product development process.

Unfortunately — like in medicine — finding ‘cures’ simply isn’t nearly as sustainable nor scalable as preventing issues from arising in the first place. After all, the main aim of quality globalisation (involving good internationalisation and localisation practices) is to build in such a way that is consistently, quickly and sustainably able to meet the local needs and expectations of customers in targeted global markets. That actually means trying to avoid the occurrence of big deltas for improvement arising in the first place via ongoing globalisation efforts in the product development process.

Nonetheless, EIJAL scoring is a valuable tool to evaluate the current quality of your globalisation, reflect and iterate on your globalisation practices to avoid future issues, and find places where you could be leaving a lot money on the table.