Image by Timo Wielink via Unsplash

Understanding Travel Risk Advice: Part One

How to spot misleading and dangerous destination risk ratings

Most of us ignore the question of how dangerous our environment is until we find ourselves in an unfamiliar location, uncomfortable circumstances, or when planning a trip abroad. When considering a travel destination, our safety and security rank higher as a deciding factor depending on our demographics, personality, travel experience, and any opinion we might already hold of the destination.

Female travelers and families with children pay greater attention to crime and safety information than single males, especially younger ones, even if the latter are more likely to become victimized or injured because of their behaviors and choices as tourists. Cities in Central and South America and most of Africa are frequently associated with greater violence and challenging environments. So do cities that have recently made the news for significant incidents or social unrest (think Paris at the end of 2018 with the yellow vests movement, or Hong Kong and the pro-democracy demonstrations in the summer of 2019.) Most of these destinations will elicit some effort to gather more risk and safety information.

Unfortunately, the ‘risk intelligence’ that you’ll gather by googling “how safe is…”, digging through Social Media posts, or soliciting Reddit advice, may very well quell your curiosity — but chances are it will be wildly inaccurate. It can likely put you in the unenviable position of either vastly underestimating, overestimating, or altogether mistake the dangers to you and your family.

Crowdsourced and sentiment-based risk ratings can be dangerously at odds with data-driven and evidence-based scales.

We’ll explore how severe bias, noise, and methodological flaws plague most open-source and commercial travel risk information products to the point that one can assume their value to be largely illusory. Any confidence in judging a destination risk that this risk intelligence may offer you has likely little in common with your chances of having an accident, getting sick, or becoming the victim of a crime.

The good news is that by understanding the main fallacies and shortcomings of personal and travel risk information and ratings, you’ll be able to immunize your judgment against some of the nonsense. Ultimately, you can develop a more acute and realistic awareness of valuable versus useless or even dangerous data or insights.

So, whether you are researching destination risks (and traveling) on your own, or are a recipient of commercial travel risk information products from your employer’s Duty of Care or Travel Risk Management (TRM) programs, beware of one or all of these shortcomings and fallacies:

• Arbitrary and qualitative risk ratings and scales with little or no data to support them
• Poor geographic or demographic specificity resulting in a fallacy known as the flaw of averages
• Emphasis on anecdotal, recent, or significant events which amplify our risk judgment biases and innumeracy rather than mitigating them

Let’s dive a little deeper into the first point above:

Nominal and/or ordinal risk ratings and scales

Assume the vast majority of the travel risk and safety ratings you find will have the same rigor and quality of a restaurant’s star rating on Google Maps — where most of the Italian restaurants in downtown Tulsa (OK) are four stars or higher! (Sorry for picking on you, Tulsans, but I suspect the Italian Food Academy would take issue with that conclusion, and that if your and your family’s life and safety depended on it, you’d all be in favor of a more disciplined rating system.)

Consider one of the most widely known examples: the US Homeland Security Advisory System (rather than diving into the many technical fallacies and inadequacy of nominal and ordinal scales in risk measurement.) Think how going from a Yellow/Elevated condition to Orange/High affects your understanding of this threat (beyond the fact that the latter is higher or worse than the former.) Can you figure out how it impacts you? How does it compare to other risks — such as being killed in a robbery or getting shot at in a road rage incident? And can you articulate how it will affect/change your behavior? What is it that you do differently in these situations?

Asking yourself these questions can help you overcome and avoid the trappings of the initial sense of mental ‘comfort’ derived from hearing about a ‘low’ or ‘high’ risk level — which plenty of research has shown to be basically uninformative and only offer the illusion of communication. If you can’t come up with satisfactory answers, remember the problem is not you, it’s the scale!

Over nearly forty years, the Travel Risk Intelligence (TRI) industry has grown into a multi-billion dollar business, helping large employers manage their duty of care liability. It also argued that a qualitative rating scale for travelers’ risk (i.e. ‘Brazil is an Orange/Medium country’) is better than no scale. Unfortunately, today we know this to be untrue.

The US Department of State is an excellent source of risk and safety information for the international traveler, but their rating system is unfortunately plagued by the same ordinal scale shortcomings affecting most of the travel risk and security industry, and can be dangerously misinforming when taken at face value.

Let’s try one quick, six-step exercise that will (hopefully) convince you of the problem associated with ordinal/nominal risk scales:

1. Visit the US Department of State Travel Information page for Italy.
2. Notice the Advisory Level at 3 (Orange) “Reconsider travel to Italy due to COVID-19”. Exercise increased caution due to terrorism.
3. Next, perform the exact search for El Salvador and find: 2 (Yellow) “Exercise increased caution… due to crime and COVID-19”.
4. Imagine having a genuine interest in visiting both countries and no other source informing your risk judgment: all signs seem to point to El Salvador as the more intelligent, safer choice for you and your family, right?
5. Now, what if you learned that El Salvador’s national murder rate has ranked among the highest in the world for almost a decade, ranging between 20 and 40 times that of Italy? (The increase of likelihood of violent crime victimization in El Salvador vs. Italy is anywhere between 2000% and 4000% — depending on age and gender.) And that…
6. El Salvador’s COVID-19 mortality rate has been five to seven times that of Italy’s? (At least at the time of this writing and for most of Summer 2021.)
7. Would this change your perception of the relative risks at these destinations? How would it affect your choice?

“Many open-source and commercial travel risk information products are plagued by severe bias, noise, and methodological flaws to the point that one can assume their value to be largely illusory.”

How can you spot a credible and helpful destination risk rating and avoid biased and misinforming ones?

• Stick to ratings and indices with clear and specific risk data, language, and methods.
  A ‘risk to travelers’ scale sounds right and closer to what you are presumably looking for, but it tells you nothing about what it measures. Infectious disease risks? Accidents and injuries? Or is it personal crimes? If so, does the rating refer to homicide or overall violent crime victimization? Or is it about street crimes like purse-snatching? How about the risk of gender or sexual orientation harassment and discrimination? Most of these risks carry substantively different impacts, and few are meaningfully correlated (requiring separate scales.) The more generic or vague the definitions, the lower the scale’s credibility and value.

Effective travel and personal risk rating systems are very specific about what they measure (i.e. Violent Crime), are based on incident numbers and rates, and present both absolute and relative metrics (i.e. home vs. destination/index). Image courtesy of Safe-esteem

• Always look for both absolute and relative measures.
  Headlines suggesting that murders or violent crimes at your destination have gone up 75% in the last year may get your attention and could be indicative of a trend, but will tell you nothing about the actual victimization risk. Professionals pay attention to victimization rates in the form or number of incidents per 100,000 because it enables us to interpret the raw numbers relative to a standard population set. Chicago, for example, is constantly making headlines with its annual 700+ murders, less so than New Orleans’ 190+ murders. Yet, the latter translates in a homicide rate of nearly 50 per 100K vs. Chicago’s 28 (making New Orleans’ homicide risk nearly twice that of Chicago and one of the highest for any city in High-Income nations.)

In part two, we’ll look closely at two additional sources of travel risk disinformation: the so-called “flaw of averages,” along with judgment bias, noise, and unhelpful heuristics — all critical in shaping more accurate travel risk insights and decisions.