“… I do not go up and down at a 360-degree angle, […] You mean to tell me that China is under us? China is under us? It’s not. The world is flat.”

— The Big Aristotle

I was 3 when the first Toy Story movie came out, and about 8 when my second grade teacher wheeled out the ye old television set on a random afternoon. In a time before iphones, and tablets, a TV with its own cassette player built in was the shit.

My favorite part, was the split second when Buzz, defiant of the whole world going against him, grits his teeth, and jumps off the stairs, goes into slow motion, and despite all the facts, actually flies. He then realizes his mistake, and crashes to the ground, the impact of which, amputates his left arm.

There is something to be said about people who are willing to go toe to toe against el mundo, believing they’ll come out on top. In this case Mr. Lightyear is 15 time All-Star and 4 time NBA champ, Shaq Diesel, and gravity is played by the baddest S.O.B that ever walked the mean streets of the Greater Muscat Area.

We Don’t Get Along So We’re Gonna Get It On!

“Man what a bunch of clowns at these airlines, wasting my time with these darn parabolas, get me a ruler so I can just go from Abu Dhabi to LA without bugging Saint Nick!”

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

“Dear Dr. Big Daddy Shaqtus,

You are correct.

It does appear that these airlines are wasting your time flying inefficient routes. A straight line, as you suggest, connecting two cities would indeed result in a much shorter flight path. Only, that is, if the world looked anything like this here map.

The fact that the flight path is curved is proof that the world is not flat, those pilots aren’t fooling you, the fastest way to travel between LAX and AUH is over Greenland.

A straight line on top of a sphere looks like a curve on top of a flat plane.”

Fine, The Earth is a Sphere Then

That was easy.

Now that the Earth is in 3 dimensions it’s clear to see how the strange parabolic flight path was reached. Instead of scrolling left and right, going up and over, connects AUH to LAX in a straight line right along the top of Greenland.

The shortest distance between any two points is always a straight line, and can be calculated using the Great Circle Distance Formula.

The Great Circle in this case is the curve uv, as it is defined by having a radius and center equal to that of the sphere, cutting it in two. In other words the shortest distance between points P and Q is the line that can connect the two points and falls on the Great Circle.

Nothing really ground breaking here, basically the shortest distance between two points on a sphere is a straight line.

Calculating the distance between P and Q is where we get to actual meat and potatoes.

Spherical Law of Cosines

Very similar to the planar law of cosine, just adjusted to fit into a spherical format.

Distance vw is found by first connecting vertices of a spherical triangle along Great Circle Arcs.

Notice vertex u is 90 degrees to the equator, in this case it is the pole of this unit sphere. We don’t know distances for a or b, but we do know the angles that are subtended by the 90 degree pole from the vertex positions. Armed with these angles all that’s required is a little cosine formula magic, and we’ve calculated the distance.

Plug and Chug, Mayor McShaq

Example: (AUH To LAX)


AUH -> 24.4419° N, 54.6501° E

LAX -> 33.9416° N, 118.4085° W

sphere_distance <- function(lat1, long1, lat2, long2) {
b<-(90 - lat1)*(pi/180) ## degree to radian
c<-(90 - lat2)*(pi/180)
A<-(long1+long2)*(pi/180) ## Angle to the pole
d<-(sin(b)*sin(c)*cos(A)) + (cos(b)*cos(c)) ## cos(d) in radian
d<-(acos(d)) * (180/pi) # d in degrees
distance<- d * 60 # final distance in nautical miles
distance<- distance * 1.852 # final distance in kilometers
> sphere_distance(24.4419, 54.6501, 33.9416, 118.4085)
[1] 13472.72
## Make sure to add negative signs for the southern hemisphere

Guess what, according to the smart folks at Google, the distance from AUH to LAX is 13, 473 Km! Pretty darn close, also I have been on this flight, it sucks.

Alright cool, we’ve proven The Big Galactus wrong, the Earth is a sphere, and those airplane pilots don’t all need a body slam through a table. Yet…

Well not exactly, the earth has mountains and valleys distorting the perfect spherical shape we assumed earlier, it’s also rotating and being pulled toward the moon via tidal forces creating bulges.

So we’re both kinda wrong, the earth is more of an ellipsoid, which complicates things a bit.

Vincenty inverse formula for ellipsoids

Luckily Thaddeus Vincenty in 1975 created two formulas that un-complicates that for us. It effectively calculates the shortest distance between two points (like above) but instead of working on a perfect sphere it works on an ellipsoid.

Having a formula that is based on a shape more closely resembling our planet in real life, generates a more accurate distance calculation, especially for shorter distances.

These formulas are what’s used to determine how far a drive to the nearest McDonald’s is from your apartment, or how far you would need to run to evade the authorities and get to the border.

Even more good news is that Vincenty’s formula has already been implemented by another academic by the name of Mario Pineda-Krch. And through the magic of copy and paste, you can see a reflection of his hard work, and my laziness and lack of morals below.

As you can see its a little more fiddly than the calculation based on a spherical assumption, but it is accurate to the millimeter thanks to its iterative nature, which also is the reason why it is a little slow.

Sydney to San Francisco (~11963 Km)
> sphere_distance(-33.8688, 151.2093, 37.7749, -122.4194) 
[1] 11939.61
> vincenty_distance(-33.8688, 151.2093, 37.7749, -122.4194)
[1] 11934.47
New York to Paris (~5840 Km)
> sphere_distance(40.6413, -73.7781, 49.0097, 2.5479) 
[1] 5829.55
> vincenty_distance(40.6413, -73.7781, 49.0097, 2.5479)
[1] 5849.151
Cape Town to Johannesburg (~1274 Km) 
> sphere_distance(-33.9249, 18.4241, -26.1367, 28.2411) 
[1] 1533.023
> vincenty_distance(-33.9249, 18.4241, -26.1367, 28.2411)
[1] 1280.118

As you can see Vincenty’s is a lot more consistent, you can see it doesn’t drop off in accuracy like spherical distance does as the distances become smaller and smaller.

Now there are a bunch of libraries that can do these calculations for you, but I think it is important to know where they come from.

They say its a small world, I think it would be useful to know how small.

So what have we learned today:

  1. The Earth is not flat
  2. The Earth is not a sphere
  3. You have to respect Shaquille O’Neal, it’s not easy standing up for what you believe in, especially when everyone says it’s wrong.
  4. Predicting distances is not as easy as it looks, especially when you need them fast
  5. I need to take a trip to South Africa soon, Cape Town looks cool
  6. You’ve got a friend in me.
Show your support

Clapping shows how much you appreciated Syed Naqvi’s story.