That definitely seems like the most logical explanation to me!
I mean, guess what? If you’re used to walking around in East St. Louis, you’re going to pay attention to the world around you if you want to survive until your next birthday! If you live in a cushy suburb? Paying attention to your surroundings is mostly optional. The same applies to facial expressions — being able to read the facial expression of the guy selling drugs on the corner is important! If that guy’s in a bad mood, you NEED to know so you can watch out. On the other hand, if the soccer mom in front of you at Whole Foods is in a bad mood, it’s probably not going to have any direct impact on your survival, so you don’t really need to pay much attention to her facial expressions. That doesn’t make you less empathetic, it simply means that your brain is responding reasonably to the safety of your environment.