Just this past summer, the internet had a freakout, as it is wont to do. On this occasion, the digital collective was cheering because one woman had “the best” interaction with her CEO when she said she was taking two days off for her mental health. The CEO thanked her for doing so and for “[cutting] through the stigma,” and reminded employees they were entitled to do the same. Praise hands emoji!

The cartoonishly joyful rapture of that split second belies the very reason for the story’s virality: It is not easy to get time off when you need to take care of yourself, and there is a stigma associated with it. Despite society’s filtered veneer of celebrating wellness (posting photos of healthy meals on Instagram, displaying a performative vested interested in health), going to doctor appointments, caring for one’s children, or taking any time to care for oneself — all of which promote actual, true wellness — can often get brushed aside. In colder terminology, the idea of being sick, weak, or unproductive for an hour is just not really helpful to the bottom line!

It should be fairly obvious to the discerning soul that prioritizing employee wellness would lead to a more sustainable workforce: If your workers aren’t dropping ill with bubonic plague (or from exhaustion), they’ll be with you for a longer haul. Besides, having flexibility and empathy is crucial, and logical, for a solid performer who hasn’t let anyone down yet. “If it’s a good performer who is dependable, I don’t think there’s anything [wrong with taking time] to clear their head or mental health or get back on track or whatever that means for that person,” says Heather McCloskey, CEO of McCloskey Partners, a human resources firm specializing in recruiting, training, and promoting retention.

In theory, an easy solution to mitigate these problems would be to allow flexible scheduling, which can benefit parents who need to care for children and anyone who falls ill, has a doctor appointment, or needs a day off to breathe. A true flexible scheduling policy, McCloskey says, is one that doesn’t count physical hours in the office, but instead measures success by whether and how well a deadline is hit.

In some companies, however, there is an unspoken stigma about going to the doctor and taking care of oneself — or taking time away during the typical workweek — that can make it difficult for actual, true, necessary self-care. For one, we’re “a culture of face time,” says Jennifer Glass, PhD, a professor of sociology at the University of Texas at Austin. “I don’t think at present there is a way to use flextime in most jobs [with] supervisory situations without facing a stigma,” she says.

The problem is that instating a flat, generic flextime policy can mask what’s actually going on in a company: overwork. “I think that sometimes we place too much emphasis on flextime, rather than really looking at the culture of overwork,” says Glass. It’s similar to the idea of unlimited paid time off — inevitably, people will end up taking the low end, she says. Considering there was a recession less than a decade ago, people will operate with a sense of fear and err on the side on caution.

When people do take flextime, it’s often women who end up getting the shaft. Glass has found in her research that an employee’s reasoning for taking time off can greatly influence the way it’s perceived: Taking time to care for a family, or any act of caregiving, faces greater difficulties than, say, taking continuing education courses or pursuing an advanced degree. There are “significant biases against people who take care of others,” Glass says. That’s often women — men frequently get praised when they need family time. A study from 2014 proved that men end up being viewed more favorably than women in the eyes of their employers when they seek work-life balance.

Practically speaking, it’s not always so easy, even on an hourly or work-from-home basis. One woman I spoke with told me she had to get an IUD put in during her lunch break, and because the appointment ran late, she felt she had to hide her discomfort. (From a practical standpoint, it’s not always easy to travel to a doctor’s office during a break; from an ethical standpoint, it assumes there’s a one-size-fits-all time frame for all doctor appointments and that minimal procedures are involved.) Another woman I spoke with told me about when she had a period that lasted for 17 days. “I had to fight to get some days recognized as [being allowed to] work from home,” she says. “They eventually gave in, but it was incredibly embarrassing for me. I had to write out an entire detailed letter with references from my gynecologist proving it had happened.” (Paid menstrual leave has been slowly spreading around the globe, though it would be nice to have transparency or a sensitive manager who just gets it rather than having to explicitly state it’s time to bleed.)

Few employees want to go through divulging specifically why they need to take an hour or a day or 17 days off, whether it be for a child’s case of mono, an IUD, an abortion, or an extended, painful period. Aside from the privacy issue, falling by the wayside and appearing weak or disinterested in a job are intrinsic, real fears that can permeate a workplace. Underscoring that fear, the question of grappling with disclosing doctor appointments recently popped up on the popular blog Ask a Manager. Alison Green, who runs the blog, encouraged the employee to alert their manager to the situation but without revealing too many details. And though a firm like McCloskey’s urges employers to consider situations like these with empathy, one can only hope all managers exercise compassion.

It’s optimistic to imagine a world in which managers smilingly approve recurring doctor appointments, such as therapy—a typically recurring appointment that, ironically, encourages general well-being and, from a bottom-line-oriented manager’s perspective, can potentially make an employee perform better. The fixation on the bottom line is unfortunate but inevitable; Glass points out that there is often an issue with “anything that might suggest, either now or in the future, you’re not going to be able to work up to capacity.” Further, she says, employees are “encouraged to lie” about why they take time off: “They’re told, ‘These are the acceptable reasons for missing [work], and a mental health day is not one.’” Yet another reason for the hoopla earlier this summer.

Which suggests that while flex scheduling is an easy solution, what’s needed more than anything are compassion and empathy for employees, especially women, who, Glass says, are “always suspect workers” with “a shorter distance to fall.” Saying someone can leave for an hour or two takes a head nod of approval. Not letting a woman feel in jeopardy — and having empathy — takes real, emotional work.

“We should be okay with those types of emails [like in the viral story],” McCloskey says, “and the employer should have been able to say, ‘I should have been able to know that person needed a day off.’”