Classifying a “privilege” as something non-essential doesn’t render the word “privilege” meaningless in any way, shape, or form. The word still means something that you don’t necessarily need in order to live your life. “Privilege”, contrary to what you may believe, is not a precious, special snowflake of a concept. Something can be a privilege even if it doesn’t come in the nice, shiny package of wealth and excess, what we normally associate with “privilege” when we think of the word.
And the means to build a fuck off fund and the fuck off fund itself are inextricably linked. You can’t have the fuck off fund if you don’t even have the means to build it to begin with. Someone living on minimum wage is not going to be able to build up a savings, because all their money is going towards sustaining themselves. So yes, a fuck off fund is absolutely a privilege.