The problem with this concept is that it enables things like “whites only”, “no jews served here” and a plethora of other more blatant discrimination in more essential services than wedding cakes — like housing, groceries, automobiles, mortgages, banking, etc.

So the power of the state is needed to prevent the unequal access and balkanization of goods and services based on race, religion, etc. It’s an outgrowth of the abuses of segregation and discrimination against various minorities. The free market didn’t regulate itself, and large swathes of people were harmed. It used to be legal to discriminate as a “private business” in rental housing against racial minorities, same sex roomates, single mothers, unmarried women, non-Christians, and a whole raft of other things.

So while freedom of association is one thing, businesses open to the public need to be open to all. Private clubs are a different matter, and there is a very gray area when it comes to public accomodation vs private clubs that own and rent property. Fortunately, no one is arguing that you have to practice non-discrimination when it comes to dealing with roommates, because that is not a public accomodation, it’s a person’s private residence.

By and large, the government and the courts end up having to step in when the free market fails, often cruelly and spectacularly, to correct itself. It is astounding how much business owners will cling to their need to discriminate when you would think that the need to get all the customers possible would be the overriding factor. If the latter were true, then no government regulation would be needed, because business owners would not elimiate potential customers based on arbitrary things like race, religion, gender or sexual preferences. But humans are stubborn creatures, refusing to act in what would seem to be logical ways, instead acting on bias and bigorty.

It took me a long time to understand why the first anti-discrimination laws were needed — after all, wouldn’t a business want to expand its customer pool to the broadest possible audience? But historically, and even today (as evidenced by the anti-gay bakers), people in business will 
“cut off their nose to spite their face” in the name of prejudice rather than think first of the business.