One of the penalties for refusing to participate in politics, is that you end up being governed by your inferiors.
From the following:
The good are not willing to rule either for the sake of money or of honor. They do not wish to collect pay openly for their service of rule and be styled hirelings nor to take it by stealth from their office and be called thieves, nor yet for the sake of honor, [347c] for they are not covetous of honor. So there must be imposed some compulsion and penalty to constrain them to rule if they are to consent to hold office. That is perhaps why to seek office oneself and not await compulsion is thought disgraceful. But the chief penalty is to be governed by someone worse if a man will not himself hold office and rule. It is from fear of this, as it appears to me, that the better sort hold office when they do, and then they go to it not in the expectation of enjoyment nor as to a good thing, but as to a necessary evil and because they are unable to turn it over to better men than themselves [347d] or to their like. For we may venture to say that, if there should be a city of good men only, immunity from office-holding would be as eagerly contended for as office is now.
from Plato’s The Republic (347) translation at https://en.wikiquote.org/wiki/Talk:Plato