@Angle The classic argument is that popular freedom depends on the virtue of the public — a society lacking in virtue will inevitably be dominated by internal or external parties, and indeed may require domination so as not to fall into anarchy.
Depending on how you define freedom (opportunity? absence of control by the state?) an unequal society can have a high degree of freedom. Consider the US vs. Germany.