Despite the liberatory potential of technology, of which I see free software playing a big role, there's a very real concern of ending up with a kind of technocratic 'vanguard party'.
You can debate the merits of vanguardism in general, but couple it with the current disproportionate skew of tech roles to white and male - which is even more pronounced in free software at present - and throw in the 'scratch your own itch' trope.
That's a huge systemic problem as vanguard becomes regime.