If we started seeing local governments having openly KKK or neo-Nazi majorities, the freedom of speech balance would substantially alter.
First amendment rights in the US are typically circumscribed only via a requirement to show actual, existential harm.
Death by a thousand cuts:
https://en.m.wikipedia.org/wiki/Creeping_normality
What about when we consider the opposite? How many insignificant-in-the-moment-seeming changes never effect any sort of larger change?
It's really off the mark to criticize the US for not having done enough against Nazis! Forget banning, we invaded and killed those fuckers.
https://www.reuters.com/article/us-global-race-portland-valo...
https://en.wikipedia.org/wiki/Gestapo#Suppression_of_resista...
But we're nowhere close to the Nazi party in the US.
Ideas versus actions and objects.