Yeah for sure. But also Orwell was extrapolating from the reality on the ground in communist countries. The scale achieved there was also massive. Your neighbors would turn you in if you criticized the people in power, etc.
This is also the reason the Soviets were ahead of the US in trying to train people like dogs (Pavlov's research) and why the communist forces ran brainwashing experiments on American POWs during the Korean War. If you look at what countries were willing to do back then it should make you concerned about what's coming.
For personalized targeting with psychological warfare, perhaps the closest analog is socialist East Germany's Zersetung https://en.wikipedia.org/wiki/Zersetzung
The main difference you get with LLMs is that it's cheaper to achieve these same goals Orwell was concerned about. You no longer have to make explicit and credible threats of violence. It's also easier to reach people in democracies and convince them that democracy is bad etc.
So for the sorts of concerns Orwell had, I think we're already seeing that. And Deepseek is one weapon in that war, since it has to comply with the Chinese regulations that LLMs must spread socialist core values.
How would you resist? I think a necessary precondition is that people continue to champion the importance of democracy and freedom of thought.
I don't know anything about Grok, but AFAIK they have different problems. Grok lacks alignment to push back against users asking for bad things, like advocating for hurting people.
But Deepseek is trained to manipulate the user into wanting bad things like hurting people.
This is also the reason the Soviets were ahead of the US in trying to train people like dogs (Pavlov's research) and why the communist forces ran brainwashing experiments on American POWs during the Korean War. If you look at what countries were willing to do back then it should make you concerned about what's coming.
For personalized targeting with psychological warfare, perhaps the closest analog is socialist East Germany's Zersetung https://en.wikipedia.org/wiki/Zersetzung
The main difference you get with LLMs is that it's cheaper to achieve these same goals Orwell was concerned about. You no longer have to make explicit and credible threats of violence. It's also easier to reach people in democracies and convince them that democracy is bad etc.
So for the sorts of concerns Orwell had, I think we're already seeing that. And Deepseek is one weapon in that war, since it has to comply with the Chinese regulations that LLMs must spread socialist core values.
How would you resist? I think a necessary precondition is that people continue to champion the importance of democracy and freedom of thought.