I'm not so certain that non-desk jobs will be safe either. What makes the current LLMs great at programming is the vast amount of training data. There might be some other breakthrough for typical jobs - some combination of reinforcement learning, training on videos of people doing things, LLMs and old-fashioned AI.