IMO this is a solvable problem though. Eventually LLMs will have more awareness of their own confidence and will be able to convincingly say “huh, I’m honestly not sure about that, can you explain a bit more about what you mean?” Or even “I’ve heard of X before but not in this context; can you please clarify what you mean here?”