But given enough time, they will figure it out on their own. LLM's cannot do that ever.

Once they can ... I am open to revisit my assumptions about AGI.

They may require training but that training is going to look vastly different. We can start chatting about AGI when AI can be trained with as few examples and as little information as humans are, when they can replace human workers 1:1 (in everything we do) and when they can self-improve over time just like humans can.