I agree that LLMs have opened modalities we didn't have before, namely:
- natural language input
- ability to synthesize information across multiple sources
- conversational interface for iterative interaction
That feels magical and similar to Star Trek.
However they fundamentally require trustworthy search to ground their knowledge in, in order to suppress hallucination and provide accurate access to real time information. I never saw someone having to double-check computer's response in Star Trek. It is a fundamental requirement of such interface. So currently we need both model and search to be great, and finding great search is increasingly hard (I know as we are trying to build one).
(fwiw, the 'actual' Star Trek computer one day might emerge through a different tech path than LLMs + search, but that's a different topic. but for now any attempt of an end-to-end system with hat ambition will have search as its weakest link)
What solution is there besides choosing the sources you will ground your truth to? We are not going to transcend intermediaries when asking for answers from an intermediary.
I'm not sure how flippant you are being, but this is the answer. A wikipedia / wikidata for everything, with some metadata about how much "scientific consensus" there is on each data point, and perhaps links to competing theory if something is not well established.
In the past year, I have seen Wikipedia go from a decent source of information to complete fantasy on a specific topic. Obviously biased mods have completely pushed the particular subject narratives.
I'm not being flippant, actually. I would pay to have a reliable source of information. I'm also overwhelmed at the thought of how to make such a thing work.
- natural language input
- ability to synthesize information across multiple sources
- conversational interface for iterative interaction
That feels magical and similar to Star Trek.
However they fundamentally require trustworthy search to ground their knowledge in, in order to suppress hallucination and provide accurate access to real time information. I never saw someone having to double-check computer's response in Star Trek. It is a fundamental requirement of such interface. So currently we need both model and search to be great, and finding great search is increasingly hard (I know as we are trying to build one).
(fwiw, the 'actual' Star Trek computer one day might emerge through a different tech path than LLMs + search, but that's a different topic. but for now any attempt of an end-to-end system with hat ambition will have search as its weakest link)