Can We Stop LLMs from Hallucinating?
Opinion
While Large Language Models (LLMs) have captured the attention of nearly everyone, wide-scale deployment of such technology is slightly limited due to a rather annoying aspect of it – these models tend to hallucinate. In simple terms, they sometimes just make things up, and worst of all, it often looks highly convincing.
Hallucinations, frequent or not, bring with them two major issues. They can't be directly implemented in many sensitive or brittle fields where a single mistake can be highly costly. In addition, it sows general distrust as users are expected to verify everything coming out of an LLM, which, at least in part, defeats the purpose of such technology.
Academia seems to also think that hallucinations are a major problem, as there are dozens of research papers in 2023 discussing and attempting to solve the issue. I, however, would tend to agree with Yann LeCun, Meta's Chief AI Scientist, that the hallucinations are not resolvable at all. We would need a complete revamp of the technology to eliminate the issue.
Hallucinating false statements
There are two important aspects to any LLM which, I think, make Hallucinations unsolvable. Starting with the rather obvious technological underpinning, LLMs, like any other machine learning model, are stochastic in nature. In simple terms, they make predictions.
While they're certainly much more advanced than "glorified autocomplete," the underlying technology still uses statistical predictions about tokens. It's both one of the strengths and weaknesses of LLMs.
On the strong part, we have seen how amazingly good they are at predicting what should come after an input (barring any intentional attempt to ruin an output). Users can make several types of mistakes, such as leaving in a typo, misunderstanding the meaning of a word, etc., and LLMs are still likely to get the output right.
Back in the day, when the first text-based games were created, users were asked to input commands without any errors or room for interpretation. A command such as "move north" would error out if the user inputted "move morth". An LLM, however, might be able to infer the meaning in both cases. In that sense, the technology is truly fascinating.
Yet, it also showcases a weakness. Any input has a wide potential decision tree for token choice. In simple terms, there's always a huge range of ways a model can create an output. Out of that large range, a relatively small bit is the "correct" decision.
While there are numerous optimization options available, the problem itself is not solvable. For example, if we increase the likelihood of providing one specific answer, the LLM becomes a lookup table, so we'd want to keep a balance. The underlying technology is simply based on stochastic predictions, and there has to be some room for a wider range of output tokens provided.
But there's another problem that LLMs cannot solve, at least in their current state. It's a bit more ephemeral and abstract as it relates to epistemology, the field of philosophy that studies the nature of knowledge. On the face of it, the problem is simple – how do we know which statements are true, and how do we gain such knowledge? After all, a hallucination is simply a set of false statements post-hoc, so if we could create a way for the model to verify that it has made a false statement and remove it, that would solve the problem.
Separating hallucinations from truthful statements
Following in the footsteps of philosophy, we can separate two types of possible statements – analytic and synthetic. The former are statements that are true by virtue of definition (one of the most common examples is "a bachelor is an unmarried man"). In simple terms, we can find statements that are true by analyzing the language itself, and no external experience is required.
Synthetic statements are any statements that are true by virtue of some form of experience, such as "there is an apple on the table in front of me." There's no way to know whether such a statement is true without referring to direct experience. Pure linguistic analysis does no good in determining whether it is true or false.
I should note that the distinction between these statements has been hotly contested for hundreds of years, but the discussion is largely irrelevant for LLMs. As their name might state, they're a highly advanced linguistic analysis and prediction machine.
Following the distinction between the two types, we can see that LLMs would have little to no issue with analytic statements (or at least as much as humans do). Yet, they have no access to experience or the world at large. There's no way for them to know that some statements are true by virtue of an event.
The major issue is that the number of analytic statements is significantly smaller than the set of all synthetic statements. Since an LLM has no way of verifying whether these statements are true, we, as humans, have to provide them with such information.
As such, LLMs run into a challenge. The set of all possible outputs will always have some number of synthetic statements, but to the model, all of them are truth-value agnostic. In simple terms, "Julius Caesar's assassin was Brutus" (there were many, but for this case, it doesn't matter) and "Julius Caesar's assassin was Abraham Lincoln" are equivalent to a model.
A counterargument might be that we have not had any direct experience about those events, either. We just read about them in books. But the discovery of the truthfulness of the statement is based on a reconstruction of surviving accounts and a wide range of other archaeological evidence.
A simpler example of an (albeit less relevant) statement would be "it is raining today." Such statements are impossible to determine as true for an LLM at all as it needs access to real-world experience at the moment of query.
In one sense, the epistemological problem is self-solving. Our literary corpus would make the output that "Julius Caesar's assassin was Brutus" significantly more likely due to it being present more frequently. Yet, again, the problem is that such a self-solving solution relies on training an LLM on absolutely all available textual information, which, obviously, is impossible. Additionally, that would make other, less truthful outputs not entirely absent from the set of all possible outputs.
As such, data quality becomes an important factor, but that quality can only be judged by human observers. Even in cases where models are trained on enormous amounts of data, there's a certain selection process that takes place, which means that the error rate for synthetic statements cannot be eliminated.
Conclusion
I believe that the problem of stopping models from hallucinating is unsolvable. For one, the technology itself is based on a stochastic process, which inevitably, over a large number of outputs, will lead to erroneous predictions.
In addition to the technological hurdle, there's the question of whether LLMs can make truth-value judgments about statements, which, again, I believe is impossible as they have no access to the real world. The issue is slightly attenuated by various search engine functions that are now available for many LLMs, according to which they may verify certain statements.
It might be possible, however, to collect a database against which statements can be tested, but that would require something beyond the technology itself, which leads us back to the initial problem.
