"Scientists at the University of Oxford have developed a method to predict and prevent hallucinations in AI models behind ChatGPT and Gemini.
In the future, the models will generate and compare multiple answers before delivering a result.
Hallucinations are a well-known problem with large language models (LLMs), where the models generate plausible-sounding but incorrect or completely made-up information. Scientists at the University of Oxford have developed a method to predict and prevent hallucinations in large language models. The research was published in the journal "Nature".
Just in May, Google's AI search function "AI Overview" was criticized for giving users absurd and health-endangering answers. This included the recommendation to use glue as a topping for a pizza or to eat a stone every day. "LLMs can formulate the same thing in many different ways. With old approaches, you couldn't distinguish whether a model was unsure what to say or whether it was unsure how to express itself," says Sebastian Farquhar, one of the study's co-authors.
The new method focuses on measuring the uncertainty in the meaning of the generated answers, rather than just looking at the uncertainty in the choice of words.
Comparing different answers to the same question
The idea is this: If the language model generates several different answers for a question that are worded differently but have the same meaning, then the model is very sure about the answer.
If a model gives many answers with different meanings to the same query, then the model is unsure. This is a warning sign that the model may be giving something wrong or unfounded.
The study proposes estimating probabilities in the meaning space and only then giving a final answer. This method makes it possible to quantify the uncertainty of an answer and thus identify hallucinations before they occur. This allows users to be warned when the model's answer may be incorrect or unfounded. During the training of a language model, answers with high meaning deviations can be rejected to improve the accuracy of the model.
However, the implementation of this method is complex, because the language model has to generate several possible answers for each question in advance in order to find a final result. The study suggests that comparing ten pre-generated answers could improve the performance of a model. This is computationally intensive, especially for longer answers. "In situations where reliability is crucial, calculating semantic uncertainty is a small price to pay," says the head of the study, Yarin Gal." [1]
1. Neue Methode soll Halluzinationen der Sprachmodelle vermeiden. Frankfurter Allgemeine Zeitung (online) Frankfurter Allgemeine Zeitung GmbH. Jun 20, 2024. Von Nina Müller
Komentarų nėra:
Rašyti komentarą