AI can give confidently wrong results. Understanding hallucinations help you avoid embarrassing or costly mistakes when using AI.
Hallucination refers to the phenomenon that the LLM confidently produces information that are factually incorrect or fabricated. E.g. making up citations that do not exist. This is especially problematic in high stake work such as legal and medical. E.g. law firm messed up not checking LLM generated data.
The name “hallucination” may sounds like the LLM would all of a sudden generate gibberish. This is not the case. In general, hallucination generally occurs as plausible but false statements (which makes them harder to detect). Examples
- “Company X launched this product in 2022” when it did not
- A paper, law case, or article that does not exist
- The source exists, but does not actually say what the AI claims
- Invents a person’s quote, experience, etc.
One may ask: if thats the case, how are LLMs useful at all if it can make up stuff? The answer is advances in LLM has significantly reduced hallucination. Today, in frontier models, hallucination can still happen, but it is the rare case.
(Double extension incoming)