AI Hallucinations
AI hallucinations refer to the generation of false, fabricated or inconsistent information presented with apparent certainty by a language model. The phenomenon is inherent to how LLMs work: they produce statistically plausible text rather than factually verified content. An LLM can invent non-existent case law references or statutory provisions.
Hallucinations are the foremost risk of using AI in legal work. An LLM does not "understand" the law: it generates sequences of words that are statistically plausible based on its training. When the training data is insufficient or ambiguous on a topic, the model can produce entirely fictitious information with misleading confidence — inventing case law references, citing statutory provisions that do not exist, or attributing imaginary scholarly positions.
The Mata v. Avianca case (2023) is a stark illustration of this risk: U.S. lawyers were fined 5,000 dollars for submitting entirely fictitious case citations generated by ChatGPT to the court, without any verification. In France, the CNB (the national bar council) has reaffirmed that the lawyer remains fully responsible for any content produced with the help of AI.
Several strategies are available to mitigate hallucinations: a RAG architecture that grounds responses in verified sources, a low temperature setting (0.1-0.3) to favour accuracy, and above all systematic human review. No AI tool relieves the legal professional of their duty of oversight and their obligation of competence.
Related terms
An LLM (Large Language Model) is an artificial intelligence model trained on massive volumes of text to learn the statistical stru…
RAG (Retrieval Augmented Generation) is a technique that combines document retrieval with an LLM. Before generating an answer, the…
Temperature is a hyperparameter that controls how much randomness goes into an LLM's responses. A low value (close to 0) produces …
AI ethics in law refers to the set of professional principles governing the use of AI by legal professionals: algorithmic transpar…
Prompt engineering is the art of crafting precise, structured instructions to get optimal results from a generative AI. It is not …