Hallucinations are the foremost risk of using AI in legal work. An LLM does not "understand" the law: it generates sequences of words that are statistically plausible based on its training. When the training data is insufficient or ambiguous on a topic, the model can produce entirely fictitious information with misleading confidence — inventing case law references, citing statutory provisions that do not exist, or attributing imaginary scholarly positions.
The Mata v. Avianca case (2023) is a stark illustration of this risk: U.S. lawyers were fined 5,000 dollars for submitting entirely fictitious case citations generated by ChatGPT to the court, without any verification. In France, the CNB (the national bar council) has reaffirmed that the lawyer remains fully responsible for any content produced with the help of AI.
Several strategies are available to mitigate hallucinations: a RAG architecture that grounds responses in verified sources, a low temperature setting (0.1-0.3) to favour accuracy, and above all systematic human review. No AI tool relieves the legal professional of their duty of oversight and their obligation of competence.