The Supreme Court of India set aside a National Company Law Tribunal judgment regarding Essel Infraprojects insolvency this week [1].
The ruling establishes a critical boundary for the use of generative technology in the Indian legal system. By striking down a decision based on fabricated precedents, the court said that the automation of legal research cannot replace human verification in judicial proceedings [2].
The case centered on a judgment from the National Company Law Tribunal (NCLT) that had been used to determine the insolvency process for Essel Infraprojects [1]. Upon review, the Supreme Court found that the NCLT judgment relied on precedents generated by artificial intelligence that were entirely fake [3].
Judges expressed concern over the nature of these hallucinations, describing the impact of such errors as catastrophic [1]. The court said that AI-generated fake judgments have no place in the legal process, a failure that could lead to systemic injustice if left unchecked [2].
The court said that human control over adjudication must remain total and absolute [3]. While technology may assist in organizing information, the responsibility for verifying the authenticity of case law remains with the legal professionals and judges [2].
This intervention comes as courts globally grapple with "hallucinations," where AI models invent plausible-sounding but nonexistent legal citations [3]. The Supreme Court of India said that the invisible and insidious nature of these errors makes them particularly dangerous for the rule of law [2].
“AI-generated fake judgments have no place in courts”
This ruling serves as a judicial warning to lawyers and lower courts in India that relying on AI-generated research without rigorous manual verification may lead to the immediate reversal of their decisions. It highlights a growing tension between the efficiency of legal tech and the necessity of factual accuracy in a system where a single fabricated precedent can invalidate an entire insolvency proceeding.
