Artificial intelligence has quietly entered almost every law firm. It drafts first versions, summarises documents, researches points of law and increasingly powers marketing. Used well, it saves hours. Used carelessly, it can end in professional embarrassment- or worse.

If any doubt remained about the stakes, the Supreme Court of India removed it.

A judicial wake-up call

In July 2026, the Supreme Court set aside orders of the National Company Law Tribunal and its appellate tribunal after finding that both had relied on fake, AI-generated citations while deciding an insolvency dispute in Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd. The precedents cited did not exist; they had been fabricated -“hallucinated” - by an AI tool and passed on without verification.

The Court’s language was unambiguous. Reliance on hallucinated material, it held, strikes at the integrity of adjudication, and courts must adopt a “zero-tolerance” approach. Crucially for practitioners, the Bench observed that citing such judgments without verification is misconduct on the part of an advocate. The Court also directed the Bar Council of India to constitute a committee to frame guiding principles on AI use.

The message to the profession is clear: the tool may be new, but the duty of care is not.

Why “hallucination” is a lawyer’s problem

Generative AI does not look up the law. It predicts plausible-sounding text. That is why it can produce a confident, well-formatted citation- complete with party names, a court and a year- for a case that was never decided. To a busy lawyer skimming a draft, a fabricated authority can look indistinguishable from a real one.

The danger is compounded by how AI output travels. An unverified citation can move from a research note into a submission, from a submission into a judgment, and from there into the public record- as the insolvency matter above demonstrates. The only reliable defence is verification at the source.

The regulatory direction of travel

Regulators are responding. The Bar Council of India’s July 2026 circular on social media and advocate conduct addresses AI directly: it prohibits deepfakes of judges, lawyers and litigants, bars the circulation of fabricated judgments and citations, and requires disclosure where AI has materially generated content. Comparable expectations- accuracy, candour and technological competence- flow from professional-conduct duties in other jurisdictions.

The common principle is that a lawyer remains fully responsible for anything produced in their name, whether a human or a machine drafted it.

Practical guardrails for your firm

Responsible AI use is not about avoiding the technology. It is about building a few disciplines into how the firm works.

1. Treat AI output as a draft, never as authority. Every citation, statute reference and factual claim must be checked against a primary source before it leaves the office. If it cannot be verified, it does not go in.
2. Keep confidential information out of public tools. Client details, case documents and privileged material should not be pasted into consumer AI services whose data handling you do not control.
3. Disclose material AI use where required. Where AI has materially generated content — and certainly where a regulator requires it- say so. Transparency protects both you and the reader.
4. Put a human name on every output. A named lawyer should review and take responsibility for anything the firm produces or files. Accountability cannot be delegated to software.
5. Apply the same discipline to marketing. An AI-written article that misstates the law or invents a statistic carries the same risk to your credibility as a fake citation — and, for regulated professionals, the same duty of accuracy applies. Everything published in the firm’s name should be verified.
6. Train your team and write it down. Juniors, interns and anyone posting on the firm’s behalf should know the rules. A short, clear AI policy- verify, protect confidentiality, disclose, review- turns good intentions into consistent practice.

The balance to strike

None of this argues against using AI. Within these guardrails, it is a genuine force multiplier: faster research, cleaner drafts, and more time for the judgment only a lawyer can provide.

The firms that will thrive are not the ones that avoid AI, nor the ones that trust it blindly. They are the ones that treat it as what it is- a powerful assistant and a dangerous author. Let it help you work; never let it think for you. As the Supreme Court reminded the profession, responsibility for what carries your name remains, always, yours.