Responsibility Cannot Be Automated

archive

Around 1750 BC the Babylonian king Hammurabi had the laws of his kingdom carved on a basalt pillar, and the two hundred and twenty-ninth of them could hang in any company that launches agents today. If a builder builds a house and does not build it properly, and the house collapses and kills its owner, the builder shall be put to death. Nassim Taleb later called this law the best risk-management rule ever devised, and the point is not its cruelty. Hammurabi simply left the builder no way of saying that the house fell down by itself.

Almost four thousand years later, the airline Air Canada tried to say that.

---

In 2022 Jake Moffatt’s grandmother died, and he went to Air Canada’s website to buy a ticket to the funeral. The chatbot on the site explained to him that the company had a bereavement discount and that it could be claimed retroactively, within three months of the flight. Moffatt bought the ticket at full price, flew, came back and applied for the discount. The company replied that the discount was not granted retroactively, and this was indeed written on another page of the same website. Moffatt went to the tribunal, and there Air Canada advanced an argument that will go into the textbooks: the chatbot, in the company’s view, was a separate legal entity responsible for its own actions. In February 2024 the British Columbia tribunal replied that the company is responsible for everything on its website, whether it is written on an ordinary page or by a chatbot, and awarded Moffatt six hundred and fifty-odd Canadian dollars, the difference in price. The sum is laughable; the precedent is serious.

A year earlier in New York two lawyers learned a similar lesson. Preparing a lawsuit against the airline Avianca, one of them asked ChatGPT to find suitable court decisions, and the model found six, with case numbers, quotations and the names of judges. All six turned out to be invented. When the lawyers for the other side could not find them, the plaintiffs first stood their ground and then explained that they had not known ChatGPT could make things up. In June 2023 Judge Kevin Castel fined both lawyers and their firm five thousand dollars, and over the following year and a half dozens of American courts introduced rules requiring lawyers to disclose whether they had used AI.