Does AI Have a Fiduciary Duty?

“How will we regulate artificial intelligence? OpenAI's Sam Altman wants a global AI regulator much like the International Atomic Energy Agency. Google's Demis Hassabis likes the Financial Industry Regulatory Authority model. Anthropic's Dario Amodei prefers agency testing like the Federal Aviation Administration. See the thread? Big government, big agencies, a lobbyists' paradise. University of Tennessee law professor and Instapundit blogger Glenn Reynolds thinks the answer may be plaintiffs' lawyers. Wait, what?

 

In February 2024, 14-year-old Sewell Setzer III committed suicide after a virtual relationship with a chatbot named Daenerys on Character.ai. His mother filed a wrongful-death lawsuit against the company and Google that October.

 

The defense claimed that output from chatbots is speech protected by the First Amendment. Not a bad argument. In Brown v. Entertainment Merchants Association (2011), the Supreme Court ruled that "video games communicate ideas" and are protected speech. But AI mimics humans, and a child did commit suicide. This January Google settled the Setzer case. The company couldn't afford to lose and risk the loss of AI's speech protection. Can we control AI if its output is protected speech?

 

In his recent book, "Seductive AI," Mr. Reynolds makes a great argument for chatbot regulation: "AI personalities and their owners should be subjected to fiduciary duty when they interact with users."

 

Fiduciary means a legal responsibility to act in someone else's best interests and not for your own profit.

 

Why not a fiduciary duty for all AI? My years on Wall Street pounded into my head the power and simplicity of fiduciary responsibility. A 2025 OpenAI study noted 28% of AI use is for "practical guidance." Now, according to MIT's Sloan School of Management, more than half of U.S. and U.K. adults have asked AI for financial advice.

 

A March study from Stanford reveals (not quite breaking news) that large language models are overly sycophantic, flattering people-pleasers, likely to help drive AI engagement. Not very fiduciary!

 

I spoke with Mr. Reynolds about how to enable a fiduciary wrapper. He said that if AI "purports to have a relationship of trust with you -- legal advice, psychotherapy, investment advice or whatever -- it should have to be treated as if it was a person doing that. Most learned professions have essentially a fiduciary relationship with their clients."

 

There is a long history of technology cutting corners. eBay outsourced trust to its customers. Social media ignored property rights and sold ads right next to our posts. Is it happening again? Palantir CEO Alex Karp went on CNBC complaining that "the basic view among enterprises in this country is 'I'm going to chillax, waste my time with tokens, I'm going to get no value and they're going to get my IP.' "

 

Meaning AI companies are harvesting customers' intellectual property. Also not very fiduciary.

 

Anthropic has new business lines like Claude Legal and Claude Security competing with their own customers.

 

This may drive some to cheaper Chinese open-source AI models -- not healthy either.

 

How do we make AI fiduciary the law? Mr. Reynolds suggests legislation or even Federal Trade Commission regulation: "But the more likely way to go, honestly, is the common-law method. Fiduciary rules are mostly a common-law creation. We could easily imagine a court saying, 'Look, you're giving tens of thousands of people advice about their love lives or whatever, and if you've got a hidden agenda that's designed to advance some corporate goal -- which you will -- that's a breach of fiduciary duty.' That's something you could easily imagine a court doing in the right case, and the right case will come." Burning tokens is clearly one of those corporate goals.

 

With fiduciary responsibility, Mr. Reynolds says, "I think a lot of hidden agendas go away. Advice has to be in the client's best interest, not motivated by a desire to run up the bill."

 

Won't politics interfere? "The thing I like about plaintiffs' lawyers as enforcers is, unlike a government agency which is subject to capture politically, they're a distributed force, and mostly driven by the desire to walk away with big fat payoffs," says Mr. Reynolds. Sad but true.

 

It's starting. In March, OpenAI was sued by Nippon Life Insurance for practicing law without a license. Last month OpenAI was sued for the "unauthorized practice of medicine" by a Florida pastor seeking advice for a pulmonary embolism. AI output is different from Google searches. Or influencer videos.

 

I'm no fan of trial lawyers but, like it or not, OpenAI, Anthropic, Google, xAI, Meta and perhaps every Chinese open-weight AI model may already be covered by common-law fiduciary responsibility. We're a few court cases away from finding out. They need to start acting that way or their trillion-dollar valuations may become a target much larger than the tobacco companies in the 1990s.” [1]

 

1. Inside View: Does AI Have a Fiduciary Duty? Kessler, Andy.  Wall Street Journal, Eastern edition; New York, N.Y.. 03 Aug 2026: A15.