A Guide to the Many Flavors of AI Today --- Open AI, Google, Meta or Anthropic? Different versions work best for different business needs

"We're all being deluged with news about how the latest generation of AI is transforming people's lives, helping businesses be more productive, and even leading to layoffs. But that flood of information doesn't help anyone answer the most basic question about these AIs: Which is best?

So I canvassed executives, engineers and researchers who are knee-deep in the process of applying the world's most powerful AIs to real world problems, to find out what they have learned.

Their answers surprised me. There was plenty of practical advice about the relative strengths and weaknesses of AIs from Google, OpenAI, Anthropic and Meta. But the overall message was that the best AI for any task depends on both the user and the task. Their insights also offer a glimpse of where the entire field of AI is going.

Companies can now either embrace the potential cost savings and productivity boost of generative AI -- which some researchers believe is on the path to a "general" or humanlike AI -- or risk losing out to competitors who will.

Today's most powerful AIs are only accessible through the cloud. This makes it easy to test them by feeding them documents, images and text, but also means that businesses have limited ability to alter their behavior.

Testing these AIs is more like hiring an employee than just buying a piece of software off the shelf, says Mark Daley, chief AI officer of Western University in Ontario.

"People expect the chatbot to work right out of the box, but you have to spend time trying them and see which of these will deliver, just like you do with an employee," he adds.

Daley has found that all of the major large language models -- including those from OpenAI, Anthropic, Google and Cohere, a startup that only offers its models to businesses -- have their strengths and weaknesses. Which one to use depends on a person's preferences and the task at hand, and it pays to experiment with them.

Other companies appear to be catching up with the capabilities of OpenAI, but OpenAI's models remain, for now, the standard by which all others are judged. Earlier this week, Anthropic rolled out Claude 3, a new large language model which the company claims beats the gold standard GPT-4 on every benchmark.

"We are using OpenAI like crazy," says Brad Schneider, chief executive of Nomad Data, a company that helps large companies use AI. Nomad Data uses OpenAI to digest, summarize and search within huge libraries of documents, such as legal briefs, court cases and insurance claims.

After trying all of the most-capable large-language models, Schneider's company found that none are as good as OpenAI for these kinds of document-processing tasks. Previous versions of Anthropic's Claude and current versions of Google's Gemini both hallucinated too often, he found.

Google senior vice president Prabhakar Raghavan recently wrote that hallucination is a challenge common to all large language models, but that "this is something that we're constantly working on improving." Anthropic President Daniela Amodei has said that it is "very, very hard" to get the hallucination rate in such models to zero. The company has said that its latest model is twice as likely as its previous one to answer questions accurately.

In addition to accuracy, the other two big considerations are speed and cost, says Eric Olson, chief executive of Consensus, a scientific search engine.

On a search engine, users expect a response within seconds. Because Consensus pairs its search results with summaries of scientific papers made by GPT-4, the company needs those summaries to be generated nearly instantaneously.

For Olson's purposes, this means the only truly suitable model is OpenAI's GPT-4 "turbo," which can get a user a response within 1.5 seconds, versus twice as long with regular GPT-4. Google's Gemini and Anthropic's Claude are also slower than OpenAI's models, he adds.

That said, this kind of performance comes at a cost. OpenAI and its competitors charge business users of their systems by the token -- in essence, by the word -- to process their requests.

"We have cases where one question someone asks can cost $50," says Schneider. That could happen if, for example, someone asks a specific question about a collection of 5,000 legal documents, because the number of calls to OpenAI's systems could be in the tens of thousands.

Generative AI is a technology evolving at a rate not seen since the go-go days of the early internet itself. Anthropic's release of a model that seems every bit as capable as OpenAI's, despite having a smaller team and having been founded much more recently, suggests that large language models may become commodities. At that point, the only thing that will matter will be which company can offer the fastest response at the lowest price.

The beneficiary of that fierce competition will be companies large and small, which could see significant increases in the productivity of their employees. Those gains will come at a fraction of the cost of paying humans to do the same knowledge work. The implications for the future of white collar jobs are obvious -- and worrisome." [1]

1. EXCHANGE --- Keywords: A Guide to the Many Flavors of AI Today --- Open AI, Google, Meta or Anthropic? Different versions work best for different business needs. Mims, Christopher.  Wall Street Journal, Eastern edition; New York, N.Y.. 09 Mar 2024: B.5.