OpenAI’s artificial intelligence system GPT- 3 can write code

"Jordan Singer is a product designer at Square, the Silicon Valley mobile-payments company. He helps design the company’s smartphone apps, building the graphics, menus, buttons and other widgets that define an app’s look and feel. When he heard about GPT-3, he wondered if this automated system could do his job.

He fed the system a simple description of a smartphone app, and the computer code needed to create the app. The description was in plain English. The code was built inside Figma, a specialized design tool used by professionals like Mr. Singer.

He did this a few more times, feeding the system several more English-language descriptions alongside the matching Figma code. And when he was done, GPT-3 could write such code on its own.

If he described a simple app for posting and viewing photos as a user would on Instagram, the system generated the code needed to build it. This code was sometimes flawed. But typically, if Mr. Singer made just a tweak or two, it worked as he wanted. “It’s not absolutely perfect,” he said. “But it is very, very close.”

This behavior was entirely new, and it surprised even the designers of GPT-3. They had not built GPT-3 to generate computer code, just as they had not built it to write like Mr. Kaufman or generate tweets or translate languages. They had built it to do just one thing: predict the next word in a sequence of words.

GPT-3 analyzed digital prose on an unprecedented scale, spending months looking for patterns in huge amounts of text posted to the internet. In this way, it learned to predict the next word in a sequence. If you type a few words into GPT-3, it will keep going, completing your thought with entire paragraphs of text.

But GPT-3 can do things that previous models could not, like write its own computer code. And, perhaps more important, you can prime it for specific tasks using just a few examples, as opposed to the thousands of examples and several hours of additional training required by its predecessors. Researchers call this “few-shot learning,” and they believe GPT-3 is the first real example of what could be a powerful phenomenon.

“It exhibits a capability that no one thought possible,” said Ilya Sutskever, OpenAI’s chief scientist and a key figure in the rise of artificial intelligence technologies over the past decade. “Any layperson can take this model and provide these examples in about five minutes and get useful behavior out of it.”

But continuing to improve this technology is far from trivial. Processing all of that internet data requires a specialized supercomputer running for months on end, an undertaking that is enormously expensive. When asked if such a project ran into the millions of dollars, Sam Altman, OpenAI’s chief executive, said the costs were actually “higher,” running into the tens of millions."