Why is artificial intelligence research physics?

Hopfield and Hinton, who laid the foundations for machine learning research, received the Nobel Prize in Physics. Why?

"— The laureates used physics tools to develop methods that became the basis of modern machine learning, — the Nobel Committee noted in a press release. — Hopfield created associative memory that can store and reconstruct images and other types of patterns in data. Hinton invented a method that helps autonomously find certain properties in data and perform tasks such as identifying certain elements in images.

 

As explained by the Nobel Committee, one of Hopfield's significant contributions is that he invented a network that uses a method for storing and recreating patterns. Thanks to it, network nodes can be represented as pixels. This network became the prototype of artificial intelligence algorithms that help restore input data based on its context.

 

— John Hopfield is known for creating the "Hopfield network." 

It is based on an elegant mathematical approach close to physics. It consists in the fact that when data is fed to the input of a neural network, it tries to go into a state with minimal energy. This is similar to associative memory. For example, if we teach a network an image and then feed it a distorted form, the program, striving for “rest”, will correct errors in the image.

 

At the same time, Geoffrey Hinton, in accordance with the statement of the Nobel Committee, used the Hopfeld network as the basis for a new network that helps to recognize characteristic elements in data of a certain type.

— In 1985, Hinton came up with the “Boltzmann machine” — a system that is capable of searching for an absolute, rather than a local minimum of energy. This means that the algorithm is more likely to find the correct solution, rather than one that “looks like the correct one”.

The software implementation of such a machine uses an algorithm for simulating annealing. This is a process that occurs when molten substances solidify. Thus, the idea of ​​​​implementing the computational process is associated with a physical effect, which was emphasized by the members of the Nobel Committee.

- Despite the fact that physics and machine learning seem to be different areas of science, they both rely on statistical approaches. When developing the Hopfield and Hinton neural networks, they used a basic physical principle - energy minimization. 

The fact that memorization is associated with the concept of energy is one of the main results for which the prize was given.

Modern neural networks operate on a famous learning principle — the so-called backpropagation of errors [1]."

1.  A-step-by-step-backpropagation-example