Hopfield and Hinton, who laid the foundations for machine learning research, received the Nobel Prize in Physics. Why?
"— The laureates used physics tools to develop methods that
became the basis of modern machine learning, — the Nobel Committee noted in a
press release. — Hopfield created associative memory that can store and
reconstruct images and other types of patterns in data. Hinton invented a
method that helps autonomously find certain properties in data and perform
tasks such as identifying certain elements in images.
As explained by the Nobel Committee, one of Hopfield's
significant contributions is that he invented a network that uses a method for
storing and recreating patterns. Thanks to it, network nodes can be represented
as pixels. This network became the prototype of artificial intelligence
algorithms that help restore input data based on its context.
— John Hopfield is known for creating the "Hopfield
network."
It is based on an elegant mathematical approach close to
physics. It consists in the fact that when data is fed to the input of a neural
network, it tries to go into a state with minimal energy. This is similar to
associative memory. For example, if we teach a network an image and then feed
it a distorted form, the program, striving for “rest”, will correct errors in
the image.
At the same time, Geoffrey Hinton, in accordance with the
statement of the Nobel Committee, used the Hopfeld network as the basis for a
new network that helps to recognize characteristic elements in data of a
certain type.
— In 1985, Hinton came up with the “Boltzmann machine” — a
system that is capable of searching for an absolute, rather than a local
minimum of energy. This means that the algorithm is more likely to find the
correct solution, rather than one that “looks like the correct one”.
The software implementation of such a
machine uses an algorithm for simulating annealing. This is a process that
occurs when molten substances solidify. Thus, the idea of implementing the
computational process is associated with a physical effect, which was
emphasized by the members of the Nobel Committee.
- Despite the fact that physics and machine learning seem to
be different areas of science, they both rely on statistical approaches. When
developing the Hopfield and Hinton neural networks, they used a basic physical
principle - energy minimization.
The fact that memorization is associated with
the concept of energy is one of the main results for which the prize was given.
Modern neural networks operate on a famous learning
principle — the so-called backpropagation of errors [1]."
1. A-step-by-step-backpropagation-example
Komentarų nėra:
Rašyti komentarą