Robotics Researchers Focus on Teamwork: Teaching robots to communicate and respond to each other could lead to the automation of more-complex work

"At a typical Amazon fulfillment center, thousands of robots work alongside people, doing a variety of jobs. A robot arm helps pick ordered items from shelves and loads them onto a mobile robot. The mobile robot rolls through the warehouse, delivering items to human employees who organize the items into orders. After the orders are packaged, yet another robot arm puts them on a robotic carrier to be shipped out for delivery.

This complicated choreography works because of sophisticated software, developed by Amazon, that tells the human workers and robots where to go and what to do to ensure they are collaborating efficiently. The robotic systems also communicate with each other on the ground, preventing backups and bottlenecks.

"The majority of the software is like a traffic controller," says Tye Brady, chief technologist for Amazon Robotics. "The movement of these robots is very coordinated and thought through at a system level."

It isn't unusual to find robots doing jobs in factories, warehouses and elsewhere these days. But if the goal is to automate more-complex tasks, and do so efficiently, researchers say groups of autonomous robots will need to be able to communicate, cooperate and respond to each other and their environment in ways many of them can't today.

Scientists have been studying how to get multiple robots to work together for decades. While most of the technology is still in the research stage, multirobot systems like the ones at Amazon fulfillment centers are starting to be deployed in the real world. 

And if the research continues to advance, researchers say these systems could be deployed in a variety of new settings -- from search-and-rescue missions where robots on the ground coordinate with drones above, to autonomous-driving situations, to space exploration.

There are different types of multirobot systems with varying levels of autonomy.

Swarm robots, for example, are designed to move together without colliding, but have limited or no ability to divvy up jobs or tasks. Researchers say swarm robotics holds promise for things like surveillance or search and rescue, where getting multiple robots to film the same area might be helpful. In other, more advanced systems, robots can move independently of each other and split up tasks -- say, prepping orders for delivery, as in the case of Amazon -- with the help of a centralized leader, either a human or software. In still other setups being studied but that remain further off, humans would assign robots an objective and the robots, using AI, would decide among themselves the best way to achieve it.

As robot systems are given more agency, the complications increase, scientists say. One big challenge is communication -- not only how to do it, but what information is necessary to share out of potentially millions of data points being created at any moment.

Each robot in a multirobot system must be aware of the other's actions and goals to avoid collisions and optimize performance. This requires developing effective communication and coordination algorithms, according to Michael Wooldridge, a professor of computer science at the University of Oxford.

"A huge amount of research has gone to try to understand the protocols that robots might use for information sharing," says Prof. Wooldridge. "Who needs to know what, when and if they have to dynamically organize themselves into teams, and who gets to be the boss and organize the others or what roles do they play? These are all extremely difficult challenges."

Expense is another challenge. The investment associated with multirobot systems likely will limit their use to companies that can afford the upfront costs, maintenance and replacement if one or more of the robots in the collective is damaged. On the other hand, multirobot systems could improve efficiency, because if one robot in the group malfunctions, others with similar capabilities could step in and finish the job.

"There could be the option of heterogeneous teams instead of relying on one single very expensive robot that can do everything," says Alyssa Pierson, an assistant professor of mechanical engineering at Boston University. "Can we divide up all the sensors on that robot into multiple cheaper robots that maybe the risk of failure is a little bit higher, but the cost of failure is lower?"

According to Prof. Wooldridge, some of the most significant leaps for multirobot systems have come from RoboCup, an annual competition started by researchers in 1997 in which teams of robots compete in soccer.

RoboCup's stated goal is to develop a team of fully autonomous robot soccer players capable of beating a human team by 2050. Peter Stone, a computer-science professor at the University of Texas at Austin who recently served as the group's president, says the goal is analogous to IBM's DeepBlue beating human champion Garry Kasparov at chess and AlphaGo beating Lee Sedol at Go, an ancient board game. "We want to do that for soccer," Prof. Stone says. "To do it, you have to solve some really important technical challenges that are going to have wide-ranging implications."

One RoboCup league features autonomous robots with humanlike bodies and humanlike senses. Early on, the robots struggled to move quickly, stop suddenly or change direction without falling over. The league used an orange ball to make it easier to see, but even so, it was difficult for the robots' cameras and algorithms to detect it reliably from a distance. These days, the league uses a small black-and-white ball, and the robots have no problem seeing it. They can move with ease while scoring multiple goals, all while maintaining an understanding of where the ball and other players are on the field.

Researchers say one focus of the past decade has been on ad hoc teamwork, or giving autonomous agents the ability to cooperate with team members they have never played with before -- in much the same way human strangers might do after gathering for a pickup soccer game.

"People are really good at quickly assessing what are the strengths and weaknesses of their teammates," Prof. Stone says. " As we get toward more long-term autonomy agents that are turned on and operational for long periods, they're going to come across people or other agents that they weren't programmed to cooperate or interact with, and they need to figure out how to do it anyway."

Some of those implications could be essential in disaster rescue scenarios, according to Prof. Stone. Eventually, for example, robots capable of ad hoc teamwork could converge on the site of a building collapse, and seamlessly start collaborating on the search effort without having to be networked together.

There are many other areas where advances in multirobot systems are expected to be critical to success.

For autonomous driving to take off, for example, self-driving cars from different companies need to be able to communicate and respond to each other, as well as the environment around them, including things like road hazards and traffic signals.

There are some difficult challenges with autonomous cars that make them unique among multirobot systems, according to Mac Schwager, associate professor of aeronautics and astronautics at Stanford University. Most important is the risk to human life if an error occurs. On the other hand, information sharing among networked cars about things like road hazards or accidents could lead to improved highway safety, he says.

"The vehicle network can alert the neighborhood of other vehicles about impending dangers or things that they see that may not be evident to the other vehicles," Prof. Schwager says.

Such a vehicle-to-vehicle network doesn't yet exist, researchers say, and can't be built without better internet coverage from 5G networks.

Space exploration is another area where multirobot systems will play a critical role, says Danette Allen, NASA's senior leader for autonomous systems.

One major need will be in the Artemis program, which plans to re-establish a human presence on the moon and expects to rely heavily on multirobot systems for everything from landing to building structures on the lunar surface. These robots will be asked to do things that aren't safe for humans and have to be resilient, Dr. Allen says. And even with extensive research and planning, robots will be thrust into situations with each other they weren't prepared for and have to react.

"They can't throw up their metaphorical arms and give up," Dr. Allen says. "We need them to think and behave more like humans when they experience a novel situation and come up with a creative solution."" [1]

1. C-Suite Strategies (A Special Report) --- Robotics Researchers Focus on Teamwork: Teaching robots to communicate and respond to each other could lead to the automation of more-complex work
Snow, Jackie.  Wall Street Journal, Eastern edition; New York, N.Y. [New York, N.Y]. 17 Apr 2023: R.10.