"Elon Musk is betting that robot cars will propel Tesla into a lucrative new era. But he's going about it in a risky fashion.
Musk's plans center on what he has called end-to-end artificial intelligence. The plan is to deluge Tesla's AI systems with video footage from existing Teslas, in the expectation that algorithms running on huge supercomputers will learn how to drive safely. He hopes this will make it possible for Tesla to deliver fully self-driving cars faster and more cheaply than his competitors. Existing Tesla owners would get access next year, and new specially designed robotaxis would be ready in 2026.
The breakthrough AI of Musk's dreams contrasts starkly with the approach of other companies pursuing autonomous vehicles. Waymo is the industry leader, already operates commercial robotaxis and just announced a $5.6 billion round of financing. Waymo, which is owned by Google-parent company Alphabet, also uses lots of AI, but its approach is to break down the problem of self driving into more distinct tasks with more input from human engineers. Waymo is using data from more sensors, including lasers and radar, which gives the company's cars a much richer view of the world.
In the simplest possible terms, Musk's vision for Tesla is about an AI system that learns by watching people drive.
Waymo and others are teaching their vehicles by correcting them as they do the driving themselves.
Musk's bet hinges on the current state of AI technology reaching a level of sophistication that it hasn't yet achieved and may not for some time, AI developers say.
One of Musk's defining characteristics has been an ability to start with a goal in mind, and work backward to the solution required.
With self-driving tech, his goal is a system affordable enough to put on most any vehicle.
Musk has said robots and self-driving cars could propel Tesla's market value to at least $30 trillion. Supporters of his plans point to successes his companies have achieved by drawing outside the lines, like the radical reduction in launch costs achieved at SpaceX, which now dominates the rocket business.
Musk has a long history of overpromising and being vague about how exactly his ideas become reality. He uses names for Tesla's driver-assistance technology -- "Autopilot" and "Full Self-Driving" -- that imply more capability than it has, and Tesla has missed all prior targets for the release of fully autonomous driving systems.
At a Tesla robotaxi event this month, unveiling an under-$30,000 Cybercab model with no steering wheel or pedals, Musk quipped: "I tend to be optimistic with time frames." He predicted production would start "before 2027." By Tesla's earnings call just days later, he was saying confidently that large-scale production would happen in 2026.
Musk and others who have worked on Tesla's self-driving tech have said its advantage is vast amounts of footage of real-world driving captured by cameras built into all of its vehicles -- including all the time people have spent using the company's existing "Full Self Driving (Supervised)" driver-assistance software, usually shortened to FSD.
Training Tesla's AI using this passively recorded data requires a technique known as imitation learning.
In essence, to gain any advantage from all this data, Tesla's AI must watch those millions of hours of humans driving, and try to copy their actions, says Timothy B. Lee, a computer scientist who writes the newsletter Understanding AI.
"It's like living millions of lives simultaneously and seeing very unusual situations that a person in their entire lifetime would not see," Musk said at the robotaxi event.
Tesla's backers express confidence that its self-driving work is better and more extensive than may be publicly apparent.
Tasha Keeney, director of investment analysis at ARK, which has long been bullish on Tesla, says Musk's company may be keeping many of the technical details about its systems secret, but that behind the scenes it is continuing to innovate its AI techniques. She recently co-wrote an analysis of Tesla's robotaxi strategy that argues that by 2029, robotaxis will account for almost 90% of the company's enterprise value, and 60% of its revenue. By then, she says, Tesla will be worth about 10 times its current value, which is around $800 billion.
In contrast, Tesla's competitors have trained their self-driving systems in the real world by putting a safety driver behind the wheel of a car, who takes control when the vehicle does something undesirable. These companies meticulously track those "disengagements" and feed the data back to their engineering teams who tweak the system so the mistake doesn't happen again.
That approach is more labor-intensive, time consuming and expensive.
But Waymo and others feed that data into more powerful and ultimately more reliable systems through a process known as reinforcement learning, says Lee.
Research into these two approaches has shown that the results can be wildly different. Systems trained primarily with imitation learning, such as Tesla's, can fail when their own actions take them too far outside of the realm of the data they've been trained on. In addition, Tesla's devotion to a fully end-to-end AI system creates a black box of tangled connections in which it can be impossible to understand why the system does certain things -- or how to correct those behaviors.
Trying to handicap the AI race between Waymo and Tesla is difficult, says Anthony Levandowski, who co-founded Waymo before leaving acrimoniously for Uber. Now head of a self-driving tech company called Pronto, he believes that Musk's goal of releasing a fully autonomous driving system in a year isn't reasonable.
Creating a self-driving system of the sort Musk wants will probably require more advances in the fundamentals of AI technology itself, and it isn't clear when those will arrive, he adds.
The cost of the sensors used by Waymo and others -- including high-resolution cameras, radar, and "lidar" technology that uses lasers to build 3-D images -- can add up to tens of thousands of dollars, not to mention the expense of mounting them and processing their data.
Tesla's vehicles have only cameras and computing hardware that is generally more modest, cost-wise, than a Waymo vehicle's.
Tesla isn't entirely ignoring the more sensor-focused approach to self-driving AI technology. Andrej Karpathy, a co-founder of OpenAI who also headed AI at Tesla from 2017 to 2022, has said that Tesla is using a small number of vehicles that drive around and use a Waymo-like suite of onboard sensors to create the Tesla driving system's maps. He said this enables Tesla to use some of the same rich data that Waymo and its competitors do, and then deploy that AI on its regular vehicles that are far cheaper to produce.
But having this small unit of Waymo-like vehicles somewhat undermines Musk's claims that the data from Tesla customers' vehicles is sufficient.
Tesla owners like to document their use of its technology, and they have posted a great deal of evidence that undercuts the argument that its vehicles can operate safely with only cameras.
Tesla's current FSD software can drive on most surface streets and highways, but requires vigilant monitoring by the person behind the wheel because it can make sudden and potentially catastrophic decisions. Social media is full of recent videos of such moments -- cars attempting to turn directly into the path of other vehicles, blowing through red lights and failing to stop for a train in foggy conditions.
Federal auto-safety regulators recently announced that they are investigating Tesla over the role its FSD system has played in fatal crashes.
Tesla didn't respond to several requests for comment on its self-driving systems.
Tesla's own operating manuals specify its system can be blinded by direct sun, fog, or other inclement weather. Other self-driving systems that rely on sensors such as radar and lidar can navigate these types of conditions.
"Tesla's computer vision-only approach is never going to work," says Mary Cummings, a professor of computer science at George Mason University and director of its autonomy and robotics center.
A longtime critic of Tesla's autonomous-driving program -- Musk has publicly complained about her -- Cummings recently served as senior safety advisor to the National Highway Traffic Safety Administration, which regulates self-driving tech. She thinks the only way for Tesla to even begin to seriously pursue fully autonomous driving is to change its approach to sensors, by adding more.
Meanwhile, Tesla still needs to get permission to test its vehicles on the road using safety drivers, as Waymo and others have.
"I think it might be true that to get to a completely autonomous vehicle, where it never needs remote assistance, you might need something close to human level intelligence," says Lee. "I do not think that's a case for optimism for Tesla, because if you look at the companies that are trying to build [that], it seems to me they're still pretty far from it."" [1]
1. EXCHANGE --- Keywords: Elon Musk's Overly Ambitious Plans for Robotaxis --- Tesla is pushing to build true self-driving vehicles by 2026. The way its CEO wants to get there seems risky. Mims, Christopher. Wall Street Journal, Eastern edition; New York, N.Y.. 02 Nov 2024: B.1.
