Google's self-driving cars drive themselves. Except when they don't, something that's happened hundreds of times. But don't worry. Only a handful of those instances might have ended badly had the human not taken over.
The company announced today that its engineers assumed control of an autonomous vehicle 341 times between September 2014, and November 2015. That sounds like a lot until you realize Google's autonomous fleet covered 423,000 miles in that time. And it's worth noting Google's cars have never been at-fault in a crash, and Google's data shows a significant drop in "driver disengagements" over the past year. Google's rate of disengagements is also far lower than those reported by other companies testing autonomous technology in California, including Nissan, VW, and Mercedes-Benz.1
Of the 341 instances where Google engineers took the wheel, 272 stemmed from the "overall stability of the autonomous driving system"---things like communication and system failures. In other words, something broke. Chris Urmson, the project's technical lead, doesn't find this terribly troubling, because hardware is not where Google is focusing its energy right now. The team's more concerned with how the car makes decisions, and will make the software and hardware more robust before going to market.
The remaining 69 takeovers are more interesting---and important. They're "related to safe operation of the vehicle," meaning those times when the autonomous car might have made a bad decision.
You haven't read any breathless reports of these incidents because of Google's simulator program. If the engineer in the car isn't fully confident that the AI will do the right thing, she'll take control of the vehicle. Later, back at HQ in Mountain View, she'll plug all of the car's data into a computer and the team will see just what the car would have done had she not taken the wheel. It's a bit like the Ghost of Christmas Yet to Come showing Scrooge what may come to pass. According to data Google recently shared with the California DMV (as required by law), 13 of those 69 incidents would have led to crashes.
Google's cars have driven 1.3 million miles since 2009. They can recognize hand signals from traffic officers and "think" at speeds no human can match. They've been involved in 17 crashes, but have never been at fault. Google's previously predicted they'll be road-ready by 2020.
At this point, the team usually can't solve problems with a quick tweak to the code. "We're not really in a situation where we say, 'We should have put a one here and we had a two,'" Urmson says. The challenges are more far subtle and complicated. In a recent case, for example, the car was on a one-lane road about to make a left, when it decided to turn right instead---just as another car was using the bike lane to pass it on the right. "Our car was anticipating that since [the other car] was in the bike lane, he was going to make a right turn," Urmson says. The Google car was ahead and had the right of way, so it was about to make the turn. Had the human not taken over, "there would have been contact," Urmson says. Avoiding repeating so fringe a case isn't easy, but Google says its simulator program "executes dozens of variations on situations we’ve encountered in the real world," which "helps us test how our car would have performed under slightly different circumstances."
Still, Google's getting better. Urmson eagerly notes that its disengagement numbers have dropped over the past year. Eight of the 13 crash-likely incidents occurred in the last three months of 2014, over the course of 53,000 miles. The other five came in the following 11 months and 370,000 miles. Assuming those 13 incidents would have ended with a crash, that's one accident every 74,000 miles---good, but not as good as humans. According to new data from the Virginia Tech Transportation Institute, Americans log one crash per 238,000 miles.
Before bringing its technology to the market, Urmson says, Google must make its cars safer than human drivers (who cause more than 90 percent of the crashes that kill more than 30,000 people in the US every year). But it's not banking on perfection. "You need to be very thoughtful in doing this, but you don't want the perfect to be the enemy of the good," he says. "We need to make sure we can get that out in the world in a timely fashion."
That means Google's disengagement numbers must keep dropping. The downward trend will continue, Urmson says, though as the team starts testing in tougher conditions, like bad weather and busier urban areas, you'll see occasional upticks. "As we push the car into more complicated situations, we would expect, naturally, to have it fail," Urmson says. "But overall, the number should go down."
1Story updated at 12:15 EST to include newly released disengagement data from other automakers.
