Hod Lipson, a professor of mechanical engineering and the director of the Creative Machines Lab at Columbia, is a roboticist who researches artificial intelligence and digital manufacturing. In a new book, Driverless: Intelligent Cars and the Road Ahead (MIT Press), Lipson and his coauthor Melba Kurman assert that recent advances in software and robotics will accelerate the development of autonomous vehicles. We asked Lipson to explain the technology and where it will take us next.
People have been talking about driverless cars for decades. Why is this a critical moment?
Even back in the 1930s, the automotive companies — General Motors in particular — thought driverless cars were the future. They talked about building smart highways, which are roads that can communicate with vehicles. But the technology was in the infrastructure, which made it impossible to implement. It was just too expensive per mile.
The more recent approach is focused on autonomous robotics. It puts the smarts and technology in the car. An intelligent machine has to deal with the same infrastructure and driving challenges as humans — lane markings, traffic lights, poor weather. It needs to see as humans see.
In the past few years there’s been an explosion in artificial intelligence (AI), and in particular “deep learning.” For driverless cars, deep learning trains the vehicle to drive by feeding it huge amounts of visual data gathered by camera during trips. That’s provided that last piece of the puzzle, which is artificial perception, or basically the ability for the car to see. There’s been this magical pivot. Even though people have been hyping driverless cars for decades, this is different. This time it’s real.
Why is deep learning so pivotal?
The challenge in making a fully autonomous driverless car is that we have to think about three levels of control.
On one level, the car has to stay in a straight line and take a turn at exactly the right angle. That’s hard, but it’s been solved by mechanical engineers. On another level, it has to get from New York to DC. That’s also been solved. We use navigation systems on our phones all the time.
What hasn’t been solved, and what has always been a challenge, is what we call “midlevel” controls. It’s not just about going in a straight line, and it’s not about calculating the optimal route. It’s about going around obstacles, stopping at a stoplight, negotiating an intersection, merging into traffic, hitting the brakes because a child is running after a ball, not hitting the brakes because there’s a shadow on the street. Everything else was solved decades ago. This was the part nobody knew how to do.
How has your own research contributed to deep learning?
Our work focuses mostly on the question of how an AI network learns and how we can transfer learning from one network to another.
That’s one of the great advantages of driverless cars. No human driver can have more than one lifetime of driving experience. A car that is part of an AI network can, within a year, have a thousand lifetimes of experience, because it can get information from all the scenes and situations that every other car has experienced. It’s another reason why these cars will drive better than any human has ever driven. They will have experienced every possible situation.
So when will we start to see these autonomous vehicles on our roads?
It’s impossible to pinpoint a year, but we can pinpoint a range. About ten years from now it will begin, and forty years from now it will end, meaning that by 2065 all cars will be autonomous. That’s kind of the range.
It also won’t be uniform, in the sense that it will start in certain places but not others. New York might determine that all cars in Manhattan should be autonomous. The technology could be adopted in vacation resorts or in some brave city like Detroit that is willing to pioneer the technology. It could start in trucks but not private cars. That’s why you can’t pinpoint beyond a range. We know it will start in spots and gradually spread.
Tech companies and car companies are taking very different approaches to this driverless world. You describe the divide as “software versus automotive.”
Tech companies are looking at this as a software or AI play. The platform is a commodity, like a cell-phone body. You can get the car body from anywhere. The motors, the engine — they’re off-the-shelf. But the real smarts is in the software. This is how the software companies are approaching the driverless car. They see it as software on wheels.
The automotive companies, on the other hand, think of it as a car with extra software. The technical term is “driver assist.” Cruise control was a very primitive driver assist; then we developed automatic brake systems and automatic lane keeping. Car companies see this as an evolution of driver assist, where the driver remains behind the wheel, ready to take over in an emergency. Only in the final stage of the technology’s evolution will the car become fully autonomous.
The software camp imagines having no human driver right from the get-go. They’re coming at the problem with a completely different attitude.
You argue that the driver-assist approach is not just worse but actually dangerous. Why is that?
This idea that an intelligent machine should hand off to a human driver in an emergency is very problematic. There are examples of recent plane crashes where the machine handed the controls to the pilots, but the pilots weren’t ready. There’s another case where pilots tried to land a plane themselves, but they were out of practice and they crashed.
The recent fatal Tesla crash demonstrates the risk of the driver-assist approach. A system that is “almost fully autonomous” is dangerously deceptive. And the more you’re automated, the more severe the problem is. If you drive for an hour, it’s hard enough to remain focused. If you’re not driving, how can you stay focused enough to take over? It’s dangerous and unreliable.
You say the technology for driverless cars is here, but elements like regulation and insurance liability are not even on the radar.
These things need to be sorted out. But they’re not technological problems. These are quantifiable safety and liability issues.
The government needs to determine the minimum level of safety that an autonomous car needs to exhibit before it can drive freely. We think that level is four hundred thousand miles between collisions. That’s twice as good as a human, on average. But I really don’t understand why this is taking so long. I think this is a fairly simple proposition, and meanwhile people are dying.
Just how big are the safety benefits of driverless cars?
Do you know how many people in the world die every week because of cars? Twenty-eight thousand. That’s a Hiroshima-scale disaster every month. For people between the ages of fifteen and twenty-nine, automobile accidents are the number-one killer. And yet we don’t talk about it. We accept it. We can have silly debates about the ethical dilemmas of driverless cars, which discuss who the car should save in the event that it can’t avoid a fatal crash. But for every week we delay, another twenty-eight thousand people die.
Can we assume that driverless cars will improve the environment?
When Melba and I began researching the book, we naively thought that autonomous vehicles would be a win for the environment. But it’s not clear. Driverless cars are more convenient, so miles driven will go up. That’s not a good thing.
But then you have to look at other factors, like the fact that driving will be more efficient. There will be more consistent speeds, less start and stop, less parking and idling and traffic jams. These things will amount to substantial improvements in performance. Also, the vehicles can be smaller and lighter, so even though we may see that increase in miles, the vehicles will be more efficient.
The economic impact of driverless cars is going to be huge, for better and for worse.
Yes, a lot of people will lose their jobs, and not just truck drivers and taxi drivers. Dozens more professions will be transformed. The body shop, where people fix cars after collisions — that’s going away. And how many healthcare hours are devoted to car accidents? A huge number. How much income do parking tickets generate for cities? It’s not negligible. There’s this cascade effect.
But then we’ll see some opportunities. There will be a huge ripple effect on new e-commerce models and business models, creating new jobs. More miles driven means more cars sold, regardless of the ownership model, and more cars mean more car maintenance.
You end the book by comparing the rise of driverless technology to a new stage of human evolution. Will it really be that significant?
Some evolutionary biologists believe that vision was an accelerating force of evolution and natural selection. The “light-switch theory” suggests that once changes in the atmosphere allowed more light to reach the earth, and we developed light-sensitive eyes in response, the ability to see enabled a lot of other new technologies, including camouflage, running quickly, and predation. Once you can see, you have to create a model of your world, and you need a bigger brain. It all unfolds from there.
That’s exactly the case with deep learning. The ability to see and perceive is not just another item on the laundry list of things you need to make a robot. It is the pivotal piece that sparks everything. This is why it was an inflection point in biology, and I believe it’s going to be the inflection point in autonomous vehicles and robotics.
We’ve been dreaming about this moment for decades, centuries, millennia perhaps. Finally it has arrived.
