Giving our Autonomous Vehicles the Power of Choice
It’s fairly straightforward to tell an autonomous vehicle (AV) how to respond to specific situations it might encounter on the road. In situation 1, do X. In situation 2, do Y. The challenge comes from the number of situations that an AV might encounter. Will the AV do the right thing in situation 43,768?
At May Mobility, one of our key technologies is Multi-Policy Decision Making (MPDM). MPDM is our way of reframing the challenge: instead of telling the vehicle when to do what (which is hopeless), we give the vehicle options and let it decide for itself which option is best.
In a sense, we’re giving our AV’s an imagination.
By giving our AV’s an imagination– the ability to imagine what might happen if it does X, or if it does Y, we can simulate the likely outcome of X (perhaps “pass the car in front of us”) versus the likely outcome of Y (perhaps “follow the slow car”). This imagination makes it much easier to determine whether X or Y is the better option. MPDM is not making decisions at the individual “action” level– for example gas/brake/steer, or specific trajectories that our car will follow. In contrast, each policy that we choose between is a driving strategy that can adapt its behavior in response to what other road users are doing.
It’s fair to say that every AV company does simulation offline. Their reason for doing so is to measure the quality of their AV software. May Mobility has differentiated our approach by doing simulation online; simulation that accounts for the actions and responses of our AVs to what’s taking place on the road is very different. Understanding the way that road users interact with each other is absolutely critical to driving well.
To achieve online simulations that enable an AV to imagine outcomes, the MPDM simulator on our vehicles has to be extremely fast. For example, MPDM runs tens of thousands of times faster than real-time, which allows May to imagine different future outcomes. We can vary both our AVs strategies as well as the strategies of other road users. No single simulation is right, but by simulating enough possible scenarios, we can decide whether strategy X is better/safer/more comfortable than strategy Y. And this is where things get interesting. A consequence of electing a particular policy with MPDM is that we actually don’t know what the vehicle will do. Not exactly. And of course we don’t know exactly what other road users are going to do either. It doesn’t make sense to say “here’s the trajectory I’m going to follow.” What we do have is a probabilistic characterization of the types of trajectories that a policy might generate. And that lets us pick the best/safest/most comfortable trajectory.
Put in layman’s terms, everyone has experienced driving on the road with “so-so” drivers – the ones who technically follow the rules of the road but create hazards for other road users. For example, you’re not required to make space for a vehicle merging in, but it makes the road safer for everyone if you do. When we consider the result of choosing strategy X or Y, we evaluate the safety for everyone, not just our own vehicle. And so our vehicle leaves a little more space for a bicycle or pedestrian, even if it didn’t have to.
Of course, MPDM isn’t magic, and there are still iterative opportunities that we can improve upon. Building better agent models– so our predictions are better– is a big one. Designing policies and ensuring our AVs learn them is another. The remarkable thing about our technology stack is how consistent it has been in design and intention. We’ve simply been pushing it forward along a path we imagined back in 2017, step by step.
The advantages of MPDM are huge. Instead of telling the AV when it should do what– which doesn’t scale– we let the AV imagine the outcomes itself. That leads to emergent behavior, where the vehicle can solve problems that we never considered. Our system gets smarter every time we add a new policy – we’re up to 14 now, because MPDM has more options to choose from. And that efficiency is one of the reasons why May Mobility has been able to launch in seven cities, despite being much smaller than most AV companies. Technology matters and we’re proving it every day.
All you need is a little bit of imagination.
To learn more about MPDM, you can watch our introductory video here.