Talking to your car is better than ever, thanks to Nuance's voice assistant

Get ready to have some real conversations with your car.

No, not the kind where you scream at the navigation system for taking you through New Jersey, or break into a crying jag because the car ran out of gas somewhere between Brooklyn and Montauk.

No, these are real conversations that might, in fact, head off those messy ones.

Nuance Communications is the company responsible for the original voice recognition and speech technology for Siri, and the parent of one of the oldest and most-respected voice recognition systems, Dragon Naturally Speaking. For years they've been busy building voice assistants for virtually every major auto manufacturer. Now, they're prepping the next phase in driver and passenger car/human interaction. And I got to test drive it.

SEE ALSO: Pour one out for Firefly, Google's quirky little self-driving car

On a very sunny and exceptionally pleasant summer morning, Nuance pulled up to my office in a brand-spanking new Chrysler Pacifica. The mini-van, while lovely, was not the story. Nuance is using it as a concept vehicle and, as such, has retrofitted it with their touch screen (the original console was inside a drawer below) and a new microphone array that provides listening for all six of the car's passengers (I could see my mic embedded just above my head in the car roof). Nuance uses something called Passenger Interference Cancellation, or PIC, to isolate background noise and shut down the listening on remaining mics when its Dragon Drive System is engaged with one speaking passenger or driver.

Image: lance ulanoff/mashable

Nuance, however, is doing more than just providing a hardware upgrade. The screen and microphones are a way for Nuance to illustrate to automotive partners what's possible with their impressive software solution and, perhaps, inspire ideas for their next lines of automobiles.

One of the key themes, said Nuance Dragon Drive Solutions Marketing Manager Robert Policano, who was seated behind the driver in the Chrysler, is artificial intelligence and "how we use it on top of speech recognition and natural language processing to make it more contextual, offer more relevant results, and make it more humanized."

Over the last two years, Nuance has started using deep neural networks and machine learning to train their voice recognition system and, especially, to add context to the engagement.

So, now there's an intelligence layer that, according to Dragon Drive Senior Product Manager Lior Ben-Gigi, who was in the driver's seat next to me, uses not only what people say, but also the context available in previous actions, speech, car metrics, and even intention.

For example, Ben-Gigi prompted Dragon Drive by saying "Hello Dragon," and then asked it to navigate a route to Atlantic City. 

"I'm not thinking about fuel, I'm just heading out to Atlantic City," said Ben-Gigi.

The system agreed to the navigation, then took stock of the car's gas mileage and the amount of gas in the tank. It warned him with a visual alert that he wouldn't make it far without refueling, and offered to find a gas station. When Ben-Gigi accepted, Dragon Drive pulled up gas stations with a focus on the brand he preferred.

Image: lance ulanoff/mashable

Similarly, when Ben-Gigi asked Dragon Drive to find parking from 1 p.m. to 3 p.m. with a garage that accepted Discover Cards, it immediately identified locations that would fit those parameters and, behind the scenes, searched for a garage that can handle the vehicle size. It would also look for garages with Accessible Parking if, on previous drives, he'd specified that request.

This is part of Dragon Drive's embedded learning function. Over time, it can build a profile of you and even your passengers, provided they've created a voice profile.

Nuance's AI, in-car voice assistant solution, is also unusual because it combines embedded learning and cloud-based smarts. While it's pulling in contextual information from the car, sensors, and passengers, Dragon Drive is also collecting necessary cloud data from third-party partners like Parkopedia (for the parking garage data) and music services like Pandora. The cloud-based information can come from a tethered smartphone or via in-car mobile broadband.

Image: lance Ulanoff/mashable

In the Chrysler Pacifica, Nuance's Dragon Drive did seem to keep a better handle on context and intention than other more modular in-car assistants I've used. At one point, we set a route to go to the Museum of Natural History but decided, mid-route, that we wanted ice cream. The Dragon Drive system recommended an ice cream place that was just a couple of miles out of the way. Then we asked it to find something close to our destination, which it did and added that to our trip itinerary. This was all done via voice and without any confusion on the Dragon Drive system's part.

Nuance is also looking forward to the future, when self-driving vehicles will leave us with more time to do something other than driving. We played a music trivia game, "Dragon Tunes," where the the system played a song and passengers had to identify the artist and song. To do so you just called out "Got it." The concept car's PIC system instantly identified who was speaking and then let them respond.

However, even with the sophisticated microphone array, Dragon occasionally got confused about who was answering. Since we weren't moving, some passengers were leaning far forward or turning their heads, which may have caused the disconnect.

Image: lance ulanoff/mashable

It did better when I said, "Hello Dragon," paused for  the listening beep, and added, "I'm cold" — and the system adjusted the car temperature for my zone. Thanks to the microphone array, Dragon Drive knew where I was sitting. 

Nuance is already working with car companies like Ford, Chrysler, and BMW to integrate current Nuance systems, but the future of Dragon Drive is coming fast. Ben-Gigi says we could see systems like this in production model cars within the next two years.

Throughout the demo, I was struck by Dragon Drive's natural tone. Ben-Gigi calls this "Expressive Text-to-Speech." It reminded me of Apple's efforts to make Siri sound more natural in iOS 11.

It's one thing Nuance and Apple have in common, though their approaches to in-car technology are very different.

Policano said Nuance is building an agnostic platform, while companies like Amazon and Apple are seeking to build something different for the car industry. They have, he said, an ecosystem agenda.

"While Apple is building an Apple experience, Nuance is helping BMW build the BMW experience," said Policano.

WATCH: