Featured Product
This Week in Quality Digest Live
Innovation Features
Ian Wright
MIT and ETH Zurich engineers use computer vision to help adjust material deposition rates in real time
NIST
Having more pixels could advance everything from biomedical imaging to astronomical observations
Chris Caldwell
Significant breakthroughs are required, but fully automated facilities are in the future
Leah Chan Grinvald
Independent repair shops are fighting for access to vehicles’ increasingly sophisticated data
Adam Zewe
How do these systems differ from other AI?

More Features

Innovation News
Exploring how a high-altitude electromagnetic pulse works
High-capacity solution using TSMC’s 3DFabric technologies
EcoBell paints plastic parts with minimal material consumption
Study of intelligent noise reduction in pediatric study
Easy to use, automated measurement collection
A tool to help detect sinister email
Funding will scale Aigen’s robotic fleet, launching on farms in spring 2024
High-end microscope camera for life science and industrial applications

More News

Abdesalam Soudi

Innovation

How Will Autonomous Vehicles Interpret Hand Signals?

Driverless cars might follow the rules of the road, but what about the language of driving?

Published: Thursday, January 11, 2018 - 12:03

Recently, while on my way to the University of Pittsburgh’s campus, I made a quick “Pittsburgh left”—taking a left turn just as the light turns green—while facing a driverless car.

Instead of jolting forward or honking as some human drivers would be tempted to do, the car allowed me to go. In this case, the interaction was pleasant. (How polite of the car to let me cut it off!)

But as a sociolinguist who studies human-computer interaction, I started thinking about how self-driving cars will communicate with the human drivers they encounter on the road. Driving can involve a range of social signals and unspoken rules, some of which vary by country—even by region or city. How will driverless cars be able to navigate this complexity? Can they ever be programmed to do so?

What driverless cars can do

Here in Pittsburgh, Uber has tested self-driving cars with a backup driver behind the wheel; in Phoenix, Waymo’s cars operate in a limited part of the city without any backup driver at all.

We know driverless cars are equipped with a technology called LiDAR, which creates a 360-degree image of the car’s surroundings. Image sensors can interpret signs, lights, and lane markings. A separate radar detects objects, while a computer incorporates all of this information along with mapping data to guide the car.

Although ideally autonomous vehicles will be able to “talk” to one another in order to allow smoother navigation and reduce crashes, this technology is still in the early stages.

But any autonomous vehicle will also need to be able to interact with traditional cars and their drivers, as well as pedestrians, bikes, and unforeseen events like lane closures, disabled stop lights, emergency vehicles, and accidents.

The complex language of driving

This is where things can get murky.

For example, if you’re driving and pass a speed trap, you might flash your headlights at drivers coming in the other direction to let them know. But flashing headlights can also mean “your high beams are too bright,” “you forgot to put your headlights on,” or “go ahead” in situations where it’s unclear who has the right of way. In order to interpret the meaning, a person will consider the context: the time of day, the type of road, the weather. But how would an autonomous vehicle react?

There are other forms of communication to help us navigate, ranging from honks and sirens, to hand signals and even bumper stickers.

Of course, humans use all sorts of hand gestures—waving a car in front of them, indicating that another driver needs to slow, and even giving the finger when angry. Sounds can communicate love, anger, arrivals, departures, warnings, and more. Drivers can express total disapproval with a hard, extended hit of the horn. Of course, emergency sirens encourage drivers to make way.

But specific meaning can vary by region or country. For example, a few years ago, Public Radio International ran a story about the language of honking in Cairo, Egypt, which is “spoken” primarily by men. These honks can have complex constructions; for example, four short honks followed by a long one mean “open your eyes” to warn someone who is not paying attention.

Traffic moves through Tahrir Square in Cairo, Egypt

In Pittsburgh, people tend to honk before going through a short, narrow, or curvy tunnel. In Morocco, where I’m originally from, drivers perform varied honks when passing; they’ll honk once before passing to secure cooperation, again as they pass (to signal progress), and lastly after they pass to say, “Thank you.” Yet this might be confusing, or even perceived as rude, to drivers in the United States.

Written communication also plays a role between cars and drivers. For example, signs such as “Baby on Board” or “Students on Board” are supposed to encourage the drivers following these vehicles to be even more careful. Bumper stickers like “Caution: Wide Right Turn” or “This Vehicle Makes Frequent Stops” can be critical to safety.

What if there’s a communication breakdown?

Vehicles can be taught to “read” road signs, and thus presumably can be taught to recognize common warnings on bumpers.

Yet navigating construction sites or accident scenes may require following directions from a human in a way that cannot be programmed. This creates a huge opportunity for error. Because hand signals vary widely from region to region (and even person to person), autonomous cars could fail to recognize a signal to go or, more catastrophically, could mistakenly follow a hand gesture into a barrier or another car.

This gives me pause: How much knowledge about our societal and linguistic values are built into the system? How can driverless cars learn to interpret hand and auditory signals?

Google cars can apparently recognize hand signals on bikers, but what if the biker doesn’t use standard signals? Who gets to embed the algorithm in the machine, and how are sociolinguistic values assigned?

In my experience, the self-driving car was very polite and didn’t honk or otherwise chastise me for my behavior (though the human passenger did communicate his displeasure with a gaze). But had I waved the car in front of me, would it have been able to respond appropriately? A 2015 story in Robotics Trends described how a bike and a Google car got stuck in a standoff when the car misread signals from the biker.

The ConversationCities (and countries) possess a variety of sociolinguistic cues. It remains to be seen if engineers working on driverless cars will be able to program the subtle but important differences into these vehicles as more and more appear on the roads.

This article was originally published on The Conversation. Read the original article.

Discuss

About The Author

Abdesalam Soudi’s picture

Abdesalam Soudi

Abdesalam Soudi is a sociolinguist at the University of Pittsburgh.

Comments

How do Human Drivers Interpret Them?

Why exactly does someone else’s autonomous vehicle need to “understand” me? When I walk into work to “push the green button” on my $2,500,000 autonomous manufacturing cell, it does not need to know I am not clear headed because was up late with a cold last night. The machine will do what the machine is supposed to do. The safety features designed into the machine will do what they are supposed to do. If I do what I am NOT supposed to do and defeat a safety feature to enter a “Do Not Enter” area, I am the one causing my accident. To expect a machine to account for “stupid human tricks” (as David letterman used to call them) is not only impractical it is entirely unreasonable.

Even today, if someone were to intentionally step off a curb into fast moving traffic, the driver of the car that strikes them is not held liable for the accident. The human has to own their part of the equation.

The rules of the road are the rules of the road. While the “Pittsburgh left” is apparently a common practice that makes it no less illegal (or rude). In the author’s scenario, the safety feature of that particular autonomous vehicle was proven effective. Had a less fortunate outcome been the result, the fault would lay with Mr. Soudi, not the other vehicle.

The easiest way to understand this is to remove the word “autonomous” from the vehicle description. In the above case, the author was simply lucky the other driver was willing to allow for the rude behavior and avoid an accident (we know all too well that all human drivers may not be that conscientious). Just because there were no hand gestures, honking or light flashing, does not change the outcome. The machines are programed to do he correct thing. Human drivers carry a legal accountability to do the right thing. The only variable with multiple degrees of freedom is the human driver.

Regarding the social interactions humans exchange while in their vehicles; not all do. Being insulted by a computer is tantamount to hating your hairbrush because it didn’t tell you how good you look.

Thanks,

M. Goodsell

Driver less cars

Some times comkents are more striking than the original artilce

thank you

comment

Thanks for sharing, I found the article very interesting and formative at the same time, I believe that it would be a good step forward in this market.

Autonomitis?

I thought the whole idea of autonomitis in motor vehicles was to provide a wiser altenative to managing safety than relying on human behaviour. 

Many high risk industries understand the stupidity of relying on human stupidity as a risk mitigation.  The safety imperative is always to create a safe system regardless of human behaviour.  For example, in theory, traveling from A to B in reach of the controls of a car shouldn't be Russion Roulette.  Drunk driving is stupid behaviour, so was drunk horse riding!  The system should cope with it because it's entirely expected, so is going to sleep, eating, gazing at the view and so on...  We are analogue beings, we get distracted, we ain't digital!

I fear vehicle "autonomitis" will promise much but achieve less until it infects also the systems responsible for the whole land tranport infrastructure.  Many road configurations are lethal, yet those responsible for them escape investigation and conviction as the driver is the only potential culprit.

As for the "autonomitic" car taking account of human stupdity at a road juction, maybe one day a police app "extra" will appear that autonomously issues a ticket!