Recent crashes have shaken the public's confidence in self-driving cars.
The number of people who distrust self-driving cars rose significantly over the past year, according to an AAA poll released Tuesday.
The survey, which was conducted in April, showed that 73% of American drivers would be "too afraid" to ride in a self-driving vehicle, compared to 63% in late 2017.
Only 20% of respondents said they trust a self-driving car.
"Any incident involving an autonomous vehicle is likely to shake consumer trust, which is a critical component to the widespread acceptance of autonomous vehicles," said Greg Brannon, director of automotive engineering and industry relations for AAA, in a statement.
Madison Gross, senior manager of customer insight for CarGurus, said that safety was a big concern for passengers.
A self-driving SUV from Uber hit and killed a pedestrian in Tempe, Arizona, in March. That was the first time a self-driving car killed someone who was not in the car. Uber pulled its self-driving cars from the road, and the state of Arizona suspended testing.
In March, a Tesla Model X was in semi-autonomous "autopilot" when it crashed into a highway median in Mountain View, California, killing the driver.
Brannon, of AAA said, in a statement, that self-driving technology "holds the promise to make our roadways safer, but the industry will need to execute testing in the safest manner possible."
Experts have said that self-driving cars are not necessarily more dangerous than human drivers -- and they have the potential to eventually be safer. More than 37,000 people died on US roads in 2016, according to the most recent figures from the Department of Transportation's National Highway Traffic Safety Administration. That's an increase of more than 5% from the year before, and the top causes include distracted driving, drowsiness, speeding and drunk driving.
"There are accidents every day, but there's a magnifying glass on autonomous vehicles because there's a perception that they're going to be flawless -- and they're not going to be flawless," said Michelle Krebs, executive analyst with AutoTrader.
Reliable statistics on the safety of self-driving cars are unavailable since the technology is relatively new and the testing is in its early stages.