Ms Herriot-Sullivan will be the first woman to hold the position, according to ABC News.
Mr Prude died in March after he was pinned to the ground by Rochester police officers.
Officers encountered him while he was having a mental health emergency. He died a week after his encounter with the officers, during which an officer put a "spit bag" on his head and pinned him to the ground.
Seven officers at the encounter were suspended without pay.
The city's former police chief, La'Ron Singletary, was fired in the wake of Mr Prude's death.
Rochester Mayor Lovely Warren announced the change during a press conference on Saturday.
"Traditional policing practices must be altered and improved to better serve and protect our citizens," she said.
She said Ms Herriott-Sullivan brings a "fresh approach to policing" and that she is "uniquely qualified to deal with the many current issues that the city of Rochester is facing."
Ms Herriott-Sullivan is a nearly-24 year veteran of the Rochester Police Department. She left the department in 2009.
"Interestingly, I left law enforcement because I wanted to have a bigger hand in helping people stay out of jail, rather than putting in that," she said. "So I moved on to roles helping deal with criminal justice disparities."
Mr Prude's death was ruled a homicide caused by "complications of asphyxia in the setting of physical restraint."
New York Attorney General Letitia James expressed outrage after Mr Prude's family released a video taken by an officer's body camera showing the incident. They claimed the police attempted to cover up the incident.
Ms James said on 5 September a grand jury will investigate Mr Prude's death.
On Sunday, Ms James announced there would be reforms in how police body camera footage is used. Under the new policy, police body camera footage will be released earlier in the investigation process. Once families have had a chance to see the video and jurisdiction has been established, the videos will be released.