Officers were called to Erdington in the northwest of the city at around midday on Thursday, West Midlands Police said in a statement.
No arrests have been made and enquiries into the incident continue. The victim’s next of kin have been informed.
Investigators remained at the scene in Church Road on Thursday afternoon.
Dozens of police cars could been seen on the street as neighbours expressed shock over the shooting.
One, who wished to remain anonymous, said: “The police swarmed the area, there was flashing blue lights everywhere and some of them had guns.
“You don't expect this, not on your doorstep, and this is a quiet residential area with lots of families and elderly people. The gun crime in this city is getting beyond a joke.
“Everyone is either getting stabbed or shot, nowhere feels safe anymore.
“You hear about it late at night in rougher areas of town but not in the middle of the afternoon in broad daylight.”
West Midlands Ambulance said paramedics had arrived at the scene to find the victim seriously injured.
“Ambulance staff performed CPR and administered advanced life support on the man,” the service said in a statement.
“However, sadly, it became apparent nothing could be done to save him and he was confirmed dead at the scene.”
Responding to the incident, Erdington local councillor and resident Robert Alden said: “Our thoughts and prayers are with anyone effected by this incident.
“Erdington is a great place to live and work. We are a wonderful community here and this kind of event is rare.
“We will be working with our excellent local police in coming days and weeks, as we currently do, to ensure that these kind of incidents remain rare and Erdington remains a safe place to live and work”.
Additional reporting by agencies