Delano Burgzorg had earlier put the Terriers ahead in the 33rd minute, before John Swift equalised midway through the second half.
Huddersfield started brightly and had the ball in the net in the eighth minute through a rising, angled volley from Rudoni – but the goal was ruled out for a push by Michael Helik on Darnell Furlong in the box.
The visitors continued to look dangerous and Ben Wiles’ low drive was deflected wide before Matty Pearson looped a header over from a free-kick.
West Brom threatened when Brandon Thomas-Asante’s glancing header was just too close to goalkeeper Lee Nicholls before Matty Phillips curled an effort wide after cutting inside.
Huddersfield went in front when Dutch forward Burgzorg, on loan from German outfit Mainz 05, arrowed a low, angled drive past Baggies goalkeeper Alex Palmer at his near post from 15 yards.
Burgzorg, making his first start for the Terriers, went close again when he fired wide before
West Brom finally mounted some pressure late in the first half when Thomas-Asante curled straight at Nicholls.
The home fans were celebrating in the 52nd minute after an equaliser following a sweeping move.
Jayson Molumby’s pass found Thomas-Asante on the half-way line and they striker turned before finding Swift to his right.
Swift drew Nicholls before firing low into the far corner on his 50th league appearance for West Brom.
Albion substitute Josh Maja was denied by Nicholls when clean though, and it was Huddersfield who snatched a late winner in the sixth minute of added time.
Substitute Kian Harratt crossed deep to the far post and Rudoni controlled in space before blasting home a fierce low drive inside Palmer’s near post.