Pym’s heroics in normal time paid dividends as substitute Rhys Oates sent the game to penalties with five minutes to go by cancelling out Anthony Musaba’s opener.
Pym kept out both spot kicks from Will Vaulks and Liam Palmer to secure a third-round spot as the Owls’ horror start to the season continued while Stags remain unbeaten.
It was the first time Mansfield had made round three of the competition since 2005.
Boss Nigel Clough made a trio of changes to the side which beat Stockport County on Saturday afternoon.
Ollie Clarke, Will Swan and Lewis Brunt, making his full debut for the club, were included in the absence of George Williams, Stephen Quinn and Oates.
The Stags had an early penalty shout waved away after Clarke went down inside the box as he tried to get on the end of Jordan Bowery’s teasing cross.
But Yorkshire’s Owls took the lead in the 28th minute as Musaba headed home Callum Paterson’s ball back across goal.
Mansfield could have restored parity within minutes as Lucas Akins nodded Aden Flint’s cross straight into the hands of stopper Cameron Dawson.
Then Davis Keillor-Dunn squandered a decent chance as he blasted over the bar in search of an equaliser before Louis Reed dragged wide of the mark.
Stags got lucky in first-half stoppage time as Josh Windass saw his powerful strike rattle the crossbar in the 47th minute.
The visitors needed stopper Pym to keep then in the game shortly after the restart.
First, he kept out Michael Ihiekwe with a stunning stop before he palmed Windass’ fiercely-struck 20-yard effort away from danger.
Mansfield saw a great chance go begging as Bowery fired wide of the mark on the hour.
Down the other end, Dominic Iorfa nodded over the bar after meeting Barry Bannan’s dangerous delivery.
Sent on after 62 minutes, Oates livened things up and seemingly took advantage of Akin Famewo’s slip at the back before the defender recovered well with a last-gasp challenge.
Pym produced a brilliant stop to tip Juan Delgado’s header onto the post to keep the visitors in the game with 10 minutes to go.
But Oates’ levelled late on with a fine strike from the edge of the box after bullying his way through a number of hosts’ challenges to force the dramatic shoot-out.
The first five penalties saw Jordan Bowery, George Maris, Keillor-Dunn and Oates net their kicks with Will Swan having his saved and Pym denying Vaulks.
Former Wednesday star Aden Flint stepped up to put Stags 5-4 ahead before Pym provided the crucial stop to send Stags’ travelling army of fans into hige celebrations.
WEDNESDAY: Dawson, Iorfa, Ihiekwe, Famewo, Valentin (Palmer, 81), Bakinson (Vaulks, 81), Bannan, Paterson (Delgado, 65), Windass, Fletcher (Smith, 71), Musaba (Gregory, 71). SUBS NOT USED: Gassama, Bernard, James, Vasquez.
STAGS: Pym, Bowery, Flint, Brunt, MacDonald, Reed, Clarke, Maris, Keillor-Dunn, Swan, Akins (Oates, 62). SUBS NOT USED: Flinders, Cooper, Williams, Abdullah, Carter, Wauchope, Kruszynski, Turner.
REFEREE: Sunny Gill
ATTENDANCE: 10,724 (1,800 away approx).