A penalty shoot-out was required to separate John Mousinho’s men and Peterborough following an intriguing second half.
Trailing to David Ajiboye’s goal at the break, Christian Saydee levelled with his maiden Blues goal, while the visitors were then asked to see out the final 31 minutes following Charlie O’Connell’s dismissal.
Inevitably Pompey dominated from that point, laying siege on the final third, without finding a way through against the 10 men.
With no extra-time, Mousinho had brought Colby Bishop, Marlon Pack and Joe Morrell off the bench for penalties in the event of a draw.
Sure enough, they were required and when Ryan De Havilland’s spot kick was saved, Abu Kamara had the chance to win, only to blaze over.
It headed into sudden death, with Morrell striking the right-hand post = and Ajiboye stepped up to win it 5-4 for the Posh.
It marked elimination for Pompey in the second round - and a first defeat in 19 matches in all competitions.
Mousinho had made 10 changes to the side which collected a goalless draw at Stevenage at the weekend.
Only Jack Sparkes kept his place, although selection was driven by the desire to rotate Pompey’s squad rather than a reflection of that Lamex Stadium performance.
Even though he was the sole survivor, Sparkes was handed a different position, switching from the left wing to his customary left-back role.
Elsewhere, Koby Mottoh was handed his second start, Zak Swanson replaced the suspended Joe Rafferty at right-back, and two-goal EFL Trophy hero Sean Raggett also returned.
Christian Saydee led the attack, while Terry Devlin, Alex Robertson and Paddy Lane also started.
As an insurance option, Mousinho named a strong nine-man bench containing a number of first-team regulars, including Regan Poole, Colby Bishop, Joe Morrell and Marlon Pack.
When the game got underway, it took five minutes before Pompey drew a first save from Fynn Talley in the Posh goal.
A poor clearance fell to Devlin outside the box and his right-footed shot took a deflection and was brilliantly clawed away by the goalkeeper for a corner.
At the other end, on 10 minutes, David Ajiboye cut inside from the right and fired off a low shot which was kept out well by Ryan Schofield, with plenty of players in front of him.
Moments later, Robertson crashed in a shot from just outside the box which forced a sprawling save from Talley.
Lane closed in for the rebound, with the keeper again blocking the attempt, only for Pompey’s Irishman to run into the post, injuring his right shoulder.
After lengthy treatment, Lane left the pitch, rubbing the back of his head, and thankfully was able to continue.
The deadlock was broken on 29 minutes following a corner from the left delivered by Kai Corbett.
That was met by Kabongo Tshimanga, whose header appeared to be going wide until kept in by Josh Knight, and there was Ajiboye to finish, with his shot going straight through Schofield.
That goal knocked the stuffing out of the Blues, who struggled to get going again for the remainder of the half, with Peterborough well in control at 1-0.
The Blues were reinvigorated after the break and levelled after 51 minutes through Saydee’s first goal for the club.
It was Raggett’s excellent crossfield pass to Lane on the right which started off the move, the Northern Irishman then delivering a left-footed cross into the box towards Saydee.
The striker controlled it on his chest and then, when the ball bounced, somehow hooked a right-footed shot into the net to make it 1-1.
On 58 minutes, Swanson produced a delightful ball down the right to put Saydee into the box, only to tumble to the turf, but no penalty was given.
The Posh were reduced to 10 men on 59 minutes when O’Connell earned his second yellow card following a foul on Devlin.
Immediately Abu Kamara and Regan Poole were introduced off the bench and, minutes later, Bishop also joined them on the pitch.
The hosts had the tempo, the crowd was up, now it was a case of making the numerical advantage count, always the hardest part.
Bishop had appeals for a penalty, then later couldn’t control a golden chance inside the box, while Sparkes fired a free-kick straight at the keeper.
In stoppage time, Pack drew a fingertip save from the keeper - and it headed into penalties.
Bishop, Raggett, Pack and Saydee all netted, yet crucially Kamara and Morrell missed, leaving the Blues nursing a rare defeat.