Gus Poyet struck two minutes from time to make sure it was seventh heaven for us against Pompey after another amazing 90 minutes at the Lane.
It looked like another batch of 'lead' jokes were in the pipeline from the critics as we led 1-0, 2-1 and then 3-2 only for Pompey to hit back on each occasion.
Robbie Keane's solo effort - he was outstanding - put us 3-2 up with 11 minutes left but Pompey newcomer Ivica Mornar slid in the third equaliser four minutes later.
So it's to the lads' credit - not least after what famously happened against Manchester City in the FA Cup just three days earlier - that they managed to find a winner.
The source of that winner was Dean Richards, who won a towering header from Simon Davies' late corner to allow Poyet to convert with his knee from close range. Thankfully, there was no way back from that.
As with the City tie, everything seemed to be going swimmingly at half-time.
Jermain Defoe showed why he's seen as one of the deadliest young strikers around to bag his first Spurs goal 12 minutes into his debut and there was some fine, free-flowing football.
Our forward line looks full of goals and it was no surprise to see Defoe and Keane link-up so early for the opener.
Ledley King deserves credit for picking out Keane, Defoe instantly found a yard of space in the box, Keane in turn found him and the ball found the bottom corner of the net from 16 yards.
Shaka Hislop did well to block a Davies chance with his legs and John Curtis looked all over Michael Brown in the box but penalty claims were waved away.
Pompey's equaliser had alarm bells ringing though. Yakubu broke the offside trap and crossed low, Stephen Carr attempted to clear only for his clearance to hit Eyal Berkovic and sail past Kasey Keller. Would it be one of those days?
Any doubts lasted only two minutes. Keane and Defoe were involved again, this time Defoe stepping over a pass from Dalmat, confusing Pompey's defence. The ball looped up and Keane fainted to go right, allowing the ball to bounce before lashing home a left-foot volley. A brilliant goal.
Returning Lane legend Teddy Sheringham limped off at the break for Pompey, replaced by the giant Ivica Mornar, as the visitors came out with renewed purpose.
They dominated for the most part in the second half but couldn't hold the likes of Keane, Davies and Brown on the break - it would prove their undoing.
Keller was forced into action twice by Berkovic and Yakubu, Berkovic curled a free-kick just wide and Mornar shot over.
Lumana Lua Lua's arrival added to the visitors' firepower and he immediately drove straight at Keller before grabbing what was a deserved equaliser. Lua Lua got the better of Richards from a high ball and hit a the perfect shot through defenders, beyond Keller and in off the post.
It was to be a memorable last 20 minutes though. Pompey were on the attack when we grabbed the lead again as Brown picked up a cleared corner and immediately set Keane on his way.
The Irish star had no-one with him as he picked the ball up on half-way but drove at the defence, into the box, jinked this way and that and somehow managed to hook the ball past Hislop, 3-2, 79 minutes.
The lead lasted for four minutes though. Yakubu picked the ball up in space on Pompey's left and drilled in the perfect cross for Mornar to slide home from close range, 3-3, 83 minutes.
With the visiting section singing 'we're going to win 4-3', a winner seemed inevitable - thankfully it was the right way this time.
With the clock showing 88 minutes, Davies swung in a deep corner, Richards headed back and Poyet 'kneed' home from six yards. Cue pandemonium on the pitch and in the stands - what price 4-3 at Charlton?
MINUTE BY MINUTE GUIDE
4mins: First effort to Pompey after bright start by Spurs, Yakubu onto Berkovic pass in box, shot parried to safety by Keller.
12mins: GOAL. Play broken up in midfield, King found Keane who in turn spotted Defoe pull away right-side of area, control and instant shot low across Hislop into bottom corner from 16 yards. 1-0.
19mins: Break after important Richards tackle on Yakubu, King and Brown involved, Davies run and curler into Hislop's arms.
24mins: SAVE. Quick-passing move, Brown released Davies, through on goal but Hislop out quickly to block with legs.
26mins: PENALTY? Curtis all over Brown as they contested high ball but claims waved away.
33mins: CLOSE. Quashie 30-yarder drifted inches wide.
37mins: Keane turn in box, shot blocked by Curtis.
38mins: Jackson intercepted pass, run and shot from 25 yards always too high.
39mins: GOAL. Bizarre reply. Yakubu broke offside, Carr's attempted clearance from low cross struck Berkovic and sailed home. 1-1.
41mins: GOAL. Ball into Keane, dummy from Defoe, ball looped up, Keane changed body shape to spin and crack home left-foot shot. 2-1.
45mins: Hislop did well to hold onto Dalmat snap shot.
Half-time: Spurs 2-1 Pompey
Half-time: SUB. Mornar for Sheringham.
47mins: CLOSE. Berkovic curler from 20-yard free-kick a foot wide.
56mins: Keller decent save low to his left from Yakubu effort.
60mins: YELLOW. Faye for foul on Dalmat.
63mins: Mornar curled over after Yakubu wriggled away from Richards and Jackson.
63mins: SUB. Lua Lua for Faye.
66mins: SAVE. Berkovic clever one-two with Yakubu in box, attempted to place ball into corner but Keller read it and made save.
67mins: Lua Lua low shot straight at Keller.
71mins: YELLOW. Dalmat for foul on Yakubu.
71mins: SUB. Ricketts for Dalmat.
72mins: GOAL. Decent strike but poor goal to concede. Lua Lua onto nothing high ball, control and curler in off post from 20 yards. 2-2.
74mins: Brown drive stung Hislop's hands.
79mins: GOAL. Brown onto clearance, fed Keane, on his own on half-way, sprint into box, held off three defenders, beat Hislop. 3-2.
81mins: SUB. Poyet for Defoe.
83mins: GOAL. Yakubu in space down left side, pinpoint cross slid in by Mornar. 3-3.
84mins: SUB. Stone for Berkovic.
88mins:GOAL. Davies deep corner headed back into danger zone by Richards, Poyet finish from 10 yards. 4-3.
Injury time: Brown curler tipped away by Hislop at full stretch.
Line-ups: Spurs: Keller, Carr, Jackson, Gardner, Richards, Davies, Brown, King, Dalmat (Ricketts, 72), Keane, Defoe (Poyet, 81). Subs: Burch, Doherty, Yeates.
Portsmouth: Hislop, Primus, De Zeeuw, Sheringham (Mornar, 45), Quashie, Taylor, Faye (Lua Lua, 63), Yakubu, Smertin, Curtis, Berkovic (Stone, 84). Subs: Wapenaar, Berger.
Referee: P Walton.