Leading 3-0 at half-time and seemingly in control, we appeared to have the game wrapped up, particularly after Joey Barton was sent off as the players left the pitch at the interval for a second yellow card.
But City battled back despite their disadvantage in terms of personnel and goals and scored four unanswered goals in the second half, the winner coming in stoppage time at the end of the game.
We made a great start to the match, opening the scoring after just two minutes when Ledley King drilled home a fine left-foot shot past Arni Arason, making his debut in the City goal. It was King's first of the season, his fourth goal in our colours.
Our lead was doubled on 18 minutes when Stephen Carr floated a wonderful ball over the defence and into Robbie Keane's path, Keane's control was immaculate and he neatly finished past the debutant stopper.
Both sides lost one of their strikers to injury, as Helder Postiga was replaced by Gus Poyet and Nicolas Anelka made way for Jon Macken.
Then, just before half-time, Barton's foul on Michael Brown presented Christian Ziege with exactly the free-kick opportunity he thrives on and our German defender curled a stunning effort home from 25 yards.
As the half-time whistle blew Barton, who had just been booked for his foul, received another yellow card from Rob Styles for dissent, and didn't reappear for the second half.
His team never missed him though. Just three minutes into the half, Michael Tarnat dinked a free-kick into our area and Sylvain Distin ran on to the ball, heading it past Kasey Keller.
On 55 minutes, Ziege had another free-kick opportunity in an almost identical position, but this time Arason made a world-class save to tip the ball onto the crossbar and when Keane headed the rebound into an apparently empty net, the City keeper appeared from nowhere to save his effort on the line.
It was to be a pivotal moment in the tie, City suddenly feeling their luck was changing and that was certainly the case for their second goal on 61 minutes.
A defensive clearance fell to Paul Bosvelt on the edge of the area, he hit his effort into the ground but it bounced up, hit Anthony Gardner on the leg and looped up over Keller into the net.
Arason was there again to deny us on 76 minutes, diving full-length to keep out Poyet's header and four minutes later City were level. Robbie Fowler sent Shaun Wright-Phillips free down the right and the young winger beat Keller to the ball and lifted it over our American stopper for 3-3.
As the game entered its dying seconds, we were dealt a final, crushing blow. The ball ran wide to Tarnat on the left, his deep cross was met by Macken and his header back across goal beat Keller. An unbelievable end to an unbelievable game.
Team news: Dean Richards replaces Gary Doherty from Saturday's team at Fulham.
MINUTE BY MINUTE GUIDE
1min GOAL. Stunning start, Ledley King driving home from edge of the box. 1-0.
8mins. Sub. Gus Poyet replaces injured Helder Postiga.
13mins. Keane one-two with Davies, 20-yard shot saved by Arason.
13mins. Straight up the other end, Fowler's dipping half volley just wide.
15mins. Free kick from Keane 25-yards out, saved comfortably by Arason.
16mins. Fowler burst into box down left-hand side but shot across the face of the goal.
18mins. GOAL. Carr's great ball over top of defence, Keane with sublime control before lifting ball over Arason. 2-0
27mins. Sub. Macken on for Anelka for City.
29mins. Wright-Phillips clear down right, cut back for Fowler in space but he didn't connect properly and ball bobbled through to Keller.
31mins. Poyet break through the middle, slid ball wide to Dalmat but right-foot shot easily saved.
41mins. YELLOW. Barton for foul on Brown.
42mins. GOAL. From resulting free-kick, Ziege curls 25-yard effort into the net. 3-0.
Half-time Spurs 3 City 0. RED CARD. Barton sent off as players came off pitch, second yellow card for dissent.
47mins GOAL. Tarnat's dinked free-kick found Distin who headed past Keller. 3-1.
52mins. Keane put in Davies, clean through down the middle, early shot straight at Arason.
55mins. SAVE. Breathtaking double save by Arason, first to tip Ziege's 25-yard free-kick onto the bar and then to deny Keane's header from the rebound.
59mins. Sub. Jackson replaces Ziege. King moves to left-back, Jacko left midfield.
60mins GOAL. Bosvelt shot hits Gardner's thigh and loops over Keller. 3-2.
69mins. Keane flick set up Poyet on the turn and volley, deflected for a corner, just wide.
76mins. SAVE. Arason full-length save low-down to his right from Poyet's header
77mins. YELLOW. Bosvelt for foul. From free-kick , Jackson sends effort just wide from 25 yards out.
79mins. GOAL. Fowler puts Wright-Phillips in, he beats Keller to ball and lifts over the keeper. 3-3.
80mins. Subs. Sibierski for Bosvelt, McManaman for Sinclair.
84mins. YELLOW. Sun Jihai for foul on Jackson.
87mins. Keane twists and turns in box but weak shot at Arason.
90mins GOAL. Tarnat cross headed in by Macken. 3-4
Final score: Spurs 3 City 4
Line-ups: Spurs: Keller, Carr, Ziege, Richards, Gardner, Dalmat, Brown, King, Davies, Postiga, Keane. Subs: Burch, Yeates, Kelly, Jackson, Poyet.
City: Arason, Sun Jihai, Tarnat, Distin, Dunne, Sinclair, Bosvelt, Barton, Wright-Phillips, Fowler, Anelka. Subs: Stuhr-Ellegaard, Sibierski, Macken, McManaman, Jordan.
Referee: Rob Styles
Disbelief, shock and disappointment don't even begin to sum up the feelings after this FA Cup Fourth Round replay, which must surely go down as one of the most incredible in the competition's long history.