They were just seven minutes away from beating a Women's Super League team for the first time in the FA Cup.
However, having led twice against WSL 2 side Aston Villa, they conceded in the 83rd minute and then lost out in extra time at Cheshunt FC.
Having already reached two cup finals, Sunday represented Spurs' first defeat in cup competition all season.
"I'm so proud of the players, they gave absolutely everything," said Karen. "We led twice and it's just unfortunate we couldn't hold on.
"We just lost to a bit of naivety, giving away a cheap penalty in extra time. That was the only difference. We took the game to them, more than matched them and it's a positive step forward for us - we proved to ourselves that we can compete with a WSL 2 team."
Spurs will have no time to rest on their cup laurels - they are back in knockout action on Sunday with another huge game, this time against high-flying Brighton in the quarter-final of the FA Women's Premier League Cup at Cheshunt FC.
SSE WOMEN'S FA CUP
SPURS LADIES 2-3 ASTON VILLA (AET)
Report from Khalid Karimullah at Cheshunt FC
Tottenham Hotspur Ladies fell short in their FA Cup 4th Round tie with Women's Super League 2 outfit Aston Villa, in a valiant 3-2 defeat.
Despite the task ahead of Karen Hills' side, Spurs came out flying. With the first attack of the game, Bianca Baptiste fired a spectacular lob over Jess Myers in the Aston Villa goal with less than two minutes on the clock. With a constant barrage of attacks continuing to arrive, Baptiste, Riana Soobadoo, Avilla Bergin and Wendy Martin combined with a flowing move, before the latter's cross was just cleared from danger.
Goalkeeper Toni-Anne Wayne then had to be at her best as she pushed Katie Wilkinson's header from just six yards onto the post, before just minutes later, the tie was level. Beth Merrick played a cutting pass into the path of Destiny Toussaint, whose effort found its way over the line despite Wayne getting a touch on the ball, 29 minutes on the clock.
Just before half-time, Wilkinson and Merrick combined well with a smart set-piece forcing Wayne to react brilliantly.
Into the second half and Mollie Rouse fired over from just outside the area for the visitors.
But it was Spurs who would soon lead again. Leah Rawle played a lovely pass into the path of Baptiste, who broke away out of the left channel before seeing her lifted effort gravitate again over Myers to restore the hosts' lead on 64 minutes.
Captain Jenna Schillaci, who led by example alongside Alex Keown at the back, made a goal-saving block from Wilkinson's curled effort, but with just seven minutes remaining, an in-swinging corner by Merricks arrived at the head of Rouse who made no mistake from close range.
With extra-time looming, Toussaint drifted into the area and forced a last-second stop from Wayne.
The tie ended 1-1 at full-time and therefore went into the extra 30 minutes.
With almost 100 minutes played, Martin saw her deft touch slide past Myers, but unfortunately not the woodwork. The same player then found Bergin, whose effort at the near post was inches wide.
As the game continued to be end-to-end, Wilkinson won a contentious penalty before stepping forward herself to give Villa the lead for the first time in the spectacle, nine minutes from time.
With the last significant action of the game, Baptiste's pass came to Lucia Leon who had her powerful drive desperately blocked.
Spurs Ladies: Toni-Anne Wayne, Lucia Leon, Alex Keown, Jenna Schillaci (c), Leah Rawle (Rosie Kmita 101), Sophie McLean, Josie Green (Shannon Moloney 60), Riana Soobadoo (Amy Cooper 77), Wendy Martin, Avilla Bergin, Bianca Baptiste. Substitute (not used): Mollie Kmita.
Aston Villa Ladies: Jess Myers, Hayley Crackle (Leanne Byrant 67), Hannah George (Amy West 76), Jade Richards, Maddy Cusack, Katie Wilkinson, Katy Moran (c), Elisha Nidow, Destiny Toussaint, Bethan Merrick, Mollie Rouse. Substitutes (not used): Chloe Beattie.
Spurs Ladies Player of the Match - Bianca Baptiste.