It was all looking good for a place in the draw for the next round when Paul Stalteri's first goal for the club put us two goals ahead on 41 minutes after Jermaine Jenas struck on 20 minutes.
Crucially, Leicester snatched a goal back before the break and seized much of the initiative from then on, levelling on 57 minutes before securing a place in the hat in the final moments.
"To be knocked out of the FA Cup in the third round is very disappointing," said Robbie afterwards. "Them getting a goal before half-time put us on the back foot a bit in the second half and we lost the goals through our own mistakes.
Take nothing away from Leicester though, full credit to them, they played well and took the game to us
We wish them well in the next round."
In what was not a full Walkers Stadium, the away section was packed to the rafters and the skipper for the night concedes the display and outcome did not match the fine support.
"We would obviously like to thank the fans for coming again in their thousands," he added. "Us getting the result we did for them is very disappointing as we wanted to put on a big performance.
"Be assured, the lads are all devastated. It is not nice to be out of the cup, but we must move on now."
Robbie Keane did not attempt to conceal his hurt over our exit from the FA Cup at the third round stage to Leicester City on Sunday evening.