Robson also claimed that the handball decision that led to Jermain Defoe crashing home our second goal was also very harsh.
Russell Hoult was penalised after a collision with Stephen Kelly - replays later showed that Stephen got there first, Hoult touched the ball and then his momentum took the full-back out. It also took him out - he had to be replaced after gashing his eye.
Robbie Keane scored the penalty to make it 1-1 after West Brom had dominated the first half but it all changed after the break thanks to a quick two-goal blast from Jermain as we ran out 3-1 winners.
Robson said: "The first goal was never a penalty. Russell got a touch on the ball. It was a decent save and the TV replays prove that.
"It was also a real harsh decision to give handball against Paul Robinson. Paul says it never hit his arm but even if it had from the distance that Fredi Kanoute was kicking the ball against him, it was a harsh decision to award a free-kick.
"It's annoying because we were playing well and with a lot of confidence. Tottenham never looked a threat but the penalty gave them good encouragement for the second half.
"For me the major turning point of the game is when the referee gave them a penalty that never should have been.
"We outplayed Tottenham in the first half and that encourages our lads to keep it going but suddenly you go in at half-time at 1-1. That gets their fans up and encourages the Tottenham players."
West Brom boss Bryan Robson was in little doubt that key decisions went against his side in Saturday's FA Cup replay.