Jamie Redknapp's stunning strike and two wonderful saves from Kasey Keller separated the sides.
Taylor said: "It would be easy to say we were unfortunate, which we were, we didn't make things happen and we're left very frustrated."
However, Taylor insisted Christian Ziege should been sent off rather than booked for hauling down Darius Vassell in full flight in the second half - even though it was fully 40 yards from goal and we had plenty of defensive cover.
Taylor added: "If Ziege had clipped Darius' heels with a foot I'm sure he would have been sent off and I can't see the difference between that and pulling him down with his hands.
"Darius was too far from goal to call it a goal-saving foul but it was serious foul play for which refs can show a red card. I don't like to see players go off and I don't want to put too much emphasis on this as the reason why we lost, but it was one of the things that upset our day.
"As for the goal, I believe our keeper would have saved Redknapp's shot if he'd not been hurt. By the time he got up and cleared his head the ball was past him."
Aston Villa boss Graham Taylor bemoaned his side's luck after our 1-0 Premiership victory at the Lane on Saturday.