Anthony Gardner's deflected shot in the 88th-minute - his first for the club - secured the points after a stirring final half-hour saw us take the lead twice, Hammers go down to 10 men but still hit back for 2-2 before the defender's late intervention.
Boss Glenn said it was a huge win, not least after throwing away a 2-0 lead at Fulham on Wednesday night.
He said: "This is a key result for us, a key match. I felt if we lost or even drew it might have been a signal that we would slip back into the next little batch and the carrot was to go back into second.
"At times we did okay but it wasn't a classical display. There is certainly more to come in terms of performance.
"There were some moments of excellent play and we enjoyed the goals but I didn't enjoy the way they scored. Those two goals shouldn't have been conceded in the way they were and we have to improve on that if we are to achieve anything.
"But we've won every game here at White Hart Lane and if we can let that momentum grow and keep churning out the wins and become hard to beat here that will be a great sign."
The three points lifted us back into second, a point behind Arsenal, and maintained our fine start despite the absence of Neil Sullivan, Steve Carr, Steffen Freund, Darren Anderton, Sergei Rebrov, Steve Clemence, Ledley King and Dean Richards.
Glenn added: "We have to make sure we are still in a good position when the likes of Ledley King, Gus Poyet, Darren Anderton, Stevie Carr, numerous players there, eight or nine, all return.
"If we can get them back having maintained the results that will be a great sign for the rest of the season."
Glenn Hoddle described Sunday's late, late 3-2 victory against West Ham as a 'key result'.