The goalkeeper - in for injured Antti Niemi - produced a series of fine saves to deny Jermain Defoe, Mido, Michael Brown and Fredi Kanoute.
Southampton also had plenty of chances and won it thanks to Nigel Quashie's header in the second half.
"Paul gave a terrific performance and he's been marvellous for us," said Redknapp. "We lost Niemi who is as good as anything in the Premiership, so it makes you fear the worst when you pick a boy who has never been in the first team.
"But Paul has been absolutely fantastic. He's played so well that he's given me a dilemma for when Niemi returns. At least I know I don't have to rush Niemi back now."
Redknapp said he believed his side would beat the drop after this win.
"Myself and the players genuinely believe we can stay up. Four or five weeks ago I thought we had no chance, but then the transfer window opened, we brought five players in and we look a different side.
"The new players have made a big difference to us. When I arrived I was thinking how am I going to get this team out of trouble? But we're a much better side now. We're in good form - the team is playing well and working hard.
"If I'd had this team at the start of the season I reckon we would have finished around 11th place."
Harry Redknapp rightly praised his goalkeeper Paul Smith after Saints 1-0 win in Saturday's encounter at St Mary's.