Goals against have been a rarity so far this campaign - just four in nine games after Monday's 1-0 loss - and the French full-back insists the defence get pretty upset at conceding after such a good start.
Yakubu was on hand to nod in from close range after fine work from Lumana Lualua for Pompey's winner in the second half.
Noe, a recent match-winner at Everton, admitted the lads were disappointed with it. "Yes, it was disappointing to concede a goal," reflected Noe at the training ground.
"My mind is asking me what I did wrong to give a goal away.
"I don't remember the action and I will look at it again. We have to do that and make sure if there are any mistakes, we don't do it again."
It wasn't the best of returns to Fratton Park for Noe having spent most of the 1999-2000 season at the south-coast club.
"It was a disapointing game for us because we gave Portsmouth opportunities to score and that's why they won in the end," he added.
"We had a couple of chances and we felt if we'd have scored, we could have won the game. But football is like that.
"It will be nice to play at home against Bolton on Saturday and we have to play well, not be worried about Bolton, not make any mistakes and it will be good for the confidence if we can score first."
Noe Pamarot will watch Portmouth's goal over and over again - part of his own routine to help keep us tight as possible defensively this season.