The incident came at the end of the first half with us trailing 1-0 to Ryan Giggs' goal on the half-hour.
Robbie Keane threaded the ball through to his livewire strike partner who seemed clear of Brown but was brought down and the ball went through safely to Roy Carroll in goal.
Referee Dermot Gallagher waved play on though and in what was always an even game in terms of possession, United went on to win 3-0 with late goals from substitutes Ronaldo and David Bellion.
Acting boss David claimed afterwards: "There was a challenge from Brown on Defoe that was a pivotal point of the game.
"That might have been a bad moment for United. I'm not saying that he should have got sent off but I definitely thought he didn't get the ball. It was a major point.
"Two or three little things also went unpunished as well, like shirt pulling, in and around the box.
"I thought if we started well there was always a chance of something but often when you come to places like this, you need to get the first goal. That is very important.
"It was a very tight game until Giggs scored, nothing in it and there were long periods after that where I thought we might have got something."
David Pleat described an unpunished Wes Brown tackle as Jermain Defoe attempted to burst through on goal as a 'pivotal point' of Saturday's defeat at Old Trafford.