The Uruguayan midfielder had talked in the week about his dream of playing in his first 'special' derby and for the man who loves scoring goals, his dream came true with a magnificent volley to finally beat the in-form Richard Wright.
It looked like Robert Pires' superb 80th-minute strike was going to grab the points but a never-say-die attitude paid dividends.
We managed another three decent efforts in that frantic last few minutes but Poyet certainly saved the best until last with a chest-high volley that Wright couldn't stop from nestling in the corner.
Defeat would have been harsh after we enjoyed the greater share of both possession and chances but Wright was in fine form and when he was beaten in the first half, Dean Richards' header hit the underside of the crossbar and agonisingly bounced on the line and out.
Sol Campbell apart - and with it, an extra couple of notches on the volume - it was a typical derby.
Sylvain Wiltord had the first attempt on goal, a dribbler comfortbly saved by Neil Sullivan, before a purple patch between 17 and 27 minutes.
Steffen Freund, outstanding in the first half, turned away from Pires and fed Gus Poyet, he spread it wide to Christian Ziege who whipped in a first-time cross towards Les Ferdinand. The striker seemed to get a touch but the ball somehow trickled wide.
Three minutes later Ferdinand got himself in front of Sol Campbell and chested down for Poyet to unleash a fierce half-volley just wide of the upright.
Ferdinand and Keown were then booked for an off-the-ball confrontation and it was dual that had a number of flashpoints - north London's version of WWF, if you like.
Another three minutes on and Arsenal were stretched again as a deep cross from Mauricio Taricco was headed back by Ziege towards Teddy Sheringham but his effort was blocked by a combination of Campbell and Keown.
Dean Richards was then inches away from the opening goal. Ziege swung in a corner and the defender met it perfectly 12 yards out only for it to hit the underside of the bar, onto the line and out to safety.
The rest of the half was scrappy although Ferdinand had the ball in the net from Ziege's cross but Jeff Winter awarded a free-kick for the challenge on Richard Wright.
The second half was no less lively as the visitors tested Sullivan seriously for the first time, a superb diving save to his left to tip Gilles Grimandi's 25-yad strike around the post.
As the temperature rose, Poyet climbed majestically above Keown to header over in the 50th minute, Wright produced a fine save to deflect Poyet's close-range effort wide and Sullivan was equal to another Pires curler.
Sergei Rebrov and Kanu entered the fray for Ferdinand and Berkgamp and the next chance soon followed as Taricco's drive was deflected into the air and, with Wright grounded anticipating saving Taricco's shot, Cole spectacularly cleared.
Then came Pires' moment of magic. Wiltord cut in from the right, toe-poked the ball to Pires and he struck sweetly beyond Sullivan into the corner.
Undeterred, we drove forward. Sheringham's diving header went just wide, Wright somehow kept Richards' powerful header out but, with injury time ticking away, Poyet had the final word.
Gus Poyet carved his own niche in blue and white hearts with an injury time equaliser in this electrically-charged north London derby.