The young striker levelled at 1-1 just before half-time and then scrambled home a second for 2-1 10 minutes into the second half.
But the impressive Saints hit back again to level on the hour before Sam Hoskins grabbed the winner five minutes from time.
Adam Smith was back in the XI's starting line-up and the full-back was heavily involved as Tim Sherwood's team made the early running.
Smith forced the first save of the game from Steve Mowthorpe in the 12th minute and then crossed for Coulibaly to head wide from 12 yards.
The visitors took the lead when Lloyd Isgrove was upended by Jonathan Miles and Hoskins slotted home from the spot.
Coulibaly was sent clear by Massimo Luongo a minute before the break and made no mistake one-on-one. Alex Pritchard curled just wide with the final kick of the half.
The lads went ahead in the 55th minute when Bongani Khumalo rose well to head down Pritchard's corner and Coulibaly was there to turn the ball home from close range.
The lead didn't last long though and Saints enjoyed a slice of fortune as James Ward-Prowse miss-hit his shot but the ball fell at the feet of Callum Chambers, who fired into the top corner.
Luongo lashed wide and then combined with Pritchard to set-up Lee Angol, but the striker was closed down before getting his shot away.
The Saints won it with a classic counter-attack. Isgrove took off down the right and crossed for Hoskins to drill home from 10 yards.
Spurs XI: Miles; Smith, Uvini, Khumalo, Byrne; Pritchard, Luongo, Veljkovic, Gomelt (Gallifuoco, 46); Coulibaly, Lancaster (Angol, 60). Unused sub: Waller Lassen.
Goals: Spurs - Coulibaly (44, 55); Southampton - Hoskins (20, penalty; 85), Chambers (60).