After dominating for lengthy spells and going all out to win the game, James Beattie stole in behind Dean Richards to score the home side's late winner.
"We thought we were having the better of the game and couldn't really see them scoring," commented Ledley. "That is what the Premiership is all about though - it is so tight.
"One goal can change the game."
Beattie is certainly one to watch when we return to St Mary's in the FA Cup on Saturday.
"He is handful and has scored a lot of goals this season. I thought we handled him quite well, but it just goes to show because he popped up at the end there and won it for them," added Ledley, who says the playing conditions looked a lot worse then they actually were.
"When it started raining it was quite difficult then. In the second half it was alright to be honest and I think the pitch looked worse than it was. It wasn't too bad, Newcastle the other day was worse."
Ledley King feels that the unfortunate defeat at Southampton sums up the unpredictable nature of the Premiership.