The Three Lions fought hard but a late goal from Italy's Lorenzo Insigne was the difference between the two sides.
Steven was involved in a moment of controversey just after half-time when he set-up Craig Dawson to head the ball home from close range with the score at 0-0.
The goal was disallowed, much to Caulker's surprise.
"The ball came across and I headed back to Daws to put it in," he said. "As far as I'm concerned it was a legitimate goal, but the referee has obviously seen something different.
"We had a bit of a wild celebration, as this tournament means a lot to us and we all want to win."
The loss ended England's nine game winning streak and our young defender knows that the team must perform better against Norway on Saturday night to give themselves a chance of progressing to the semi-finals.
"We're not kids, we know ourselves that it was not acceptable performance and we can do a lot better than that. We've been training really well, so it came as a bit of a shock, maybe that's the reality check we needed."
"So we are bitterly disappointed, but we'll get together and prepare for Norway.
"It's a must win now. We wanted to get off to a good start against Italy, but we'll find our rhythm and go again."
In Group B, Germany Under-21s captain Lewis Holtby got on the scoresheet with a fantastic strike but his side lost 3-2 to the Netherlands in a thrilling encounter.
The Dutch were 2-0 up at half-time thanks to goals from Adam Maher and Georginio Wijnaldum, but a Sebastien Rudy penalty - awarded after Holtby was brought down by goalkeeper Jeroen Zoet - brought the Germans back into the game.
Lewis then equalised with nine minutes remaining. After receiving a throw-in on the left corner of the area, he span away from his marker, beat another tackle and drilled a low shot into the corner of the net.
But as the clock ticked down, Fer showed great strength to head fome from close-range to secure three points for the Dutch.
Germany play their second group game against Spain on Sunday night.