The defender - nicknamed 'Steo' - wore the armband throughout most of the team's Irish record of 13 matches unbeaten.
He will be there again when the Republic line-up for a crucial European U21 Championship qualifier in Cyrpus on Friday night.
"The captaincy gives you a bit more responsibility and when new players come into the squad they tend to look up to you," said Stephen.
"It's a good challenge and a good experience. It will probably help me to become more vocal in my game. I've really enjoyed it. There is obviously pressure but it's been a fantastic run. Hopefully I'll lead by example and we'll get a good win in Israel."
As well as the record run, Stephen's group also won the prestigious Madeira tournament last February — Ireland’s only tournament victory at this level — and they are the only ever Irish side to qualify for the the UEFA Under 16 and Under 19 Championships as well as the World Youth Championships.
But they now face a battle to reach the European finals.
After starting the qualifying campaign with a win against Israel, Ireland have lost in Switzerland and France and face an uphill task to make it into the group's top two.
"Israel will be a tough game," added Steo. "They are getting better and it will be hard going over there and getting a result but we really need to.
"It's a big game for us having not won the last couple of games, we need to go and get results really.
"We've done really well over the last few years basically until this competition. We won our first game against Cyprus but then lost against France and Switzerland. It's been difficult. Both of those games were away and France scored late, that was tough to take.
"If we reverse the results at home we can go on to progress to the finals and we need to be there.
"The team has come through together and qualified for every competition through the age groups - European finals, Worlds, won a competition in Madiera where we beat Portugal and Italy and also put together a record of 13 games unbeaten. We have a good team but we need to click again, go to Israel and get a result."
Stephen Kelly feels he's grown as a player since being handed the captaincy of Republic of Ireland Under-21s.