Dempsey played ninety minutes and scored an 80th minute consolation penalty as a crowd of 27720 at the FirstEnergy Stadium witnessed the Belgians take advantage of poor American defending.
Everton's Kevin Mirallas opened the scoring for Belgium in the 6th minute with a deft chip before Geoff Cameron equalised midway through the second half, heading home a Dempsey assist.
Benteke was introduced for Dembele in the 41st minute and his goal shortly after the start of the second half began a Belgian onslaught, with the Aston Villa player and Marouane Fellaini both netting before Dempsey's late penalty.
Jermain Defoe was introduced as a 33rd minute substitute in place of Daniel Sturridge as England drew 1-1 with the Republic of Ireland at Wembley.
A fantastic header from Shane Long gave the Irish the lead before Frank Lampard bundled home an equaliser in the 23rd minute as both sides prepare for vital World Cup qualification matches later this year.