During four years at Celtic, Caldwell helped the club win two league championships, the 2006-07 Scottish Cup and the 2008-09 Scottish League Cup, although he was subject to criticism from Celtic fans. Caldwell himself later said that this was due to injuries in his first season, and then having to play out of position at right back during the early part of the 2007-08 season.
Caldwell subsequently gained recognition and respect from the fans, becoming something of a cult hero, as shown by the chants of “heid” when he was in possession of the football. After being named as the Writers’ Player of the Year for the 2008-09 season, Caldwell stated his belief that this was due to him having a long run in his preferred position, and improved concentration and decision making on his part.