The Toronto Sun confirmed the report and said that the team's management has suggested "staff changes."
The Sun also suggested that the media was made aware of Casey's return to the team before Casey was made aware of it himself.
Toronto finished the regular season with a 49-33 record, good enough to finish in first place in the Atlantic Division and earn the Eastern Conference's fourth seed.
But the team's 2014-15 season ended in a first-round playoff sweep at the hands of the Washington Wizards.
Toronto started the season with a 24-7 record before going 25-26 to finish the year. The team has not made it out of the first round of the playoffs since the 2000-01 season, and is 1-6 all time in NBA playoff series.
Casey, an assistant coach on the 2011 Dallas Mavericks championship team, has coached the Raptors since the 2011-12 season, amassing a 154-158 record with the team during that time span.
Casey also coached the Minnesota Timberwolves during parts of the 2005-06 and 2006-07 seasons.