Orlando fired Van Gundy in 2012 after he coached the team for five seasons. The Magic made the playoffs in each season of Van Gundy's tenure.
In May, the Detroit Pistons hired Van Gundy as their coach and president of basketball operations.
Van Gundy made his remarks about the Magic in the wake of the team's recent decision to fire Jacque Vaughn.
Under Vaughn, Orlando failed to make the playoffs in 2012-13 and 2013-14 and has fallen out of the race this season.
"All those things are the same," Van Gundy said today. "You get in the press conference, they loved Jacque. He did a great job. You listen to all that, and then I listen to Rob say it doesn't fall on Jacque, but he takes full responsibility. I laugh, obviously, because that's not true.
"You're still sitting there with a job, and Jacque doesn't have one, so I don't know what full responsibility means. But I would say they left the full responsibility on Jacque."
Van Gundy also said of the Magic: "They're an organization that turns over coaches a lot. I don't know if anybody coached there longer than I did. There's no such thing as a longtime Magic coach."