Sandberg was promoted on an interim basis in August 2013 after the firing of longtime manager Charlie Manuel.
Philadelphia removed the interim tag off Sandberg in September of that year and announced he would return the next season on a three-year contract.
After going 20-22 under Sandberg in 2013, the Phillies posted a 73-89 record the next season, good for fourth in the National League East division.
This season Philadelphia started 26-48 and had dropped 15 of its last 20 games. Third-base coach Pete Mackanin will replace Sandberg on an interim basis.
The Phillies open a three-game series against the Washington Nationals on Friday at Citizens Bank Park.
• POWER RANKINGS: Cardinals take first for fourth straight week