The Athletics designated the right-hander for assignment before Thursday's game with the Houston Astros.
''It started out bad for him and he just couldn't get into a spot where he could find himself and not have to think about some of the failures or whatever that were going on,'' A's manager Bob Melvin said. ''I feel awful about it, too, because it's my job to get him in a position to succeed and it just didn't happen here.''
Johnson, acquired in a trade with the Baltimore Orioles, was 4-2 with a 6.92 ERA and two saves in 38 games with the A's.
Brought in to replace Grant Belfour as the team's closer, Johnson faltered early, losing his first two outings before losing his job.
''I kept feeling like it was only a matter of time before he was going to right the ship,'' A's closer Sean Doolittle said. ''You wouldn't have known that he was struggling based on the way he was going about his business every day.''
Johnson allowed four runs on three hits without retiring a batter in Wednesday night's 9-7 victory over the Astros.
''Everybody in this clubhouse knows how good he can be,'' Doolittle said. ''He saved 101 games in two seasons in the toughest division in baseball. I hope for his sake a change of scenery does him good and he can latch on somewhere else and land on his feet.''
Johnson led the AL in saves in each of the past two seasons, with 50 last year and 51 in 2012.
Evan Scribner was recalled to replace Johnson on the 25-man roster.