The team announced the move after beating the Washington Nationals 4-3 Saturday in their last exhibition game.
Romine is out of options and described his status as ''just kind of limbo.''
Romine batted .182 this spring, while Murphy hit .238.
''I said this was going to be a really tough decision,'' manager Joe Girardi said about picking a backup to Brian McCann. ''Austin was prepared. Worked extremely hard this winter to earn the job. And he had a tough camp. It's hard, because he's been with us a long time and there's feelings for the kid and you want to see him do well, but it's just the nature of the game.''