Just two nights after a lacklustre 5-3 home loss to the New York Islanders that snapped a four-game winning streak and caused a rare post-game rant from coach Tom Renney, the Rangers fell behind 5-0 early in the third period.
Ray Emery and the Senators were cruising when Alfredsson's goal at 1:02 of the third sent many Rangers fans home early.
They missed a lot. This one wasn't settled until Dany Heatley's empty-net goal with 34 seconds left.
Emery, 6-1-1 in his past eight starts, allowed only 13 goals in that span before the Rangers broke out with four in 7:10.
Petr Prucha netted the first, drawing sarcastic cheers at 3:29. Jason Ward followed 45 seconds later, and when Marcel Hossa cut the deficit to 5-3 at 6:20, Madison Square Garden was solidly behind the home team.
Blair Betts' then deflected Jed Ortmeyer's shot past Emery at 10:39 to send fans into a frenzy.
All that work didn't matter in the end as the Senators won their eighth in 10 games (8-1-1).
Patrick Eaves scored the first of his two goals 10:31 in, and Chris Kelly, Antoine Vermette, and Eaves again connected in the second to make it 4-0 and turn the Garden against the Rangers.
The Senators know all about comebacks as they scored five in the third on Tuesday to wipe out Boston's 2-0 lead. Ottawa gave up double-digit shots in each period, and was outshot 14-6 in the final frame.