Capitals right wing Joel Ward scored with two seconds remaining in the third period to lift Washington over the Rangers.
The Capitals opened the game's scoring in the first period on a goal by Alex Ovechkin. Neither team scored again until New York's Jesper Fast tied the game up with just under five minutes remaining in regulation.
You can hear Capitals play-by-play announcer John Walton's radio call of the play here.
Right before the goal, Capitals center Niklas Backstrom had a hit on the Rangers' Dan Boyle that allowed Ovechkin to pick up the loose puck.
After the game, Backstrom told reporters he felt the hit was clean.
"In the playoffs you've got to finish checks. That's what I tried to do," he said.
The Capitals lead the series 1-0. Game 2 will be played Saturday in New York.
- Molly Geary