Kessel's 10th goal, which opened the scoring, came on a solo rush as the Leafs forward skated through two Devils in the defensive zone before beating Cory Schneider between the legs at 8:12 of the third period. It ended a Toronto scoreless stretch of almost 110 minutes.
''Those are difference-maker goals,'' Carlyle said of Kessel's swooping rush. ''It reminded me of Frank Mahovlich - the windup, to come back inside your own (blue) line and attack. That's old-time hockey. I mean real old-time hockey.''
Of course the 26-year-old American was born six years after Mahovlich, now 75, entered the Hockey Hall of Fame in 1981.
It looked like Kessel's goal might be enough until Toronto goalie Jonathan Bernier, who had been excellent, seemed to misplay a shot from the right flank from Michael Ryder, with the puck angling high off the goalie's stick into the net with 4:45 remaining in regulation. Ryder got credit for his fifth of the season.
''I'm probably going to do that play a hundred times and, I don't know, if you ask me to do it again I probably wouldn't be able to,'' Bernier lamented.
Kessel was in the box, with 11 second left for a slashing penalty.
''Those thing happens in hockey,'' Carlyle said of Bernier's miscue. ''There wasn't a person in the building who felt as bad as he did about giving it up. You could just see his reaction. But I thought our bench remained composed. We gathered our composure and we didn't sit back, we attacked. And we didn't stop.''
Added Bernier: ''We showed character. We came back and won that game. It was a great team effort.''
Schneider, who also had a strong game, made a great save on Kessel on a Leafs' 3-on-1 in overtime. And he stopped Toronto's Mason Raymond and Kessel before van Riemsdyk's goal in the shootout. Adam Henrique then fired wide for the Devils.
Bernier stopped Ryder in the shootout while Travis Zajac had the Leafs' goalie beat but hit the post.
Toronto has now won five of its last seven. The Leafs were well-rested, having not played since a 4-0 loss in Vancouver last Saturday.
The Devils were coming off a 3-0 win in Philadelphia on Thursday, with Martin Brodeur posting a 22-save shutout - the 122nd of his career. New Jersey had lost its two previous games, both shutouts.
''I'm disappointed we didn't capitalize on our chances,'' Devils coach Peter DeBoer said. ''We got a bounce there at the end to get a point. We had enough chances to win the game. We didn't finish enough.''
The Leafs came into the game fifth in the league on the power play and finished 1 for 6 in man-advantage opportunities.
Van Riemsdyk, whose last duty in the middle was at the University of New Hampshire, failed to get a shot on net but his linemates combined for 12 of Toronto's 28 on the night.
At times the line overelaborated, especially with the Devils in the penalty box. But it paid dividends when it counted.
Bernier, who stopped 41 of 43 shots in Calgary last time out, had to be sharp to keep the Leafs in it as Toronto was outshot 35-28 - the 13th straight game and 14th in 16 this season that the opposition has outshot the Leafs.
Former Devils player David Clarkson almost scored on the Leafs' first shot, cruising in from the slot while protecting the puck from a New Jersey defenseman. His shot got past Schneider but a Devils player poked it away before it could trickle over the line.
Toronto had several glorious chances to go ahead on a power play late in the first period that started with a four-minute high-sticking penalty to New Jersey's Mattias Tedenby and turned into a 44-second 5-on-3 when Peter Harrold was called for tripping.
The Leafs came very close but could not take advantage.
Toronto had a sluggish start to the second period with an overly elaborate Kessel continuing to give the puck away. Bernier had to be sharp on several occasions as Toronto, after outshooting the Devils 8-7 in the first period, was outshot 15-8 in the second.
Devils forward Stephen Gionta was helped off the ice in the second period after sliding awkwardly into the boards.
NOTES: The game marked the first Friday night home contest in Toronto since Dec. 23, 2005 - a 2-1 win over Boston. It also represented Randy Carlyle's 82nd game as Toronto coach. He came in with a 42-31-8 record. ... The Leafs won all three regular-season meetings with the Devils last season.