Chris Neil had two goals and Ryan Shannon scored the other for the Senators, who snapped a two-game slide.
Brian Rolston had the lone goal while Martin Brodeur stopped 22-of-24 shots for the Devils, who had won seven of eight coming into the game.
“We played decent. We had our chances, we just didn’t score,” Devils head coach Jacques Lemaire said.
New Jersey, which at one point this season was the worst team in the league, has climbed back into the playoff chase, but the loss leaves the team six points back of Buffalo for the final spot.
The only goal of the first period came in the final minute on the power play as Jason Spezza’s blast from the top of the left circle was tipped in by Shannon for a 1-0 Ottawa lead.
McElhinney stopped 10 shots in the first and 11 in the second to keep the Devils off the board.
“It makes it easier to play with a lead,” McElhinney said.
Ottawa made it a 2-0 game 55 seconds into the third when a shot from Erik Karlsson at the right circle popped into the air and Neil banged it into the net from in front.
Rolston made it a one-goal game just over four minutes later on the power play when his blast from the high slot made its way through traffic.
But New Jersey was unable to get any closer and Neil sealed the win with an empty-netter in the final minute.
New Jersey hosts Washington on Friday…Ottawa hosts Tampa Bay on Saturday…The Senators won three of the four games with New Jersey this season…Both teams went 1-for-3 on the power play…Spezza had two assists in the game.