John Maddaluna caught five passes for 134 yards.
Colgate (7-4, 6-0 Patriot League), which has won seven of its last eight games, jumped out to a 14-0 halftime advantage but then had to hold on to secure the win.
The Raiders, who have already claimed the 2015 Patriot League trophy, finished with their best conference record since 2003.
Trailing 14-0 at the half Bucknell charged back when Nitti hit Will Carter on a 47-yard scoring strike to close to 14-7 with 11:58 left in the third. Alex Pechin booted a 19-yard field goal early in the fourth for the final score.
Nitti finished with 295 yards passing for Bucknell (4-7, 1-5).