Maodo Lo was 4 of 6 from 3-point range with 20 points and the Lions (5-2) were 10 of 22 as a team from long range.
Bucknell (3-6) fought back to a six-point halftime deficit after Columbia opened the game on a 14-2 run.
Chris McComber opened the second half with back-to-back 3s and Lo added a layup to give the Lions a 32-18 lead. But Bucknell stayed within striking distance until Osetkowski's three-point play and McComber's 3-pointer made it 52-33 with 6:30 to go.
Chris Hass had nine points to lead the Bison who shot 15 of 53 overall and 2 of 15 from deep.
Columbia will visit No. 1 Kentucky on Wednesday.