Driphus Jackson passed for 248 yards and two touchdowns for the Owls (2-3).
Dillard's first touchdown, a 2-yarder, tied the score with 19 seconds remaining in the third quarter. He scored again from 4 yards out with 4:18 left in the game, one play after Jackson completed a 50-yarder to Mario Hull. Jackson capped the win with 12 seconds left on a 31-yard scoring pass to Jordan Taylor, who had 103 yards receiving. Jackson had an 81-yard completion to James Mayden for a first-half score.
The Owls sacked Woolsey seven times, the most for Rice since 2004.
The Rainbow Warriors (1-4) lost their 15th straight road game.