Angel Nunez scored 16 points to lead USF (1-4) and Jaleel Cousins added 13 points and 12 rebounds. Perry had eight points.
The Great Danes (2-3) were led by Evan Singletary with 14 points.
Albany took a 61-60 lead with 1:34 left on Peter Hooley's layup after a steal.
Nunez missed a corner 3-pointer on the ensuing possession. Roddy Peters also missed a 3-pointer that would have given the Bulls the lead with less than 30 seconds left, but the ball deflected out of bounds off of an Albany player.
USF inbounded the ball with 23 seconds left and Peters' pass appeared to result in another turnover, but the deflected ball found Perry inside for the go-ahead score.
Albany raced up court, but Peters picked off a pass and the ball eventually found Jahmal McMurray who was fouled with 2.7 seconds left. McMurray made the first free throw and missed the second for a 63-61 lead.
Singletary's half-court heave missed.
USF shot 44.1 percent (26 of 59) from the field while holding Albany to 36.5 percent (19 of 52).
Both teams struggled from behind the arc with USF making just 1 of 11 and Albany 3 of 20.
Albany trailed early before going on a 16-2 run to take a 27-20 lead. USF battled back in the final five minutes of the first half to tie the game at the break at 33-all.
USF: With the win, USF avoided tying the school record for most losses to start a season as the 1987-88 team started 0-5.
Albany: Last season Albany went 13-2 when scoring in the 60s, but is 0-3 this season.
USF faces Kentucky in Miami on Friday.
Albany hosts NJIT on Friday.