Brzoja converted 12 of 13 from the line as Evansville attempted 11 more free throws than Fresno State.
D.J. Balentine had 13 for Evansville (3-0) and Egidijus Mockevicius added 10 and 10 rebounds.
Karachi Edo scored 17 points for Fresno State (1-3), Paul Watson had 13 and Julien Lewis added 11.
Watson's three-point play gave Fresno State a 40-38 lead midway through the second half, but the Bulldogs didn't score again for over six minutes.
Evansville's Duane Gibson scored with 50 seconds left for a 53-46 lead. Fresno State's Darnell Taylor responded with a 3-pointer, but Brzoja and Balentine combined to make 5 of 6 free throws to seal it.
Evansville will play Green Bay in the semifinals on Tuesday.