Andre Nation's basket gave James Madison a 48-44 lead with 39 seconds left. Richmond's Trey Davis hit two free throws to pull within two points, but James Madison's Jackson Kent converted a pair and Ron Curry made 1 of 2 to close out the game.
Kent added nine points and grabbed 10 rebounds to help James Madison (6-2) hold a 39-27 advantage on the glass.
Kendall Anthony led Richmond (3-4) with 16 points.
It was a defensive battle as both teams shot under 40 percent from the field and committed 13 turnovers apiece. Richmond hit seven of its 25 3-pointers and James Madison was just 1 of 13.