Ramsey's 19-yard touchdown run with 3:32 left in the first half gave Western Carolina (3-1, 1-0 Southern Conference) all the points it would need with a 21-3 lead.
Troy Mitchell was 12 of 19 for 182 yards and added 70 yards rushing on 16 carries. The Catamounts outrushed Furman 233-106 to beat the Paladins (2-3, 1-1) for the first time since 2005 and the first time on the road since 1994.
Detrez Newsome added two touchdown runs for the Catamounts, but fumbled a kickoff return early in the fourth quarter to set up Furman at the 24, trailing 28-17.
The Western Carolina defense held, Jon Croft Hollingsworth missed a 35-yard field goal, and Furman would get no closer.