The school announced the moves Wednesday.
Smith has been part of two conference titles and six bowl appearances in two stints with the Pirates, most recently under fired former coach Ruffin McNeill.
Montgomery hired former Vanderbilt associate head coach and linebackers coach Kenwick Thompson as defensive coordinator, as well as former South Carolina defensive line coach Deke Adams to the same position. Montgomery also hired Miami Dolphins receivers coach Phil McGeoghan to coach the Pirates' pass-catchers.
The school also said Terrell Smith will serve as assistant athletic director for football operations.