Martin gave UMES a 64-61 lead with 2:23 left and Devon Walker added two free throws with 33 seconds remaining for a 66-61 advantage.
Maryland-Eastern Shore (17-13, 10-5) remains in third place in the Mid-Eastern Athletic Conference standings behind North Carolina Central and Norfolk State. The Shore Hawks close out the regular season against Morgan State on Thursday.
Bell scored eight consecutive Hornets' points, with two 3-pointers and two free throws, to give Delaware State a 56-51 lead. Then UMES answered with a 9-0 run, with scoring by four different players, for a 60-56 lead.