United showed their intent on 14 minutes when Regan Linney was firstly dispossessed by Michael Williams before goalkeeper Lewis King saved low to deny Finlay Sinclair-Smith.
The hosts went close again later when a free-kick was deflected down and the resulting shot was rifled over the bar.
However, slightly against the run of play Rangers took a 26th minute lead with Joe Kenton played through to Muhammed Sebbeh-Njie who fired home.
After going behind United looked for an immediate response and Chris Doyle caused King to make a full stretch save.
King was the saviour again moments later saving from the penalty spot to deny Paul Charles Ennis after a penalty was awarded for handball.
United were level four minutes before the break when a lucky deflection fell to Daniel Cockerline who headed home at the front post.
The hosts eventually made their pressure pay and Adam Dodd crossed to substitute Michael Fowler to nod home the winner.