Boro ran out 3-1 winners thanks to a Joe Cuff double and Callum Coyle strike in the third qualifying round.
Hereford were seen as the favourites as they play in National League North and took the lead through Tom Owen-Evans in the 15th minute.
But Rangers, who play a level below in the Northern Premier, soon levelled, with Cuff firing home.
The winger then doubled his and his side’s tally following a tense period in the second half – finding the bottom corner from the edge of the box – before Coyle rounded the keeper and slotted home in added-on time.
Jody Banim’s charges have bagged £5,625 in prize money.
Halesowen Town were knocked out as they suffered a 2-0 defeat at Cambridge City.
Paul Smith was missing several key players through injury and fell to goals from Mikey Davis and Jarid Robson.
A disappointing night for the Yeltz, but they will still receive £1,875.
Meanwhile, Hednesford Town have signed Tayor Allen on loan from Forest Green Rovers and added Graham Deakin to the coaching team.