Stourbridge had looked the brighter in the early exchanges as striker Jason Cowley saw an audacious long range effort turned behind for a corner in the third minute.
And he went one better just a minute later when he was set free and rounded Hednesford keeper Andy Wycherley to put the Glassboys ahead at the War Memorial Ground.
However, Hednesford were awarded a penalty in the 18th minute which was converted by Courtney Richards.
The sides went into the break level but just seconds after the restart an attack by the Pitmen saw a rash challenge in the home box and another spot-kick awarded.
That too was converted to give the visitors the lead.
Hednesford had the upper hand and looked the better drilled side as they closed out the game for their second victory in their opening two matches of the season.
The Pitmen visit Alvechurch on Saturday while Stourbridge travel to Peterborough Sports.