The Japanese Akita dog, which was in the garden where the waste was alight, jumped at the firefighter, knocking him down and biting him on the back.
Police are investigating the attack at around 5pm on Saturday outside a house off Hill Road, Tividale.
The firefighter, who does not wish to be named, managed to scramble away moments later as the dog walked away.
But he was left with grazes and cuts to his back and right upper and lower arm.
"It was a bit of a shock as it is was a big dog," said the 31-year-old male firefighter who had to be treated in hospital. "The dog jumped on me and I fell to the floor where it bit me on the back and arm. The police are investigating so I can't say much more."
West Midlands Fire Service's commander for Sandwell and Dudley, Neil Griffiths, tweeted a picture of the firefighter's injuries after he was taken for treatment at Sandwell Hospital.
Firefighters from Oldbury station attended the fire started in a small pile of garden waste to the rear of the home.
They fought the flames from a neighbouring garden. The dog was in the garden where the fire was taking place.
It managed to to walk out of the garden, and as the fire crew tried to coax it back inside it attacked the firefighter.
The firefighter was back on duty on Sunday.