Six fire crews are at the scene in Erdington and buses including the number 28 have been diverted.
A West Midlands Fire Service spokesman said: "At 10.40am we started to receive several 999 calls to a building on fire in Erdington, Birmingham.
"Six fire crews responded from a number of city fire stations, with the first arriving four minutes after being mobilised."
The spokesman added: "The building, believed to be unused commercial premises, is near the junction of Sutton New Road and Summer Road.
"Some local roads are closed. People in the area affected by smoke are asked to keep doors and windows closed as a precaution.
"This is an ongoing incident."