Firefighters were called to Lower City Road Just before 12.30pm on Thursday.
Three fire engines and a brigade response vehicle initially responded, crewed by firefighters from Oldbury, Tipton and Haden Cross fire stations.
A statement from West Midlands Fire Service says: "The building, measuring approx. 50 metres x 20 metres, is well alight. Due to concerns of asbestos in the roofing, crews are fighting the fire externally from a safe distance using main jets.
"One of our hazardous materials advisors has also been mobilised to monitor situation.
"An aerial hydraulic platform has also been brought into use at the rear of the building.
"There are currently no reports of any casualties."
Police are also at the scene and have set up a cordon.
Lower City Road is closed from Dudley Road East to Birmingham New Road.
Fire crews on site have now advised residents in the area that they are safe to open doors and windows as smoke from the incident has ceased, after they were advised to keep them closed earlier in the afternoon.
West Midlands Fire Service have also said that by isolating power to the building involved in the fire, some nearby residential properties and a nearby school lost power temporarily, but Western Power are currently attending the scene and the power is expected to be restored to these properties in the next 3-4 hours.
Lower City Road is also home to Ormiston Sandwell Community Academy.
In a statement the school said: "There is a fire on a building on Lower City Road. The road is currently closed to traffic. The police and fire service are in attendance. Students and staff are safe at OSCA."
Oldbury Ward Councillor, Rizwan Jalil was at the scene earlier this afternoon and said: "I attended the scene as residents called me to say their is a fire and the fumes in the air are very strong.
"When I attended both sides of Lower City Road where closed and the area was controlled by the police and fire service. I spoke to residents in the area and advised them not to stand around as breathing in fumes can cause irritation or swelling in your air passages.
"After 15 minutes I left the scene as I didn't have any breathing apparatus to allow me to stay at the scene. I would like to assure the community that the fire service and police are doing a amazing job tackling the issue.
"If anyone in the area who is affected by the fire needs to talk to me about any help or assistance I will be available over the next few days in my capacity as a councillor.
"At this point I can not confirm how the fire was erected and will wait for the fire service and police to investigate further."