Officers had arrested a 25-year-old man on suspicion of possessing Class A drugs.
Following his arrest, police then carried out a warrant at his address in Sparkbrook on Monday morning where officers discovered a double-barrelled sawn-off shotgun.
He is now being questioned on suspicion of possessing a firearm.
A spokesperson for West Midlands Police said: "Operation Target sees us take a defiant stand against a range of serious and organised crime offences – from drug dealing and burglary to cyber crime and fraud.
"Officers use local intelligence, seize goods, carry out warrants and target offenders as part of Op Target’s ongoing crackdown against serious and organised crime."
