The car was abandoned in Wideacre Drive at around 9pm on Saturday.
Photos show yellow paint had been used to used to draw the Nazi swastika symbol and 'EDL' on the car.
EDL is an abbreviation for the English Defence League, a far-right anti-Muslim group that has held demonstrations in Walsall and used to be led by Tommy Robinson.
West Midlands Police officers have seized the car and are investigating it as a hate crime.
A spokesman for the force said: "We are investigating reports of a hate crime after a car, vandalised with offensive graffiti, was dumped on Wideacre Drive, Birmingham at 9pm on Saturday.
"The car has been seized and enquiries are ongoing. Neighbourhood officers will be patrolling the area today [Sunday] to offer reassurance to local residents.
"Anyone with information can contact us."
Witnesses can call West Midlands Police on 101, quoting reference number 20BW/181434D/20.
Alternatively, call Crimestoppers anonymously on 0800 555111.