Conservative leader Mike Bird said the idea was 'more trouble than its worth' and may lead to 'confrontation' at polling stations.
The Government is looking at running the pilot schemes at local elections next year after recommendations by ex-Local Government Secretary Sir Eric Pickles.
It would see participating councils request ID from voters at polling stations to crackdown on electoral fraud.
But Walsall Council's chief executive Paul Sheehan has raised concerns about the idea.
Among a list of questions, he asked what the protocol would be for checking the ID of Muslim women wearing a traditional Hijab veil.
In a report he said: "We could have people arriving to vote with passports as ID who are wearing Hijabs.
"What protocol would be required so that the polling station staff could be satisfied that the ID was sufficient?"
Councillor Bird said: "We are not taking part. It is a sledgehammer to crack a walnut and to my knowledge we don't have a problem with people voting in other people's name. It is more trouble than it's worth."
He added: "There are women within the Muslim community who will be wearing a Hijab, and if they were asked for ID I can see confrontation at election times. It is an issue, there are several others and this is why we won't take part."
Mr Sheehan also feared that imposing ID restrictions would increase the chances of results being challenged.
He said: "It would have been more practicable to warn people at the 2017 election that ID would be trialled next time they were in the polling station.
"As mentioned previously the pilot is not without significant risk of election petition and damage to the council reputation as a consequence. It is also not clear whether or not there is a significant issue with people impersonating other voters at polling stations in Walsall."
A council spokeswoman added: "The information on electoral pilot schemes is very limited at present.
"Taking part in the pilot appears to carry significant risk to the council in terms of the possibility of electoral petition and subsequent damage to the council's reputation and there is therefore no appetite for Walsall Council to take part in the pilot scheme."