People living in Whitmore Reans insisted their "voices must be heard" and that they should "be given a reasonable time to take legal advice and put forward our objections" in an open letter addressed to local councillors, who are coming under pressure to back their cause.
MP Eleanor Smith has also waded into the row, saying the views of residents must be considered and that the area should not be "forced to take things no-one else wants".
The city council has earmarked a plot of land on Gorsebrook Road for the site, which will cater for up to 13 traveller families at one time.
Bosses say they must provide a legal site for travellers to pitch up if they are to retain powers to protect 60 public spaces in the city. Environment chief Steve Evans said without it the authority is "not holding up our end of the agreement".
Time is of the essence too, with those powers due to be reviewed at the High Court in December. If it was to lose them it could leave public spaces vulnerable to traveller camps.
But residents say they do not want the travellers site on their doorstep. A petition has also been launched.
In the letter, the residents said: "We have not been given adequate time to consider or to prepare our thoughts on the proposed changes in our community.
"There has been no consultation proposed by the local authority and we feel that the local authority has deliberately given the people insufficient time to avoid our objections to the proposals.
"As a collective our voices must be heard, and we must be given a reasonable time to take legal advice and to put forward our objections to any proposal."
Eleanor Smith, Labour MP for Wolverhampton South West, said: "The Whitmore Reans area of Wolverhampton is one of the most deprived in the city. It deserves good things, not be forced to take the things no one else wants.
"The same goes for the traveller community, it deserves to be respected and consulted, not be offered a former dump.
"The council’s letter to local residents does not come across as the start of a consultation but the final stages of a done deal. The council needs to make sure it listens to residents about how they feel."