About 15 vehicles moved onto Shelfield Park at around 9pm last night.
A Shelfield resident who has lived in the area for a number of years said it was the first time she was aware of travellers parking up on the site off Parkstone Close.
The woman, who wanted to remain anonymous, said dogs had been left loose to run around the field and nappies had been left on the floor.
Walsall Council said today that its officers were beginning the eviction process under Section 77 of the Criminal Justice Act.
Under the act, local authorities can direct travellers to leave their land but must carry out welfare assessments before serving notice on the group.
The travellers can only be arrested and prosecuted if they fail to leave the area by the date specified by the council in the eviction notice.
Under Section 78 of the same act, councils can apply for an immediate court order to evict travellers but the campers must be given 24 hours to leave before action is taken to remove them.
A spokeswoman for the authority said: "Walsall Council is aware of the recent unauthorised encampment at Shelfield Park, Parkstone Close.
"Community Protection Officers are commencing the process to regain possession of the land using the Section 77 Magistrates process."
It comes as calls for a dedicated area for travellers to pitch up in the borough intensified following a raft of illegal encampments in the past fortnight.
Walsall Council and police have been called out on a number of occasions when groups have arrived on land at various locations in the borough – including New Invention Junior School playing fields, Coppice Farm, Holland Park in Brownhills and the Cairn in Bentley, as well private land in the town centre.
A spokesman for West Midlands Police said: "We are currently working with the local authority and assessing the situation."