Last week 100 plus caravans moved to the open space between Schofield Avenue and Wyntor Lane, in West Bromwich.
They were issued with a section 61 ordering them to move. So they headed to nearby Ball's Hill play area, between Chester Road and Brindley Road.
Resident Dawn Bennett said: "I spent years volunteering as part of Millfields Green Spaces to get the children's play area created there.
"We raised thousands of pounds, including getting funding from the lottery, and it is all going to be ruined. It is just bedlam."
In response to the repeated calls about anti-social behaviour, West Midlands Police stepped up patrols of the area.
The fear of disorder also led to an annual fund-raising for Birmingham Children's Hospital at a nearby pub being scaled back. The group is looking for a new date for the event.
A petition to get permanent barriers erected round the open space to protect it from travellers has so far gained 300 signatures.