The Campaign to Protect Rural England (CPRE) has urged council chiefs to review the Black Country Plan, which would see more than 7,700 homes built on green belt sites.
Thousands of other homes have been earmarked for green spaces that are not officially classed as green belt.
The charity said there was no justification for building on such sites and called for housing allocations to focus on areas where homes were "really needed" such as town centres and brownfield plots.
It also questioned the methodology behind the region's "high" housing target, and said the plan's policy on climate change was "inadequate".
A report highlighted 13 sites across Walsall that should be removed from the plan on the grounds that housing would have a "high impact" on the green belt.
They include Calderfields West next to the Arboretum, which has been lined up for 592 homes, a huge plot for around 1,000 homes on Doe Bank Lane in Streetly, and a site off Sutton Road where 202 homes could be built.
The move would remove just over 8,000 homes from the plan, which aims to build 76,076 homes across the region over the next 18 years in order to meet government housing targets.
Peter King, vice chair of CPRE in the West Midlands, said the charity was not convinced that the evidence supporting housing plans for the Black Country green belt was "robust".
He said: "We do not believe the loss of countryside, cherished landscapes and amenity spaces is justified and we call on the four councils to review the opportunities for new housing where it’s really needed in our town centres, and other derelict brownfield sites around the Black Country."
In the report, the charity said the potential for developing existing retail, leisure and employment sites had "not being adequately explored".
Questioning the methodology used to calculate the "high housing targets" used to justify releasing land from the green belt, it cited ONS projections suggesting around 60,000 homes are required over the period - more than 16,000 fewer than the plan.
Council chiefs in all four boroughs have spoken out against green belt development but say there are not enough brownfield sites available to meet housing targets.
Councillor Ian Brookfield, leader of Wolverhampton Council, has asked ministers to reassess the number of new homes the Black Country is expected to deliver.
A much-criticised public consultation into the draft plan concluded on Monday. It is set to go before the ruling cabinet of each council in the coming weeks.
Here is the full list of sites the CPRE wants to see removed from the plan:
Walsall: land next to the Arboretum (592 homes); Doe Bank Lane, Streetly (1,000 homes); Star Service Station, Queslett Road East (53 homes); Sutton Road (202 homes); Pacific Nurseries, Aldridge (three sites totalling 415 homes); north of Stonnall Road, Aldridge (363 homes); Home Farm, Sandhills, Walsall Wood (1,417 homes); land at Vicarage Road/Coronation Road, High Heath (504 homes); land at Mob Lane, High Heath (209 homes); land at Yorks Bridge, Lichfield Road, Pelsall (580 homes); Yieldsfield Farm, Stafford Road, Bloxwich (978 homes).
Wolverhampton: Northycote Lane, Bushbury (182 homes); Moseley Road, Bushbury (three sites totalling 350 homes); Grassy Lane, Fallings Park (three sites totalling 263 homes); land east of Wood Hayes Road, Wolverhampton (40 homes).
Dudley: Holbeache Lane, Kingswinford (330 homes); The Triangle, Kingswinford (533 homes); Worcester Lane , Stourbridge (three sites totalling 115 homes.