The 563-acre site near Stone would be part of a sell-off by the Ministry of Defence and pave the way for the biggest house building project in Stafford borough for a generation.
A school, doctor's surgery, and a new railway station could all feature as part of the proposals.
The huge housing project will be sparked if a new rail depot for HS2 is nearby at Yarnfield goes ahead.
The works for the high-speed rail line would see a new junction created on the M6 between junctions 14 and 15, creating a major link to the planned development.
Surrounding land between Eccleshall, Norton Bridge and Yarnfield would also be developed.
There would also be more than 200 acres set aside for businesses.
Stafford Borough Council says the proposals, which are at an early stage, could relieve the pressure of building more homes in other towns and villages across the borough and retain the area’s ‘shire identity.’
Swynnerton training camp is used by reserve forces and cadets from across the Midlands and Wales, including the Wolverhampton-based 4th Battalion The Mercian Regiment.
This site at Cold Meece is classified as brownfield land and was a munitions factory during the Second World War.
Council leader Patrick Farrington, said: “The idea of a new garden settlement is only at the concept stage and entirely dependent on HS2 and the Government confirming their plans before we could develop ours in any detail.
“I have always been clear that if what is proposed as part of the HS2 project goes ahead, then as a borough we should capitalise on any economic gains that this brings.
“If and when they progress they would be part of the local plan and subject to detailed consultation with local residents.
“If there were to be new sites put forward as part of the local plan process then it would obviously reduce the need to look for other sites to build on in our existing towns and villages.
“And a permanent junction would allow better access to businesses and communities and would provide a boost for the economy of Stone.”
The council has not put a figure on the number of new homes at this stage but garden villages are developments of between between 1,500 and 10,000 properties.
Earlier this year the Government announced 14 new garden villages would be built across the country.