Speaking to the Express & Star during a visit to Wolverhampton, he said he recognised there were problems at the £150 million Featherstone site – but claimed they would be resolved by management.
When pressed about a deadline for the changes, he said: "I wouldn't put a time-frame on it."
HMP Oakwood – the UK's biggest prison with 1,600 inmates – has been dubbed 'Jokewood' after a damning report listed a string of criticisms over its performance.
The report found criminals were abusing drugs and alcohol while serving their sentences, as well as raising concerns over high levels of violence. It also claimed hundreds of sex offenders were due for release without having behaviour properly addressed.
Last month inmates managed to get onto the roof to stage a protest, while a trainee prison officer is suing the jail, claiming he was badly hurt during training.
Despite the catalogue of problems, Conservative MP Mr Grayling said he was optimistic that once initial 'teething problems' had been overcome the prison would be 'very good'. He said:
"It's a newly-opened prison. Every new prison has teething problems, whether public or private.
"I am very optimistic for Oakwood. It is a first-class facility. It is the most impressive set of facilities I have seen on a prison estate. Clearly the management of the prison need to address the problems but it's a prison that will be very good."
"I wouldn't put a time-frame on it," he added.
HMP Oakwood has faced criticism in recent weeks following the publication of the damning report by inspectors.
Announcing the report, the chief inspector of prisons Nick Hardwick said the jail needed to make urgent improvements and said the results of the surprise inspection were 'very concerning'.
Fresh concerns have also been raised about Oakwood by shadow justice secretary Sadiq Khan on a visit to the region.
Mr Khan claimed he had 'serious concerns' that a number of issues at the jail were not being addressed.
Mr Grayling's comments come after the Express & Star revealed proposals to turn part of Oakwood and nearby HMP Featherstone into open prisons had been shelved by the Ministry of Justice following an outcry from villagers.
The proposals would have seen prisoners at Featherstone allowed into the community, as well as a Category D wing created at Oakwood. But Jeremy Wright MP, parliamentary under secretary of state for justice, confirmed the plan had been dropped in a letter to South Staffordshire MP Gavin Williamson.
Mr Grayling was speaking during a visit to find out first-hand how the Wolverhampton Youth Offending Team operates – and he hailed the organisation a shining example for others to learn from. He met staff members and youth offenders at the organisation's base in Beckminster House, in Birches Barn Road, along with Wolverhampton South West MP Paul Uppal.
Mr Grayling said: "What has struck me here is the relationship that exists between staff and offenders. This is one of the best performing youth offending teams in the country. It is impressive work they are doing here. What is happening here in Wolverhampton is something other parts of the country could learn from."