Few cabinet ministers in history can have endured such a painful ride, with the South Staffordshire MP facing negative headlines on a daily basis since the pandemic plunged education into crisis.
When he was appointed to the role in July 2019, it was viewed as a reward for his efforts in helping Boris Johnson land win the Tory leadership race.
With the PM making no secret of the fact that education was a key part of his levelling up agenda, Mr Williamson's new position – so often a poisoned chalice – looked like it might be a cushty gig.
All of that changed with Covid of course, and Mr Williamson has spent the bulk of the last 18 months attempting – without much success, it must be said – to put out a series of fires.
There was last year's free school meals debacle, and the exams fiasco surrounding the now infamous 'algorithm method' of calculating results. This was followed swiftly by this year's two-pronged exams fiasco, culminating in fury over the achievement gap between state and private school pupils.
Along the way there have also been a liberal smattering of gaffes, most notably when he appeared to get Marcus Rashford mixed up with rugby player Maro Itoje.
As Mr Williamson himself has conceded, mistakes have been made, but even among the seemingly endless criticism there has to be a degree of balance.
Quite frankly, no one had a clue what to do when classrooms were suddenly shut down for months on end, and it was nigh on impossible to plan ahead in the face of a pandemic which changes by the day.
In such circumstances, no Tory education minister was ever going to get much change out of teaching unions that are hostile at the best of times.
Many of the challenges he faced were unprecedented.
For example, in his worst nightmares it is unlikely that Mr Williamson would have envisaged being plunged into a national debate about the mass vaccination of children.
But this, along with many other contentious issues, is exactly what he came up against.
His own government hardly helped matters, shafting him over Covid catch up funding back in June by giving him less than a third of what had been requested.
Amid the almighty pile on, Mr Williamson's flagship policies – such as a major school rebuilding programme and reforms to post-16 education – have been somewhat lost in the noise.
Yet despite all the criticism that has been thrown at him and all the abuse he has received – some of which has been truly horrific – he has maintained a stiff upper lip.
Indeed, in a recent interview he spoke of his "rhino characteristics", highlighting the "grit and determination" that had carried him through some dark times in politics.
He remains a hugely popular figure in his constituency, where he reigns with a majority of more than 36,000 and has continued to campaign on local issues throughout his time in government.
Not for the first time, Mr Williamson will now be written off as a political heavyweight.
It was in May 2019, after he was unceremoniously dumped as Defence Secretary, that many among the political commentariat gleefully called an end to his time at the top table of politics.
Yet within a few weeks he had returned.
After the 18 months he's endured, Mr Williamson may welcome a spell on the backbenches, but don't rule out another comeback.