Councillor Neville Patten led the authority from 2008 to 2010 when it was run by the Conservatives in an alliance with the Liberal Democrats.
The Honorary Alderman, who was first elected to the council in 1987, had represented Oxley Park and Bushbury North before stepping down from the role in 2014.
Councillor Wendy Thompson, leader of the Conservative Group, said: "Neville was a long-standing councillor for Wolverhampton who also became not only the group leader, but the leader of the council for a short period of time.
"And during that time Neville was immensely proud of being the leader of Wolverhampton and it meant a great deal to him.
"He was a very caring person and he was someone who was popular not just among Conservatives, but certainly he had a lot of respect from other people too.
"He was very local to Wolverhampton and had a good sense of humour too. It's a sad loss and we obviously feel for his family at this very difficult time.
"It must be a terrible loss to them but also to Wolverhampton because it meant so much to him – he tried to do as much as he could to benefit the city."
Mr Patten had been a former apprentice at Guy Motors, had run a pub, a newsagent and then drove buses for a living. He served on Wolverhampton Council for the best part of 27 years.
A spokesman for Wolverhampton Council said: "It is with sadness that we announce the death of honorary alderman and former council leader Neville Patten following a short illness.
"Neville, a proud Wulfrunian, served the city as a councillor for many years. We offer our sincere condolences to his family at this time."
In 2008 Mr Patten had just been chosen to lead his group when a huge swing of support moved from Labour to the Conservatives – but the party didn't have enough seats to control the council alone.
It led them to team up with the Liberal Democrats – two years before the same would happen in Westminster – with the council's finances left "in a bad way", facing a £5 million black hole.
Cuts were drawn up to tackle the issue with the coalition also delivering a council tax freeze a year before then-Chancellor George Osborne made it standard practice across the country.