Councillor Chris Tranter, who was first elected in 2010, served the Rowley ward and sat on numerous committees including planning, licensing and the health scrutiny board.
He also represented Sandwell Council on West Midlands Fire Service's authority committee.
The council's interim leader, Councillor Maria Crompton, said: "Chris was a lovely man and we’re all really going to miss him.
"We would like to express our condolences to his family at this very sad time and let them know we’re thinking of them.
"Chris was a dedicated councillor for the people of Rowley ward and he will be remembered for his friendly, approachable manner and his many contributions serving the local community here in Sandwell.
"We will make sure all councillors have the opportunity to pay tribute to Chris as our next full council meeting."
Councillor Susan Eaves, a fellow Rowley ward councillor, said: "Chris was one of the nicest people you could know and a great friend and colleague.
"He was a mentor to me before I was elected as a councillor and he did so much for our community, representing local people and helping them to sort out all kinds of issues.
"Just one example was helping to get much-needed traffic lights installed at the Doulton Road junction, to improve safety for everyone.
"That was just one of the many projects we worked on so well together. I shall miss him greatly."