Abdi Rehman Esa, aged 21, assaulted John Henry Smith, 26, at HMP Brinsford, in Featherstone, near Wolverhampton, on December 5.
Esa punched Mr Smith against the side of the head from behind, while he spoke with two other inmates, causing him to fall and hit his head against a door frame, and then against the ground. The force of the punch broke his jaw and rendered him unconscious.
Mr Smith suffered serious brain injuries which he could not recover from. He was taken to hospital the following day where he died.
Due to overwhelming evidence against the defendant, Esa pleaded guilty to manslaughter on January 22.
He was due for release in May this year, but, following his guilty plea, Esa was sentenced at Stafford Crown Court to a further two years and seven months behind bars.
Sabhia Pathan, of the Crown Prosecution Service, said: "This was a completely unprovoked attack. Abdi Esa struck Mr Smith from behind while he chatted to two other inmates and was not prepared for the attack. His reckless actions led to the death of Mr Smith.
"Esa was due to be released in May 2021, but he will now remain in prison for considerably more time where he will have time to reflect on his actions."
In a family statement, Mr Smith's mother Yvonne Arbuckle said: "The entire family are suffering because John is no longer with us and we can’t come to terms that we shall never see him again.
"John’s siblings and I feel a part of us is now missing that he has been taken from us in these circumstances. It is as if a light has gone out in our hearts.
"I’m sorry I am not able to fully reflect the pain and suffering we are feeling, it’s not something that can easily be put into words."
Police were called to the prison just after 8am on December 6 last year, following reports a prisoner had been assaulted by another inmate the previous afternoon.
Mr Smith was initially taken to New Cross Hospital in Wolverhampton, and then to the Queen Elizabeth Hospital in Birmingham, but sadly died.
A police investigation established that Esa had punched John once which resulted in Mr Smith sustaining his catastrophic injuries.
Senior investigating officer, detective inspector Alan Lyford, said: "This was a tragic case and we’d like to offer our condolences to Mr Smith’s friends and family."