Burglars struck at the house in Wolverhampton on Friday while family and friends paid their respects to the baby boy, who had died suddenly in his sleep on Father's Day.
The boy's grandfather, who has asked to remain anonymous along with the parents, said the family found the house had been turned upside down when they returned from the temple.
Jewellery, money, watches and a ring were taken in the raid in Fourth Avenue, Low Hill, and the grandfather said he couldn't understand how the burglars could be so callous.
"I am more than a little bit depressed and very sad about this, as is the rest of the family, with my son taking it very hard," he said.
"I reckon they must be the scum of the earth and are so heartless as they have no thought about what people are going through with the funeral.
"It's worse to think that they've gone and done this while we're at the funeral and I just don't know what goes through their minds."
He said West Midlands Police had taken CCTV footage from a neighbour and had taken photos of the jewellery to pre-warn any local pawn shops.
Conrad Brissett, a long-term friend of the family, said the burglary had left the community angered.
He said: "I'm just saddened by it all and disgusted that they knew the funeral was taking place and saw the horse and cart leave the home before doing what they did.
"The father found out on Father's Day that his son had died and the family has already been through so much, so I can't imagine how they felt coming back to this.
"I would ask anyone who saw anything to call the police as these cold-hearted people need to be caught."
A spokesman for West Midlands Police said: "We are investigating a burglary at a house on Fourth Avenue in Low Hill, Wolverhampton which took place sometime between 10am and 3pm yesterday, while the occupants were attending a funeral.
"We don’t underestimate the distress this has caused the residents and would ask for anyone who has any information or noticed any suspicious behaviour in the area recently to contact us as soon as possible.
"Enquiries continue and anyone with information is asked to contact us via Live Chat at www.west-midlands.police.uk 8am - midnight or call 101 anytime.
"Crimestoppers can be called anonymously on 0800 555 111. Quote crime reference 20WV/160633P/20."