The Saddlers put in a superb first half performance and were unfortunate not to score, before the hosts improved in the second 45.
The game seemed destined for a goalless draw before Paul Mullin netted a 94th minute winner and Dutton insists the Walsall players must improve their fitness - but he was pleased with their performance.
“I’m gutted, absolutely gutted,” Dutton said.
“It was a bittersweet performance. The boys have given me everything and I couldn’t be more proud of them.
“They’ve come to a team who are second in the league on their home patch and put in a performance like that. That is the level we need to reach to make things happen for us.
“We had that in abundance but you have to be physically at the top of your game. Top players are so fit because they have to go right through the 93rd, 94th and 95th minute.
“If you’re not able to do that you get punished and that’s what’s happened today, we’ve been punished.
“I’m gutted with the result but really pleased with the performance.
“It became end to end, there were chances for us to win it and chances for them.
“The easy thing to do at Cambridge would be to shut up shop and settle for a point, but I don’t want that mentality at Walsall football club.
“I want us to be able to win at home and away. It’s not happening at the minute for us but I am more than confident that if this group of lads put in more performances like that, we’ll become a force to be reckoned with in time.”
Meanwhile, when asked about the injury sustained to Cameron Norman in the first half, Dutton added: “It’s a dead leg. It usually takes one or two days to settle down so he won’t be far away.
“I’m sure he’ll be available for the Barrow game.”