The Saddlers fell to a 2-1 defeat at Rodney Parade as former defender James Clarke headed in at the death.
Taylor's lot had Tyrese Shade – straight after Carl Rushworth saved Matty Dolan's spot-kick – cancel out Mickey Demetriou's opener in a dramatic end to the first half.
Walsall, though, struggled to carve out significant opportunities despite enjoying more possession after the break.
And Taylor said: "The first half, I thought that Newport were the dominant team.
"But we managed to stay in the game. Carl made a fantastic save before half-time and we got a very good goal.
"In the second half, I thought we were confident and dominated the football. Our movement patterns were very good to see.
"But it comes down to the final third, and we didn't make good enough decisions when we opened up the opposition.
"You always run the risk when you don't take your chances, and we got punished. It was just one too far for Carl to make a save.
"We scored a goal, hit the bar, and got into good areas.
"If we look at where we can improve in terms of decision-making, most definitely we can get better in the final third.
"We got into enough offensive areas to hurt the opposition. That's where we've fallen down with our performance.
"But this group of players are giving everything, every day. The group are together."
The positive from the afternoon was Shade opening his account for the club with a scorcher from the edge of the box.
He was later taken off for Brendan Kiernan, though, while the Saddlers had Liam Kinsella limp off with a knee issue late on.
Walsall now sit 19th in the League Two table with eight points from the same number of games.
On Shade, Taylor said: "It was a good goal, but we were asking Tyrese to play three games in seven days.
"He got injured the last time we asked him to do that.
"He took his goal well, and there's far more to come from him."
Taylor added: "I thought the players gave the performance, but we haven't got the result.
"It's difficult to take, but we've got make sure we train well ahead of a home game against Bristol Rovers."