The Saddlers lacked a killer touch as they were downed by two Harry Charsley strikes at Field Mill.
Just before the Stags took the lead, Walsall had George Miller have a shot cleared off the line, and Taylor was left to rue missed opportunities while feeling the scoreline was harsh on them.
On the Miller miss and Charsley opener, Taylor said: "That's the game there and then.
"We've created some really good opportunities by playing a different way – we had to because of the surface and the opposition.
"When the players stuck to the game-plan, we got chances. We looked competitive.
"I was pleased at half-time, with the desire the players showed.
"When you get those chances away from home, you've got to score.
"We didn't do that, and then we didn't defend a set play and make them work hard enough for that goal.
"The second goal flatters them as we were pushing and trying to get back into the game.
"We'd gone two up front, so you're always going to be vulnerable.
"I'm disappointed. There were signs of real competitiveness, and the players are extremely disappointed.
"I don't think it was a 2-0 game. I've got to be perfectly honest.
"I'll accept it when we get beaten well, but we had enough opportunities to get something from the game.
"When you don't take chances, you make it very difficult for yourself."
It was the third game without a win for the Saddlers, while they have only won one of their last eight across all competitions.
Charsley's first came after Walsall failed to deal with a long throw-in, while the second saw him finish off a neat counter-attack and punish slack marking.
"We dealt with their threat for the majority and got into the final third ourselves, but we weren't ruthless enough," said Taylor.
"We've got to score. I can't stand here and make excuses, we've got to score.
"George is fully aware he should score there, and if you score that, it changes the complexion of the game.
"That's football in a nutshell. When you don't take your opportunities, you're always vulnerable."
Meanwhile, Taylor addressed Otis Khan's absence from the squad and said it was due to the winger being ill.
Khan's current deal with the Saddlers is days from expiring, and the club have been in discussions over fresh terms.
But the boss said him not being involved was nothing to do with his contract situation.
Taylor added: "He's ill. I found out this morning that he's been ill during the night, so he wasn't available.
"It's difficult. Hopefully, the illness isn't severe, and he'll be back with us soon."