The Saddlers lost 4-3 on spot-kicks to the League One visitors after a goalless 90 minutes at the Banks's Stadium.
Taylor's new-look side created a string of opportunities – especially in the first half – and while they were unable to find the net, he felt it was a very strong performance.
"I'm probably disappointed it got that far. I didn't feel the game warranted that, with our performance," he said.
"But again, it's a really positive performance. I thought our players were exceptional at times.
"We were strong and fit out of possession and played more football than we did at Tranmere.
"There's a huge amount of positives to take from the game.
"Obviously, we would've loved to be in the hat for the next round, but it wasn't to be.
"The challenge now is for the players to replicate that type of performance against teams in our own division."
The cup exit follows an opening-day loss to Tranmere in League Two but Taylor feels Walsall are making brilliant progress.
They outclassed Doncaster for the first hour, with Conor Wilkinson and Kieran Phillips both having chances to score.
AJ Greaves was sent off for the away side before the Saddlers just fell short in the shootout.
Skipper Joss Labadie went off in what Taylor described as a 'precaution'.
"We're here to win games and be successful, players and staff alike," said Taylor.
"But the signs are exceptional. I'm really pleased.
"There's nothing I can look at and say I'm disappointed with, other than missing the penalties.
"But that's a lottery having been in that position myself as a player. Their players dealt with the pressure a bit better than ours."
Walsall are back in league action at the Banks's as they host Forest Green Rovers on Saturday.
Taylor added on the cup exit: "I thought we were the better team and deserved to win, but football doesn't always work like that.
"So, now the challenge for the players is to make sure they are ready and raring to go for Forest Green on Saturday."