The Saddlers' winless run was extended to 11 matches as they conceded a last-gasp Elliott Nevitt header to lose 1-0 against Tranmere Rovers at Bescot.
It means they have picked up just two points from their last seven League Two matches.
Boos rang out after the final whistle, and Flynn said: "I feel their pain. I'm putting my life and soul into Walsall Football Club, and so is Wayne (Hatswell, assistant manager).
"We need a little bit more back from a few of the players.
"I'm an experienced manager. I know what I'm doing.
"We're trying to build a team here that will get Walsall into the next division.
"It's not going to be done in one transfer window. We've got to be positive and know things will turn around, because they will. But we do need a win."
Walsall conceded in the fifth minute of stoppage time as Nevitt headed in from close range – after a shot from Kieron Morris was parried by Owen Evans.
"The players have been brilliant in terms of not getting too down or too high, in the early part of the season," said Flynn.
"I have just told them they're letting us down and letting the supporters down, because we can't keep saying it.
"Everybody knows you can't make that turnover of players in one transfer window and expect everything to gel straight away, but I am also conscious that you've got to give them something to be happy about as well.
"I am conscious of that. I'll work as hard as I can, day in day out, like I always do.
"Hopefully, we get a bit of luck and we start getting some wins on the board."
Flynn added on Tranmere's late winner: "I've said it for weeks, we weren't good enough in both boxes and we need to improve.
"We've had a chance to score and win the game. It gets cleared and then it's a throw-in, our throw-in, and two of our midfielders – Comley and Kinsella – switch off and let their runners go.
"Donervon then has a decision to make, whether to try to block the shot or stay with his man.
"Owen should have done better, the lad's the other side of Donervon and it's 1-0.
"It's a kick in the teeth, and we need to be better in both boxes."