Matt Taylor's side went in ahead at half-time thanks to Kieran Phillips' cool finish.
But a horrid final half an hour saw Joey Barton's Rovers, who had gone 19 without winning on the road and picked up just four points from a possible 57, prevail at the Banks's Stadium.
Connor Taylor started the comeback, with substitute Sion Spence completing it in stoppage time – the Saddlers failing to deal with corners for both goals.
Walsall now sit 22nd in the League Two table with eight points from nine games.
They will remain in the lower reaches if such poor defending carries on as well.
It is now an obvious weakness that teams like Rovers, who struggled to get going themselves for large spells, will happily capitalise upon.
If the Saddlers are to do anything significant this campaign, they must put a stop to it as soon as possible.
Walsall made three changes from the 2-1 loss against Newport County.
One of them was enforced, with Liam Kinsella, who needs a knee operation after limping off at Rodney Parade, dropping out.
Rollin Menayese was among the substitutes, along with Emmanuel Osadebe. Ash Taylor, Sam Perry and Brendan Kiernan came into the starting XI.
Rovers, meanwhile, arrived with a dour away record, having not won on their travels since December.
Their experienced line-up included former Villa midfielder Glenn Whelan and centre forward Brett Pitman.
It was the visitors who started the sharper of the two teams, penning the Saddlers in and getting forward quickly.
They could have taken the lead, too, after Taylor got caught underneath a long clearance.
Harvey Saunders burst down the left and the ball was eventually worked out to Harry Anderson – denied by a combination of Manny Monthe and Carl Rushworth. Cian Harries then fired narrowly over from the resulting corner.
Saunders looked especially dangerous with his pace and trickery for Rovers, and he was soon scythed down by Jack Earing for a clear yellow card.
To Walsall's credit, though, they withstood that spell of pressure and began to find their feet.
Attacks down the right were causing problems as, firstly, Phillips got to the byline following Rushworth's accurate long throw. His delivery was met by Kiernan, who could not keep his strike below the bar.
Hayden White – who has expressed his desire to offer more going forward – also surged up from right-back and saw a cross fall to Earing, whose attempt was blocked.
The Saddlers' confidence was growing, and they converted their next opportunity.
Kiernan released Phillips – having switched to the left – and he did the rest with aplomb, cutting inside and firing underneath James Belshaw. A fine finish from the Huddersfield loanee, with the ball bouncing up a bit awkwardly.
Rovers enjoyed more territory again as the half drew to a close, and Anderson had a fierce long-range shot sail wide.
Walsall, though, had just shaded it on the whole in the first period. At the start of the second, the away side continued to huff and puff.
Anderson tried to catch out Rushworth at his near post but the Brighton loanee was predictably equal to it. And the Saddlers, in the main, were not living on the edge.
The Rovers faithful grew increasingly frustrated with their team's work, and you sensed a second would kill the game off.
Phillips should have doubled his and Walsall's tally when put through one-on-one with Belshaw, but the keeper won the battle this time around.
George Miller also had a big chance after an excellent through-ball from defender Monthe, with the onrushing Belshaw standing tall once more.
The Saddlers' attacking influence then fizzled out entirely.
Osadebe replaced Kiernan but was forced off with an injury just minutes later, with Menayese stepping in. Zak Mills also entered the fray for Phillips, joining Perry in midfield while Earing moved forward.
It was disjointed and Rovers – despite not being brilliant themselves – took advantage.
Walsall's frailty from set plays was clear to see as Taylor poked home at the far post following a corner.
And Spence, with his first touch after coming on, completed the comeback in added time as the Saddlers were undone from yet another corner.
The travelling supporters were in delirium while the home crowd made their disappointment known. Defending like that is simply not acceptable.
Walsall (4-2-3-1): Rushworth; White, Taylor, Monthe, Ward (c); Earing, Perry; Phillips (Mills, 70), Kiernan (Osadebe, 60) (Menayese, 78), Shade; Miller
Subs not used: Rose (gk), Leak, Willis, Bates
Goal: Phillips (32)
Bristol Rovers (3-4-2-1): Belshaw; Kilgour, Taylor, Harries; Anderson, Whelan (Collins, 69), Coutts (c), Saunders (Spence, 89); Finley; Evans (Brown, 75); Pitman
Subs not used: Ward (gk), Westbrooke, Anderton, Hoole
Goals: Taylor (79), Spence (90)
Referee: Sebastian Stockbridge (Tyne and Wear)