The 29-year-old has turned down the chance to remain at the Memorial Stadium in favour of a move to the Banks's where he will again team up with manager Darrell Clarke.
Boss Clarke first managed James at Salisbury before then taking him to Rovers in 2015.
Predominately a centre-half, the defender can play anywhere across the back four and made 48 appearances for the Gas last season.
The length of his contract is undisclosed.
“I’m buzzing to be here,” Clarke said.
"It’s a great project from what the gaffer has explained to me. The club have got lovely training facilities and a great stadium and I am really happy to be here.
“I am a versatile player. In the last few seasons, I have played across the back four, predominantly as a centre-half, but I can do a job anywhere across the back four.
“I am somebody who will give everything for the shirt and for the club.
“In my only season in League Two, I was fortunate enough to be in the gaffer’s side that went up with Bristol Rovers.
“He knows how to build a good changing room and that is a big part of being successful.
“We need to kick on in this new era and hopefully it is going to be a positive season.”
With James Clarke having been offered new terms to remain at Bristol Rovers, Darrell Clarke admitted he was a touched surprised to secure his signature.
“I’m pleasantly surprised to be honest with you," the boss said.
"I didn’t think we would have a chance of getting him.
“He’s a lad I know very well and he can play anywhere across the back four. He’s more of a centre-half but he’s played a lot of times at right-back.
“We’re not going to be working with as big a squad this year so it’s important we’ve got players who can fill in a few positions.
“He is 100 per cent commitment and it’s about getting the environment right to the football club.
“He’s quick, strong and uses the ball well. He’s one of the best man-to-man markers from set-pieces I’ve ever worked with; I can’t ever remember his attacker scoring a goal. He’s got all the attributes.”
Meanwhile, Walsall have announced they will play Tranmere in a pre-season friendly. The game will take place at Prenton Park on Saturday, July 27.