The 39-year-old became the favourite among bookmakers in the last 24 hours when his odds slashed to 1/2.
But now, Taylor had revealed there is no contact between him and the Saddlers and he is not interested in the role.
He told Devon Live: “Someone messaged me that! It’s absolutely incredible. Are you sure it’s me and not the real Matt Taylor, the Portsmouth and Swindon one?”
“I honestly don’t know where that’s come from. I certainly haven’t had any contact and as far as I know the club haven’t had any contact. I can honestly say that and I think the club would tell me if there’d been any contact.
“To be the favourite would suggest things are further down the line when it clearly isn’t.
“All I would say is lump it on someone else. Whoever is the next favourite, go and lump it on them. There are a lot of jobs available at our level and the same names are being linked, but I can honestly say I have had no contact and I have no interest in the Walsall one.
“In terms of what I am concerned with, there is nothing in relation to that.”
Taylor has only ever managed Exeter in his short career, taking over in 2018, but he did take them to the play-off final last season, where they lost 4-0 to Northampton Town.
Exeter have proved to be a dangerous force in League Two for the last two seasons and they missed out on the play-offs by three points this year, finishing ninth.
In the week since Dutton departed former Reading and Crystal Palace manager Steve Coppell has been the firm favourite, but now his odds have dropped to 4/1.
The other names mentioned among the bookmakers are all outsiders, with Dave Reddington, John McGreal, and Derek Adams are all joint-third favourites – priced at 16/1.