Smith’s team have been left to rue the intervention of the video official in their last two matches, seeing a stoppage time penalty decision overturned against Brighton, before Ollie Watkins’ late equaliser at West Ham last Monday was ruled out for offside.
Villa’s head coach believes there is currently too much interference on subjective decisions and acknowledges his team is far from the only one to have suffered.
Albion saw a penalty award overturned in controversial circumstances during last month’s defeat at Manchester United, while Brighton were awarded a late spot-kick in their draw with Liverpool.
Smith said: “I certainly think it (VAR) is ruining enjoyment of the game. We all wanted it because it gives good decisions when you want good decisions.
“But at the moment I am looking at some of the subjective decisions – West Brom at Manchester United was a penalty all day long, Liverpool at Brighton is never a foul.
“The idea is if it is a foul, tell them. If it is not a foul just leave it. We can’t have people at Stockley Park refereeing games for subjective decisions.
“It is there for clear and obvious and nothing else. Unfortunately I think there are too many grey areas at Stockley Park at the moment and they are refereeing games.”
This season is the second VAR has been used in the Premier League and Smith continued: “We didn’t get VAR quite right last season. We have made adjustments and we are not getting it quite right this season.
“There needs to be a lot of work done on it, from my point of view. It is here to stay, whether we like it or not.
“I don’t know where we are going with it. Stockley Park should not be getting involved in subjective decisions.
“I don’t think referees are giving up responsibility. I feel sorry for referees who are being told to go and look at a screen to review decisions which are not clear and obvious mistakes.”
Villa skipper Jack Grealish has been nominated for the Premier League player of the month award.