Trailing 2-1 as the match entered stoppage time, Villa looked to have a chance to level when Oliver pointed to the spot after Trezeguet tumbled under the challenge of Solly March.
But after consulting with the pitchside monitor, the referee then changed his mind as the Seagulls held out for their first-ever win at Villa Park.
Smith said: “I'm at a loss to explain it. I thought VAR was for clear and obvious mistakes. It wasn't clear and obvious to me.
“I don't know what a penalty is now. We could all hear the contact inside the ground.
"If VAR thinks it's not a penalty then just tell the referee it's not a penalty. It wasn't given so we have to take it on the chin.”
The defeat was Villa’s third on the spin at home and their form remains wildly unpredictable.
Danny Welbeck took advantage of dire defending to open the scoring before Ezri Konsa equalised early in the second half.
After March had restored the visitors’ lead, Brighton saw Tariq Lamptey sent-off in a chaotic finish.
Smith added: "I don't think we were bad. We missed chances and we've made big mistakes for their goals. We have to defend much better. Brighton were a threat on the counter but I wasn't overly concerned.
"I thought we had control of the game and if we had taken our chances we would have won.
“I didn’t think we deserved to be beaten. I didn’t think we were at our best but I thought we were good enough to get at least a point.”
Villa are sweating on the fitness of Ross Barkley, with the midfielder due to undergo a scan on a hamstring injury which forced him off after just five minutes