On a night dominated by spot-kicks, Hal Robson-Kanu twice put the Baggies ahead with calmly despatched penalties.
Those goals were sandwiched between a stunning Emiliano Marcondes over-head kick.
Brentford then got themselves on terms again with a penalty of their own - with Cedric Kipre very harshly deemed to have fouled Marcus Forss - who then fired in from 12-yards.
With the game tied at 2-2 after 90 minutes, that meant it would be decided by even more penalties.
But after netting their first four, Grady Diangana saw his effort saved by David Raya. Christian Norgaard then scored for Brentford to send them through to the next round.
Throughout his time at The Hawthorns, Bilic has used the League Cup to rotate his squad.
And that proved the case once again with the boss changing his entire starting XI from Saturday's defeat at Everton.
Plumping for a 4-3-3 system, Branislav Ivanovic and Conor Gallagher both made their Albion debuts having signed for the club in the past week.
Ivanovic partnered Cedric Kipre at the heart of the defence with Gallagher joining Sam Field and Rekeem Harper in a midfield three.
Up front, Robson-Kanu was the lone centre-forward with Matt Phillips on the right and Kyle Edwards on the left.
Brentford, under the guidance of Thomas Frank, also lined up in a 4-3-3 formation.
Former Albion youngster Forss led the line with Sergi Canos and Tariqe Fosu out wide.
In what was a keenly contested clash, both teams largely cancelled each other out in the opening 45 minutes.
Albion almost broke the deadlock early when Phillips did well to charge down the right and send in a low cross.
Darting into the box, Gallagher then met the delivery but couldn’t find the fairytale finish and instead side-footed over from 10-yards.
Brentford responded and came a whisker away from opening the scoring midway through the half.
Emiliano Marcondes latched onto a through-ball before arrowing a low drive which smashed against the post.
Fosu-Henry reacted quickest but his strike was blocked by Lee Peltier - denying a certain goal.
Those chances aside, there had been little to separate the sides.
But 10 minutes after the re-start, Albion were ahead.
A promising move down the left looked to have broken down when Townsend miss-placed a pass which should have been collected by Mads Sorensen.
But the defender let the ball go under his football with Robson-Kanu then darting through on goal only to be hacked down by Dominic Thompson.
Referee Darren England had little choice but to point to the spot - with Robson-Kanu then calmly rolling the ball home.
Just two minutes later, though, the Bees were level thanks to a brilliant goal.
Sorensen sent in a cross which Charlie Goode headed back across goal.
And that was met by Marcondes who sent a stunning overhead kick into the roof of the net.
Incredibly, Albion were then awarded a second penalty just past the hour mark when Edwards was tripped in the box by Goode.
Robson-Kanu again stood over it and casually side-footed the ball into the bottom left corner for his second of the night.
If the game hadn't been crazy enough, though, Brentford were then awarded a penalty of three own in the 73rd minute when Kipre was harshly judged to have fouled Forss - with the Finland under-21 international smashing home the spot-kick.
Albion responded with Gallagher hitting a curling effort from just outside the box that looked destined to hit the back of the net.
But at full-stretch, Raya got across his goal to expertly push the ball away.
That save meant penalties.
Charlie Austin, Gallagher, Townsend and Edwards scored Albion’s first four.
But Diangana missed the fifth with Norgaard scoring for Brentord who netted all of their penalties to clinch a spot in the fourth round.
Albion (4-3-3): Button, Peltier, Ivanovic (O'Shea 65), Kipre, Townsend, Field, Harper, Gallagher, Phillips (Diangana 70), Edwards, Robson-Kanu (Austin 85).
Subs not used: Bond, Furlong, Sawyers, Pereira.
Brentford (4-3-3): Raya, Thompson, Sorensen, Goode, Henry, Baptiste, Zamburek (Dasilva 66), Marcondes (Toney 71), Fosu-Henry, Canos (Norgaard 66), Forss.
Subs not used: Daniels, Dalsgaard, Pinnock, Dervisoglyu.