In what was a game lacking in quality, the Baggies should have gone in at the break 1-0 up but Karlan Grant saw a penalty saved just seconds before the interval.
Valerien Ismael's side did open the scoring shortly after the re-start - with Kyle Bartley producing a brilliant header to beat goalkeeper Bartosz Bialkowski.
Millwall pulled level just minutes later thanks to a header from Daniel Ballard that was just as impressive as Bartley's.
Overall, though, this was a frustrating afternoon for Albion who constantly sent long balls into the Millwall box.
But with a deep back three full of tall central defenders, the visitors were able to comfortably deal with that threat.
And that meant the game, in summary, proved a drab affair with Albion not showing enough quality to break down their opponents who came away deserving of their point.
With Matt Clarke and Dara O'Shea injured - and new recruit Kean Bryan not yet ready to start - Ismael had a decision to make as to who would partner Kyle Bartley and Semi Ajayi in his back three.
And it was Cedric Kipre who got the nod with the Frenchman earning a recall having been left out the squad that won at Peterborough.
The Baggies lined up as expected in midfield with Darnell Furlong and Conor Townsend at wing-back and Jake Livermore and Alex Mowatt in the midfield of the park.
Up front, Matt Phillips was the central striker with Grady Diangana on the right and Karlan Grant on the left.
Millwall lined up in a similar 3-4-3 system - with Matt Smith their central striker. He was supported by Jed Wallace and Sheyi Ojo.
In what proved to be a subdued and uneventful opening half-hour, both teams - in their matching systems - cancelled each other out.
Albion did create one chance with Diangana, who had been Albion's brightest player, doing well to carry the ball past Jake Cooper and enter the box.
But around 10-yards out, he then hit a shot across goal that sailed wide.
The game continued to be scrappy from that point with neither side showing any quality.
But in the 39th minute, Millwall really should have got themselves ahead when a poor header from Ajayi put Wallace through on goal.
Having already scored three times this season, you expected the Lion's talisman to simply slide the ball past Sam Johnstone.
But the England international did really well to stick out a boot and somehow turn the ball behind for a corner.
In what had been an incredibly scrappy half of football, Albion's 'vertical' style of play was proving meat and drink for the visitors who simply sat deep and headed the ball away time after time.
It had all been too comfortable for Rowett's side/.
But in the first-half stoppage time, Albion won a penalty thanks to a brilliant corner routine straight from the training.
A clever pass from Furlong allowed Diangana to get the wrong side of his marker before being tripped.
Referee Andy Woolmer had no choice but to point to the spot.
Grant, though, saw his penalty saved with the forward hitting a poor strike that was the perfect height for Bartosz Bialkowski to parry away.
Clearly unhappy with what he has seen from his side, Ismael made a change at the break with Jordan Hugill replacing Diangana.
And that changed sparked an incredible couple of minutes that saw Albion open the scoring and Millwall miss a glorious chance to equalise.
The Baggies broke the deadlock thanks to an outstanding header from Bartley.
It was Townsend who got the assist with the wing-back whipping in an excellent cross from the left.
Around 10-yards out, though, Bartley still had it all to do to generate enough power to beat Bialkowski.
But he did just that with the centre-back producing an brilliant header to plant the ball into the corner and beyond the reach of the keeper.
Just moments later the visitors should have been level when a hopeful freekick into the box fell for Murray Wallace just six-yards out.
The defender simply had to make the net bulge but instead blasted wide.
If Millwall were worried that miss would be costly, they didn't need to be with their team level just moments later.
In the second phase following a corner, Jed Wallace whipped in a dangerous cross.
And that was nodded in at the back post by Ballard - whose header was just as impressive as Bartley's.
From that point on the game was much more frantic with Albion seeing more of the ball but with Millwall looking dangerous on the counter.
With around 15 minutes to go, though, the visitors should have got themselves in front when Ajayi played his third poor backpass of the match.
That allowed Ojo to charge towards goal but Johnstone once again spared his team mates blushes with an excellent stop to deny the Liverpool loanee.
And that proved the last meaningful chance of the match with Albion huffing and puffing but never producing the quality needed to break through Millwall's resolute defence.
Albion (3-4-3): Johnstone, Kipre, Bartley, Ajayi, Furlong, Mowatt, Livermore, Townsend, Diangana (Hugill 45), Phillips, Grant (Robinson 59).
Subs not used: Button, Taylor, Reach, Molumby, Snodgrass.
Millwall (3-4-2-1): Bialkowski, Wallace, Cooper, Ballard, Malone, Kieftenbald, Saville, Mitchell (Leonard 80), Wallace, Ojo (Bennett 81), Smith.
Subs not used: Long, Thompson, Bradshaw, Pearce, Mahoney.