Carlos Corberan's side were the better side in the first period but spurned a number of openings, while the visitors improved after the break and might have netted themselves.
Albion did not do enough to trouble George Long's goal in the second period but were denied what appeared a clear penalty a few minutes from the end as referee Jeremy Simpson - whose second-half decisions left fans furious - incorrectly called an offside when substitute Daryl Dike was upended in the box.
A point saw the Baggies back above Preston, who played earlier, to ninth but the gap to sixth - where Millwall started the day - remained five points.
The three sides directly above Albion; Coventry, Norwich and Blackburn all lost but Corberan's men failed to truly dent the gap having only taken a point on an afternoon of what might have been after a 17-day extended international break.
It was an 11th Hawthorns clean sheet in 12 as the near-impeccable rearguard record continues, but a blank at the other end was the first time Albion have failed to net on their home patch since Corberan's first game in October, the 2-0 Sheffield United defeat.
Albion had a few days to dwell on news that Republic of Ireland duo Dara O'Shea and Jayson Molumby returned from duty with their nation injured.
In stand-in skipper O'Shea's case it ruled out the remainder of his season, for Molumby it is much of the remainder of the season.
There were four changes in all, Darnell Furlong and Nathaniel Chalobah replaced the injured duo, with Adam Reach selected over Marc Albrighton on the left flank and Brandon Thomas-Asante favoured to Dike - who had travelled with the United States in the break - up front.
Okay Yokuslu was named captain in O'Shea's absence.
Gary Rowett's visitors stated the game four places - after Albion lost a place to early kick-off winners Preston - and five points above their hosts.
Millwall made just one change to their starting XI, with Duncan Watmore in on the wing for Andreas Voglsammer.
Unlike sleepier previous home contests against struggling Wigan and Huddersfield, The Hawthorns carried a big-game atmosphere in the early stages, where there was also a hint of nerves in the air
The opening stages were cagey with little to shout about in front of their goal. Chalobah, with Thomas-Asante starting for the first time since defeat Watford in February, was busy and showed an intent to get forward into advanced positions.
The January free agent signing lunged himself at a John Swift corner but could only direct a tough effort off target.
Millwall also carried a threat from the dead ball - of sorts. George Saville's long throw was troublesome and a delivery from the right was met by in-form striker Tom Bradshaw, formerly of Walsall, whose diving header beat a retreating Ajayi, but the effort was comfortably over.
Albion gained some momentum after 15 or 20 minutes and worked a couple of clearer openings, but failed to take advantage.
The first was created wonderfully from Thomas-Asante, whose body-swerve sold a defender, before a low cross was met on the stretch by Reach - but the winger was unable to get the desired purchase to really trouble George Long's goal.
The Baggies went closer still shortly afterwards with Millwall unable to clear Furlong's long throw. Former Lion Jed Wallace - the away end's panto boo boy - lashed a half-volleyed cross across goal but Albion were wasteful as both Ajayi and Chalobah efforts lacking conviction.
A correct offside flag denied Wallace racing clear on goal against his former club as Corberan's men broke three on two with intent.
Albion cranked up the pressure as the home stands cranked up the volume in the final 10 minutes of the first period.
Millwall again struggled to clear and Albion's threat was from the right of the box once more. Yokuslu was clever with the ball and worked space to pick out a fine low cross.
It found Wallace in a crowded box and the winger's first-time strike looked goalbound until it crashed into a huddle of bodies, possibly cannoning of team-mate Thomas-Asante or a red shirt.
The rebound dropped outside the box for the waiting Reach - who connected excellent with his first-time hit, which whistled like an arrow narrowly wide of the top right corner, with Chalobah unable to turn off target.
The hosts were the better side. Millwall did just about enough at times with a final touch or challenge to deter Albion - but while being the better side, the Baggies' first-half display lacked conviction in front of goal.
Millwall started the second period with intent and that was a theme that continued. Albion's first-half form deserted them and the visitors upped the ante and threat levels.
Though Albion's only notable effort between the interval and the closing stages was close as the impressive Thomas-Asante sent a rocket narrowly over from distance after Reach's fine flick.
Home fans slid towards the edge of their seats as the Londoners came forward time and again.
George Honeyman sent a clever looping volley narrowly over Josh Griffiths' crossbar before the newly-capped England under-21 international made a good parried save to his right from Bradshaw's header.
Crosses looked like undoing Albion as another delivery from the Millwall left found the unmarked Zian Flemming, who was bright throughout, but his header was poor.
Referee Jeremy Simpson did his utmost to add spice to the contest with unnecessary and pedantic whistling, four bookings - two each - in the space of about 10 minutes riled up both sets of fans, especially Baggies.
Corberan tried to stem the tide of pressure after the hour by turning to Dike for the quiet John Swift. The head coach typically changes one striker for another, but clearly Albion needed a goal.
Thomas-Asante then moved to the left as Tom Rogic and Taylor Gardner-Hickman were introduced for Chalobah and Reach.
Still Albion struggled to exert constant pressure with a cagey and crucial contest in the balance.
A Dike strike from a low right-sided cross looked on target before it cannoned into a Millwall shirt. There were half-hearted appeals for a penalty.
But the clash, and certainly its closest stages, might be remembered for what should have been a Baggies penalty a few minutes from time - VAR would certainly have overturned what appeared an incorrect call.
Yokuslu was challenged in midfield as replays highlighted a fair challenge, but the ball travelled through towards the penalty area with Dike on the charge.
Out charged keeper Long to the left edge of the box and he cannoned into Dike bringing the striker down - replays highlighted a clear foul with the Baggies man first to the ball.
But referee Simpson's assistant on the far side raised his flag for offside - incorrectly as the ball came through off the Millwall midfielder's challenge. The assistant was a long way away and wouldn't have seen, but the referee was close to the incident.
Albion were incensed but there was no changing the mind.
Six minutes were added on but they came and went with no further clear opening with the spoils shared between the play-off rivals.
23 - Good chance Albion. Jed Wallace fizzes a low cross in from the right but neither defender Semi Ajayi or midfielder Nathaniel Chalobah could find the required connection to send on target.
34 - Okay Yokuslu crosses low from the right and Jed Wallace's goalbound strike is blocked, falls to Adam Reach whose 20-yard rocket is just a few yards wide of the top corner.
59 - Josh Griffiths makes a good parries save to deny Tom Bradshaw.
86 - The really big moment as Daryl Dike races through on a ball into the box after Okay Yokuslu is challenged in midfield. Ball cannons forward off Millwall leg and Dike is clearly felled in box by goalkeeper George Long. Officials give offside after assistant raised his flag, but referee Jeremy Simpson in close quarters should have seen ball come off Millwall leg.
Albion (4-2-3-1): Griffiths; Furlong (Albrighton, 89), Ajayi, Pieters, Townsend; Yokuslu (c) (Bartley, 89), Chalobah (Gardner-Hickman, 74); Wallace, Swift (Dike, 63), Reach (Rogic, 74); Thomas-Asante.
Subs not used: Button, Livermore.
Millwall (4-2-3-1): Long; McNamara, Cooper (c), Cresswell (Malone, 90+8), Wallace; Leonard, Saville; Honeyman (Shackleton, 90+2), Flemming (Hutchinson, 90+2), Watmore (Burke, 63); Bradshaw (Voglsammer, 84).
Subs not used: Bialkowski, Esse.
Referee: Jeremy Simpson