The Baggies thumped Thomas Tuchel's men 5-2 at Stamford Bridge on Saturday – a result that sent shockwaves through the division.
It's likely that result has come too late with Albion still 1/14 on with bookmakers to be relegated at the end of the campaign.
But they face Southampton a week today in what, on paper, looks a winnable fixture with Ralph Hasenhüttl's side having struggled for form in 2021.
And Allardyce believes the win over Chelsea will give his players bundles of confidence and belief moving forward.
"We know there is a long way to go," the boss said when asked about Albion's hopes of staying in the Premier League.
"But I hope the Chelsea game has filled the players with the confidence and belief that, in the Premier League, they can play one of the top sides and not only beat them but outplay them.
"Our finishing was so good, it gave us a great victory that will have shocked many, many, teams and many, many people.
"And I have to say it wasn't down to luck whatsoever. It was down to the quality of our play and the fact our performance was better than Chelsea's.
"All I can ask of the players now is to go out every week and produce the best performance they can.
"Winning like we did at Chelsea has to give us plenty of confidence going into the next game which is Southampton.
"If we win that, that will give us even more belief and then we'll take it from there."
Speaking after Albion's loss at Crystal Palace, Allardyce said his side would need to win six of their last nine games to stay up.
"I said we needed to win six out of the last nine, now after beating Chelsea we need five out of the last eight to have a chance," he added.
"Still that might not be enough but let’s start with Southampton and let’s try and see if we can get another fantastic victory there."