The Chelsea fans booed Foster during today's 1-0 defeat at Stamford Bridge, because they believed he was taking too long to get play going again.
But Foster - who hasn't been booked for time-wasting this season - doesn't understand what all the fuss is about.
"I'm just going about my business," he said. "If they want to get all upset about that then let them.
"I haven't been booked for time-wasting this season so I can't be doing it too bad.
"A lot of away fans get wind that we're a bit of a time-wasting team so they're on it from the first kick.
"I don't think it's taking the mick, it's just using the rules to your benefit."
Chelsea ended Albion's four-match unbeaten run after Diego Costa capitalised on a Gareth McAuley error to lash home a 76th-minute winner.
However, the league-leaders only had two shots on target all game and Foster is adamant Albion deserved a point.
He said: "I haven't had a save to make have I?
"I thought we did really well and could have scored ourselves. We had the better chances as well, which just couldn't find the back of the net.
"Obviously the goal was disappointing and G's very disappointed.
"But you can't blame Gareth McAuley. He's an absolute monster for us.
"He gives seven, eight, nine out of ten every week so you could never ever point a finger at Gareth McAuley."
Antonio Conte's hosts had the lion's share of the ball but rarely got in behind the Baggies back-line.
"I'm sure the possession stats will show they've had a lot more of the ball," admitted Foster.
"But you could look at that Leicester v Man City game. I think they had 78 per cent possession and lose 4-2.
"You can read whatever you want from possession.
"I thought on the balance of play we probably had the better chances.
"It would have carried on our brilliant form, but we've got to forget about this now.
"We've got a big game against Swansea on Wednesday.
"They're the sort of team we need to be taking points from and then (Manchester) United afterwards.
"It's going to be a tough week, with our squad especially because of how old we all are.
"It's about rest and recovery and getting ready for the games."