The Flyers – who have started well in the Midland Premier – are the favourites going into their second qualifying round tie at AFC Bridgnorth.
But Joshua knows they cannot take the hosts lightly after falling at the first hurdle in the competition last season.
"It's about avoiding a repeat of last year," he said.
"We went to Aylestone Park, who were the league below and in a very similar position to Bridgnorth – quite low down in the table.
"We went 2-0 up but then completely switched off and ended up losing on penalties.
"So, this is all about mentality, really. If we turn up with the right mentality, I'm pretty confident we'll go through.
"But if we turn up with the wrong mentality like we had at Aylestone Park, it's going to be a tough afternoon.
"Bridgnorth will work hard and do the ugly things, and we need to match that.
"We'll put the strongest team out we possibly can.
"We haven't had a good cup run for quite a while now, so we're after a run in the Vase.
"We'll take it as seriously as any league game and put our strongest team out – and hopefully that will be good enough to get through this time."
Lye made it to the Vase's fourth round in 2018/19.
And while the Wembley dream and serious prize money both remain a long way away, the clash stills offers the opportunity to bring in funds.
Winners at this stage receive £725 and Joshua added: "For all clubs at our level, a good cup run is vital.
"It gets money into the club. Anything is better than nothing.
"Clubs are struggling financially and a good cup run, if you can get a few grand in the bank, it's something.
"It's not massive amounts, but it's something to work towards."
Also in the Vase, Bilston Town are in for a tough test as they travel to Worcester City.
AFC Wulfrunians and Darlaston Town 1874 are both on the road, too, at Rugby Town and Heather St John's respectively.