The sixth-tier side stunned the Vanarama National League outfit thanks to first-half goals from Amari Morgan-Smith and Ashley Hemmings at Aggborough.
“I’m really proud of this group of players,” said Penn, whose side also beat Grimsby in the first round.
“We want one of the big boys. That would be great.
“It would be amazing for the players and everybody involved with the football club.
“They deserve it. We were the underdogs here again and we keep proving people wrong.”
The hosts were two up inside 20 minutes and never looked back.
Former Luton and Oldham forward Morgan-Smith opened the scoring and provider Hemmings secured a dream third-round tie when he fired home the second 14 minutes later.
Halifax boss Pete Wild said: “I’m frustrated because it’s them wanting it more than us that’s won it for them.
“When I come out and say that that’s ridiculous.
“It’s two avoidable goals that we’ve conceded. I think the second goal’s offside, but that means nothing now.
“The game was done in the first half. The second half was a non-entity.
“We can’t be disappointed because we didn’t threaten their goalkeeper.
“Unfortunately the first 20 minutes lost us the game and that was it.
“We didn’t stand up to the test, it was a big game in the second round of the FA Cup and we didn’t stand up.
“It’s one of the worst first halves since I’ve been at the football club.”