The A5 Watling Street at Ivetsey Bank was closed as dozens of firefighters tackled the flames at the 18th century Wheaton Aston Old Hall in South Staffordshire.
Formerly known as Utopia, the swingers club was a popular adult venue for couples.
The venue had a 15-metre heated indoor swimming pool, 16-person jacuzzi and sauna and steam room.
Staffordshire Fire and Rescue crews were called to the scene at about 10.12pm last night.
The whole 98ft by 98ft (30m x 30m) building was completely involved in fire which also affected the roof.
Crews from Penkridge, Cannock and Codsall fire stations were among those in attendance.
Three main water jets and an aerial ladder platform was used by firefighters to stop the blaze from destroying the building completely.
Five men – a 55-year-old from Rugeley, a 25-year-old from Willenhall and three men from Wolverhampton, aged 20, 22 and 33 respectively – have been arrested on suspicion of arson and burglary.
They remain in police custody on Friday.
An investigation officer is still at the scene today to establish the cause of the fire.
Utopia dissolved in 2007 and the building has remained derelict ever since.