Woodside Library, in Stourbridge Road, Holly Hall, Dudley, is being taken down ready to be built again at the museum in Tipton Road.
The library which closed in 2008 will be the centrepiece of the attraction's new 1940s to 1960s Black Country town which will see 22 new buildings added to the site.
Other buildings include the old Stanton's Music Shop from Castle Street, Dudley, and a recreation of the Elephant & Castle pub, which once dominated the corner of Stafford Street and Cannock Road in Wolverhampton.
Black Country Living Museum chief executive, Andrew Lovett, said: "We are so proud to be able to give Woodside Library a new home as an integral part of our new capital development, BCLM: Forging Ahead.
"The project is now more important than ever to the future of Black Country Living Museum.
"The library will be set in the 1960s when it was an important community hub – a time when its well-known children’s library was to the fore and when the dances held on its first-floor room were some of the most notable occasions in Dudley’s social calendar.
"We’re aiming to depict a genuine slice of Black Country life from the period with this much-loved building."
The library opened in 1894 after the Earl of Dudley offered the land for the library and Woodside Park in 1890 - land which was previously part of the Earl’s large Woodside Colliery.
On the day of the opening ceremony, the Earl and Countess of Dudley were escorted by the Worcestershire Hussars with a procession by carriage from Dudley Town Hall, to Netherton, and on to Woodside.
Woodside Library featured a reading room and lending library on the ground floor, and recreation and retiring rooms on the first. The upstairs rooms were also used for a variety of local clubs and, most famously, dances led by local compere Horace Robinson.
After World War Two, the library service began to expand, with Woodside incorporating both a children’s library and gramophone record library. It was renovated in the early 1970s to brighten it up and continued in use until 2008.
Councillor Ian Kettle, Dudley Council's cabinet member elect for regeneration and enterprise, said: "There is fantastic regeneration work happening all over Dudley and the latest developments at the Black Country Living Museum are hugely exciting.
"Woodside is an important building with lots of history behind it and I am glad it is being kept in the borough for future generations to see."