The record store chain opened a giant new HMV Vault in Dale End last August.
It is now to shut its site in the Bullring shopping centre and has started a closing down sale.
No date has yet been announced for the closure.
The Bullring store replaced former sites in the city centre including one in New Street which served the city for many years.
HMV, which celebrates its centenary in 2021, was bought out of administration by Canadian businessman Doug Putman in February last year.